[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#838514: marked as done (britney: patch to speed up loop performance)



Your message dated Thu, 27 Oct 2016 06:04:00 +0000
with message-id <bdb38b5a-1c5d-82ee-1d4b-4d63cd5b2371@thykier.net>
and subject line Re: ITM: Britney2 branch britney-patch-bundle-2016-09 into master
has caused the Debian Bug report #838514,
regarding britney: patch to speed up loop performance
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
838514: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=838514
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message ---
Package: release.debian.org
Severity: normal
User: release.debian.org@packages.debian.org
Usertags: britney, patch

Hi,

In Britney.write_excuses, there's a number of for loops doing slow operations redundantly. I've created a small patch that optimizes those loops with a good performance improvement. Here's some python profiling that proves the concept:

http://paste.ubuntu.com/23212621/

-- System Information:
Debian Release: stretch/sid
  APT prefers xenial-updates
  APT policy: (500, 'xenial-updates'), (500, 'xenial-security'), (500, 'xenial'), (100, 'xenial-backports')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.4.0-36-generic (SMP w/8 CPU cores)
Locale: LANG=en_CA.UTF-8, LC_CTYPE=en_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
>From daf514d526d47646bb15f5b0a91cd9b8abe69820 Mon Sep 17 00:00:00 2001
From: Robert Bruce Park <robert.park@canonical.com>
Date: Tue, 20 Sep 2016 14:03:02 -0700
Subject: [PATCH] Python loop performance enhancements.

---
 britney.py | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/britney.py b/britney.py
index b28c62f..79ad49c 100755
--- a/britney.py
+++ b/britney.py
@@ -1751,59 +1751,63 @@ class Britney(object):
         should_upgrade_srcarch = self.should_upgrade_srcarch
         should_upgrade_src = self.should_upgrade_src
 
+        unstable = sources['unstable']
+        testing = sources['testing']
+
         # this list will contain the packages which are valid candidates;
         # if a package is going to be removed, it will have a "-" prefix
         upgrade_me = []
+        upgrade_me_append = upgrade_me.append  # Every . in a loop slows it down
 
         excuses = self.excuses = {}
 
         # for every source package in testing, check if it should be removed
-        for pkg in sources['testing']:
+        for pkg in testing:
             if should_remove_source(pkg):
-                upgrade_me.append("-" + pkg)
+                upgrade_me_append("-" + pkg)
 
         # for every source package in unstable check if it should be upgraded
-        for pkg in sources['unstable']:
-            if sources['unstable'][pkg][FAKESRC]: continue
+        for pkg in unstable:
+            if unstable[pkg][FAKESRC]: continue
             # if the source package is already present in testing,
             # check if it should be upgraded for every binary package
-            if pkg in sources['testing'] and not sources['testing'][pkg][FAKESRC]:
+            if pkg in testing and not testing[pkg][FAKESRC]:
                 for arch in architectures:
                     if should_upgrade_srcarch(pkg, arch, 'unstable'):
-                        upgrade_me.append("%s/%s" % (pkg, arch))
+                        upgrade_me_append("%s/%s" % (pkg, arch))
 
             # check if the source package should be upgraded
             if should_upgrade_src(pkg, 'unstable'):
-                upgrade_me.append(pkg)
+                upgrade_me_append(pkg)
 
         # for every source package in *-proposed-updates, check if it should be upgraded
         for suite in ['pu', 'tpu']:
             for pkg in sources[suite]:
                 # if the source package is already present in testing,
                 # check if it should be upgraded for every binary package
-                if pkg in sources['testing']:
+                if pkg in testing:
                     for arch in architectures:
                         if should_upgrade_srcarch(pkg, arch, suite):
-                            upgrade_me.append("%s/%s_%s" % (pkg, arch, suite))
+                            upgrade_me_append("%s/%s_%s" % (pkg, arch, suite))
 
                 # check if the source package should be upgraded
                 if should_upgrade_src(pkg, suite):
-                    upgrade_me.append("%s_%s" % (pkg, suite))
+                    upgrade_me_append("%s_%s" % (pkg, suite))
 
         # process the `remove' hints, if the given package is not yet in upgrade_me
         for hint in self.hints['remove']:
             src = hint.package
             if src in upgrade_me: continue
             if ("-"+src) in upgrade_me: continue
-            if src not in sources['testing']: continue
+            if src not in testing: continue
 
             # check if the version specified in the hint is the same as the considered package
-            tsrcv = sources['testing'][src][VERSION]
+            tsrcv = testing[src][VERSION]
             if tsrcv != hint.version:
                 continue
 
             # add the removal of the package to upgrade_me and build a new excuse
-            upgrade_me.append("-%s" % (src))
+            upgrade_me_append("-%s" % (src))
             excuse = Excuse("-%s" % (src))
             excuse.set_vers(tsrcv, None)
             excuse.addhtml("Removal request by %s" % (hint.user))
-- 
2.7.4


--- End Message ---
--- Begin Message ---
On Sun, 25 Sep 2016 16:47:00 +0000 Niels Thykier <niels@thykier.net> wrote:
> Hi,
> 
> I intend to merge the branch [britney-patch-bundle-2016-09] consisting
> of 22 patches into master no later than Monday the 3th of October.  It
> has been tested with [no regressions].
>  * CC: People known to run Britney
>  * BCC: contributors + the debian bug
> 
> [...]
> Bugs fixed/Merges:
>  * #838514 (patch to speed up loop performance)
>  * GH#12 (Handle missing arches in p-u gracefully)
>  * GH#13 (Use better exception handling)
> 
> [...]
> 
>   Robert Bruce Park (1):
>       Python loop performance enhancements.
> 
> 
> Thanks,

This was merged a while back, closing the bug.

Thanks,
~Niels

--- End Message ---

Reply to: