--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: popularity-contest: Improve performance (4x faster)
- From: Benoît <bdejean@gmail.com>
- Date: Thu, 27 Dec 2018 23:19:45 +0100
- Message-id: <154594918572.7184.11507870222942796814.reportbug@powerbook>
Package: popularity-contest
Version: 1.67
Severity: minor
Tags: patch
Dear Maintainer,
For each installed packages, popcon globs the complete list of files
in /var/lib/dpkg/info.
This is very slow as I noticed that popcon takes more than a minute of CPU
time on my modest laptop, which is enough to start the fan.
I'm attaching a patch that lists only once /var/lib/dpkg/info and associates
each .list file with a package.
I don't see any difference in /usr/sbin/popularity-contest output.
And the CPU time goes from 1min08s to 0min14s.
-- System Information:
Debian Release: buster/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'unstable'), (500, 'testing'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 4.17.0-3-amd64 (SMP w/2 CPU cores)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages popularity-contest depends on:
ii debconf [debconf-2.0] 1.5.69
ii dpkg 1.19.2
Versions of packages popularity-contest recommends:
ii cron [cron-daemon] 3.0pl1-130
ii exim4-daemon-light [mail-transport-agent] 4.91-8+b1
ii gnupg 2.2.10-3
Versions of packages popularity-contest suggests:
ii anacron 2.3-27
pn tor <none>
pn torsocks <none>
-- debconf-show failed
--- /usr/sbin/popularity-contest 2018-08-09 20:41:19.000000000 +0200
+++ popularity-contest 2018-12-27 22:51:27.710362640 +0100
@@ -119,6 +119,19 @@
close DIVERSIONS;
}
+my %pkgs_files = ();
+
+if (opendir(my $DPKG_DB, $dpkg_db))
+{
+ for my $e (readdir($DPKG_DB)) {
+ if ($e =~ m/^([^:]+) .*? \. list$/x) {
+ $pkgs_files{$1} ||= [];
+ push @{$pkgs_files{$1}}, "$dpkg_db/$e";
+ }
+ }
+ closedir($DPKG_DB);
+}
+
# Read dpkg database of installed packages
open PACKAGES, "dpkg-query --show --showformat='\${status} \${package}\\n'|";
while (<PACKAGES>)
@@ -128,7 +141,7 @@
my $bestatime = undef;
my $list;
$popcon{$pkg}=[0,0,$pkg,"<NOFILES>"];
- foreach ("$dpkg_db/$pkg.list", glob("$dpkg_db/$pkg:*.list"))
+ foreach (@{$pkgs_files{$pkg}})
{
open FILES, $_ or next;
while (<FILES>)
--- End Message ---
--- Begin Message ---
Source: popularity-contest
Source-Version: 1.68
We believe that the bug you reported is fixed in the latest version of
popularity-contest, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 917478@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Bill Allombert <ballombe@debian.org> (supplier of updated popularity-contest package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Sun, 07 Jul 2019 15:30:07 +0200
Source: popularity-contest
Binary: popularity-contest
Architecture: source all
Version: 1.68
Distribution: unstable
Urgency: low
Maintainer: Popularity Contest Developers <debian-popcon@lists.debian.org>
Changed-By: Bill Allombert <ballombe@debian.org>
Description:
popularity-contest - Vote for your favourite packages automatically
Closes: 917478
Changes:
popularity-contest (1.68) unstable; urgency=low
.
* Please note when reading stats that Buster was released with 1.67.
* Apply patch from Benoît to speed up popcon. Closes: #917478.
* debian/control:
- Updated Standards-Version from 4.2.0 to 4.4.0. No change needed.
Checksums-Sha1:
b2526681c37284153d0ca7d5f27ce6b9ccd0d4bd 1724 popularity-contest_1.68.dsc
7c41964e74f51320057d05c05a0afcc155a279ec 79540 popularity-contest_1.68.tar.xz
8866f4286c32d920c38c49d18ace738f43e54869 70100 popularity-contest_1.68_all.deb
490e259dce325744c2971a468d737139f15f0c4b 5461 popularity-contest_1.68_amd64.buildinfo
Checksums-Sha256:
a5c5b6b460822325ff7bc31fc79a5cb88df80c25cced51c5e49b9177edeafd4d 1724 popularity-contest_1.68.dsc
e8a9e5d8432f94b7c751b3d1b4b150ea9d97c3eb2d8dd4a646424ad82384a7db 79540 popularity-contest_1.68.tar.xz
946d1c9db4c7528cd63e6d47fed422c7299310084bd599001fcc76ff5ad6e097 70100 popularity-contest_1.68_all.deb
c18ff3ac0f57cfc90f12f2a12871609537d9c0fb1b262c52c20eb2ccde8acbbc 5461 popularity-contest_1.68_amd64.buildinfo
Files:
febec692611a4ab360570746c6e2a4c1 1724 misc optional popularity-contest_1.68.dsc
941d24e3e67305c6292a2b53433b69c3 79540 misc optional popularity-contest_1.68.tar.xz
6f1302852cd759ddf0a142b611e3a237 70100 misc optional popularity-contest_1.68_all.deb
cc5e9b8c836d37eff6d66f45c11903e8 5461 misc optional popularity-contest_1.68_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEQgKOpASi6dgKxFMUjw58K0Ui44cFAl0jK5sACgkQjw58K0Ui
44cBKhAAo8FtpkFRVePvG2wv7JeqdbkjkFDAZDkbff4yVSdBkomCgGvLX27Xtjs7
0N5oA0hDD4ToShHy4f8Cw+dhwAEcyZ24bNiCms5Y7npnzuSzb6hQvOjzKxvdeTmO
2i/ykU4xn2/WXl0rwJDAskPRaRMLmbeKU5JxFKJMT21dXJWHp/O8l9j4zxdwPQYH
rVJMmT1AhKcEHe3f0dKuFoa4GxQbbRONow9h42OTf7i0nUZZJLI6fyyd1RjHVEPD
9xCRAukbfZ2s4ijdZcnXg76LZPVtUEbZIwML7WATJLaSaiCcGPjJYN6sxL3orjPO
kvwlc4n+UEP8+5WPycdSUrG5L6vY5QdCDfButm3cuMhNfkEMZvMNjSJwZ8KXafiv
63OhG6aWC1x1iB1SJgCGRC/6Lps5t6otUWvW79Ht91wtoytaD4SksRpR3IQdg2cf
8b9/L4Czqyw8+u3EaPPRfTofoJIZLEFNgrXV939Aw5J6hGLutiWYwudtTClJCymQ
BfWso3JIZKZsXf6K3hVro8m2sw3wS1iUxYh4JWx1CTf2Xpi5cTgUCb+STjkvFyf3
k/2GMJSm7gxtEtRVkG4tgzGY/zZqHsnFDsrhTn9Rdcs3hODFi0cohGtDfE0jtAbL
uWRSKR0x8pP30WeDEGQSFzgCeaiq87pWjsegbPqrV5eXAasM/9Q=
=HmHz
-----END PGP SIGNATURE-----
--- End Message ---