Developer.php performance (+patch)
Hi,
developer.php is quite slow sometimes, especially on people with much
packages, you can notice delays, it even timeouts sometimes.
The main performance hog is a linear (O(n)) search in bugs.txt for every
row==package.
On my tests, a linear search in a 8000 line (number of source packages)
bugs.txt would take 160ms on average (for every package), while a db2
lookup, including opening and such, takes 0.09ms on average. The huge
timehit for the linear search is also parially because of two perl
regex's for every line.
cvs:qa/data/cronjobs/ddpo.bug is out of sync with reality unfortunately.
Anyway, after a new bugs.txt is generated, execute this php script
(should take about 200ms with 8000 packages, so isn't much):
<SCRIPT>
#!/usr/bin/php4
<?php
// Copyright 2004 Jeroen van Wolffelaar <jeroen@wolffelaar.nl>
// This script may be redistributed under the conditions of the GPL
// 17 feb 2004
unlink('bugs.db.new');
$db = dba_open('bugs.db.new', 'cwdt', 'db2')
or die("Opening bugs.db.new.db failed");
$fp = fopen("bugs.txt", "r");
while ($line = fgets($fp, 512)) {
list($package, $bugdata) = preg_split('/\\s+/', $line, 2);
$bugdata = trim($bugdata);
// optionally: do sanity checking here, so developer.php can
// rely on it. But my guess is that sanity checking is unneeded
dba_insert($package, $bugdata, $db)
or die("Insert for $package failed");
}
dba_optimize($db)
or die("Optimize failed");
dba_close($db);
move('bugs.db.new', 'bugs.db');
</SCRIPT>
and then modify developer.wml as such:
diff -u -r1.49 developer.wml
--- developer.wml 20 Nov 2003 14:09:21 -0000 1.49
+++ developer.wml 17 Feb 2004 14:24:07 -0000
@@ -410,18 +410,12 @@
$priority = $results[6];
$section = $results[7];
- $fp = fopen("$prefix/bugs.txt", "r");
- $bug_list = explode(" ", "0(0) 0(0) 0(0) 0(0)");
- $temp = preg_replace("/\+/", "_", $package);
- while(!feof($fp)) {
- $line = fgets($fp, "512");
- $line = preg_replace("/\+/", "_", $line);
- if(preg_match("/^$temp\s+(\d+\(\d+\)\s\d+\(\d+\)\s\d+\(\d+\)\s\d+\(\d+\))/", $line, $results)) {
- $bug_list = explode(" ", $results[1]);
- break;
- }
+ $db = dba_open('bugs.db', 'rd', 'db2');
+ if (!($bug_list = dba_fetch($package, $db))) {
+ $bug_list = explode(' ', '0(0) 0(0) 0(0) 0(0)');
}
- fclose($fp);
+ dba_close($db);
+ $bug_list = explode(' ', $buglist);
if(strcmp($version_stable, "-") == 0 || !debcheckavailable('stable', $package)) {
$stable_dep = "<small>-</small>";
Not tested of course, as I'm not able to, but this is a really
straightforward fix.
--Jeroen
--
Jeroen van Wolffelaar
Jeroen@wolffelaar.nl (also for Jabber & MSN; ICQ: 33944357)
http://Jeroen.A-Eskwadraat.nl
Reply to: