Measuring Debian progress
In various web log entries (see http://planet.debian.org), Marc 'HE'
Brockschmidt, Joey Hess, Marc 'HE' Brockschmidt, Eduard Bloch, Bernhard
R. Link, and possibly others, discuss the significance of having lots of
bugs open against a package.
This has reminded me that during Debconf6, I meant to propose that we
set up some carefully selected statistics to measure the progress of
Debian development. The statistics needs to be selected very carefully
because there is a possibility that people will optimize their behavior
according to the statistics, and we'd want them to optimize in the right
way.
Here's what I'd like to propose: we publish, once per week or so, the
following numbers, on debian-devel or so:
* Total number of non-pseudo-packages that bugs.debian.org knows about.
* Number of open bugs, in total, and for each severity, and a count of
all non-wishlist, non-wontfix bugs.
* Number of bugs opened or closed since the previous report.
* Median and maximum times for first reply to a non-wishlist bug.
* Median and maximum times since latest reply to a non-wishlist bug.
* Median and maximum age of currently open bugs.
* Median and maximum duration from opening to closing of closed bugs
(total, and since previous report).
* Number of bugs younger than a month, younger than a year, or older
than a year.
* Number of packages with 0 bugs, 1-5 bugs, 6-20 bugs, 21-100 bugs, more
than 100 bugs.
* Minimum, median, and maximum "Bogonic Quality Numbers".
With the exception of the last one, I think it would be helpful in
guiding ourselves to improving some aspects of the development of Debian
in the right direction.
The above would be published as an aggregate for all packages. In
addition, it would be great for the maintainers of a package to find out
how they compare to the aggregate statistics, but without publishing
those. This way, there is no embarrassment or annoyance factor and
people can use the statistics for their benefit, or ignore them, as
suits them best. On the whole, we should be able to rely on the
self-motivation of our package maintainers to do their job as well as
they can.
It would be unfortunate if the statistics were to be used for badgering
maintainers: they should be a measurement of how the project as a whole
is doing. The motivation for each number chosen for the report may be
wrong for many individual packages, yet be good for the project as a
whole. Therefore, judging individual packages (or maintainers) based on
these numbers is unwise.
The "Bogonic Quality Number" or B is calculated by the following
formula:
n = number of open non-wishlist, non-wontfix bugs
s = size of source package (.dsc, etc) in units of 1518 bytes
i = number of installations according to popcon
B = n / (s*i)
The lower the B number, the higher the quality of the package.
(It is, of course, completely bogus, but might introduce some fun to
otherwise boring statistics.)
--
Debian is a beast that speaks with many voices -- Richard Braakman
Reply to: