[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: bug rates

Joey Hess wrote:

> Steve Langasek wrote (elsewhere):
>> something changes in the archive (often for a transition that needs
>> to happen), a large number of packages are broken by this, and some
>> appreciable number of the maintainers don't respond in a reasonable
>> amount of time ("reasonable" defined as "sufficient to make headway
>> against the RC bug count"; average time to bugfix << average interval
>> between archive-breaking transitions).
> I wonder if you have any numbers on this? My gut feeling for a while has
> been that part of what's keeping the RC bug levels in balance is that
> we have a constant stream of RC bugs being filed for issues that are not
> new but have only turned up as things get tested.


> A certian percentage 
> of these affect stable too. One obvious example is security bugs. It
> seems to me that if you look at the RC bug graph, most of the sharp
> spikes and dips are due to the kind of transitional RC bugs you're
> talking about, which says that we do pretty well at dealing with those,
> at least for significant transitions that cause a lot of similar bugs.
> One of my problems with our current release process is that it doesn't
> seem to take into account the bugs that exist in stable. By the current
> metrics we could easily be in a position today where testing has many
> fewer RC bugs than stable, but still be far from a release. And even if
> we get the RC bugs to zero and make a release, that release will have
> many undiscovered RC bugs which will turn up later.


> That makes it hard 
> for me to worry about bringing the count down to zero, since it's really
> just pushing the iceberg underwater for an instant.

Yeah, I think you have a good point here.

Personally, I would prioritize RC bugs which apply to "key" packages
 -- the benefit from getting those RC-bug-free is
larger than that of getting other packages RC-bug-free.  Unfortunately,
some vital packages like the toolchain packages and the kernel have some of
the oldest and most irritating RC bugs.  Licensing....

> I wonder if it would be worthwhile to try to quantify this some more,
> by looking at all the RC bugs over a period of time and determining:
> a. What percentage were caused by issues in new uploads.

Weirdly, these are often the quickest to be fixed.  Exceptions: Bugs
caused by large upstream changes; bugs in packages with inactive
or inattentive maintainers.

> b. What percentage by transitions.

Gah, probably most of them.  Time-to-fix is weird for transitions; most
packages are fixed very quickly, and then there's a long tail, primarily
caused by inactive maintainers.  Amaya's been working on finishing up
some of those long tails, as have I.  Mostly they don't get finished.

> c. What percentage also affect stable.

I try to versiontag for this.  This should be a detectable quantity now, 
thanks to versiontagging.  These, oddly (or perhaps not so oddly) are 
often the ones which take the longest to fix.  However, they're also where
I put my priorities, because long-standing bugs are substantially more
annoying than new bugs.

A release where we fixed all the (discovered and undiscovered) RC bugs from
the *previous* release would be a very successful release.  :-)

> and finding the relative time-to-fix of each of these.

Nathanael Nerode  <neroden@fastmail.fm>

Bush admitted to violating FISA and said he was proud of it.
So why isn't he in prison yet?...

Reply to: