Re: bug rates
Joey Hess wrote:
> Steve Langasek wrote (elsewhere):
>> something changes in the archive (often for a transition that needs
>> to happen), a large number of packages are broken by this, and some
>> appreciable number of the maintainers don't respond in a reasonable
>> amount of time ("reasonable" defined as "sufficient to make headway
>> against the RC bug count"; average time to bugfix << average interval
>> between archive-breaking transitions).
>
> I wonder if you have any numbers on this? My gut feeling for a while has
> been that part of what's keeping the RC bug levels in balance is that
> we have a constant stream of RC bugs being filed for issues that are not
> new but have only turned up as things get tested.
Yep.
> A certian percentage
> of these affect stable too. One obvious example is security bugs. It
> seems to me that if you look at the RC bug graph, most of the sharp
> spikes and dips are due to the kind of transitional RC bugs you're
> talking about, which says that we do pretty well at dealing with those,
> at least for significant transitions that cause a lot of similar bugs.
>
> One of my problems with our current release process is that it doesn't
> seem to take into account the bugs that exist in stable. By the current
> metrics we could easily be in a position today where testing has many
> fewer RC bugs than stable, but still be far from a release. And even if
> we get the RC bugs to zero and make a release, that release will have
> many undiscovered RC bugs which will turn up later.
Yep.
> That makes it hard
> for me to worry about bringing the count down to zero, since it's really
> just pushing the iceberg underwater for an instant.
Yeah, I think you have a good point here.
Personally, I would prioritize RC bugs which apply to "key" packages
-- the benefit from getting those RC-bug-free is
larger than that of getting other packages RC-bug-free. Unfortunately,
some vital packages like the toolchain packages and the kernel have some of
the oldest and most irritating RC bugs. Licensing....
> I wonder if it would be worthwhile to try to quantify this some more,
> by looking at all the RC bugs over a period of time and determining:
>
> a. What percentage were caused by issues in new uploads.
Weirdly, these are often the quickest to be fixed. Exceptions: Bugs
caused by large upstream changes; bugs in packages with inactive
or inattentive maintainers.
> b. What percentage by transitions.
Gah, probably most of them. Time-to-fix is weird for transitions; most
packages are fixed very quickly, and then there's a long tail, primarily
caused by inactive maintainers. Amaya's been working on finishing up
some of those long tails, as have I. Mostly they don't get finished.
> c. What percentage also affect stable.
I try to versiontag for this. This should be a detectable quantity now,
thanks to versiontagging. These, oddly (or perhaps not so oddly) are
often the ones which take the longest to fix. However, they're also where
I put my priorities, because long-standing bugs are substantially more
annoying than new bugs.
A release where we fixed all the (discovered and undiscovered) RC bugs from
the *previous* release would be a very successful release. :-)
> and finding the relative time-to-fix of each of these.
--
Nathanael Nerode <neroden@fastmail.fm>
Bush admitted to violating FISA and said he was proud of it.
So why isn't he in prison yet?...
Reply to: