Re: The sarge release disaster - some thoughts
On Mon, Apr 04, 2005 at 05:19:41PM +0200, Martin Schulze wrote:
> Adrian Bunk wrote:
> > The milestone that included the start of the official security support
> > for sarge was only 6 days after the announcement, but is was missed by
> > more than 6 months.
> >
> > Whyever it was expected to get testing-security for sarge that quick, it
> > should have been obvious 6 days later that it wasn't possible that
> > quick.
> >
> > What would have been a second plan?
> > Use testing-proposed-updates.
> >
> > Using testing-proposed-updates for security fixes, users might have
> > gotten security updates one or two days after the DSA on some
> > architectures.
> >
> > Would this have been an ideal solution?
> > No.
> > But it would have worked without a great impact on the release date.
>
> No, it wouldn't, since t-p-u isn't autobuilt for all architectures
> either. We would win nothing by using it without manually building
> the packages on the missing architectures.
Please correct me if my understanding was wrong.
As far as I understood it, the situation was:
t-p-u lacks autobuilders on some architectures.
s-p-u required a non-trivial amount of changes in Debian-internal
software.
My point is:
It turned out that the work to get s-p-u running requires more than half
a year.
If this was the _the_ show-stopper for the release, I can't believe it
could take more than a few weeks for getting running autobuilders for
architectures lacking autobuilders for getting t-p-u running.
Then you can work around the missing s-p-u by using t-p-u.
This way, you'd have reduced a delay by half a year to a delay by a few
weeks.
> > RC bugs - only a metric
> > -----------------------
> >
> > Nowadays, it seems the main metric to measure the quality of a release
> > inside Debian is the RC bug count.
> >
> > As with any metrics, work on improving the metric might make the metric
> > look much better, but doesn't at the same time imply that the overall
> > quality improved that much.
> >
> >
> > An example:
> >
> > A major problem in Debian are MIA developers.
> >
> > Consider a MUA maintained by a MIA developer with the following bugs:
> > - #1 missing build dependency (RC)
> > - #2 MUA segfaults twice a day (not RC)
>
> These are fixed during BSPs, so no problem to spend more time on.
Unfortunately not.
BSPs tend to focus on RC bugs and the success is often measured only in
the RC bugs metric.
Quoting what I already said in a different email in this thread:
The first example that comes into my mind is the info package where the
two NMUs didn't fix the many open segfault bugs (e.g. #259561 contained
a trivial patch ACK'ed by upstream being one and a half months in the
BTS before the latest NMU - note that this NMU was even done by a
Release Assistant).
This shouldn't be a personal offense against the person who did this
particular NMU - it's simply the first example coming into my mind.
>...
> > Dump testing?
> > -------------
> >
> > It seems noone asks the following question:
> > Testing - is it worth it?
>
> As a preparation for stable and an interim "solution" between stable
> and unstable it's quite well.
>
> > Several people have stated that with the size of Debian today, it
> > wasn't possible to manage a release without testing with a "traditional"
> > freeze (unstable will be frozen at a date, announced several months
> > before), and that only testing makes releasing possible.
>
> I believe that it helps a lot to get and keep the software in proper
> shape for a release with all supported architectures and depending
> packages.
Testing has it's advantages.
There's always some manual work required for keeping testing running.
And the complete release team signed the announcement stating that the
testing release process is not capable of release cycles in the order
of 12-18 months with 11 architectures.
Are the advantages of the testing release process worth both the extra
work and dropping two thirds of the Debian architectures from releases?
>...
> > testing was expected to make shorter freezes possible.
> > Neither the woody nor the sarge freeze support this claim.
> > This might not only be the fault of testing, but the positive effects of
> > testing (if any) aren't visible.
>
> Hmm, we're not waiting for testing to freeze but we're waiting for
> missing infrastructure to be implemented. Or am I mistaken?
I don't know more than you.
My point is:
If there are areas where the testing release process has advantages over
the previous release process, they weren't visible during the two
releases that used the testing release process.
> > This might make a freeze a bit longer?
> > Perhaps.
> > But consider the disadvantages of testing:
> > - Testing causes additional work for both the release team and all
> > Debian developers.
> > As an example, library transitions are always a pain due to testing.
>
> Uh? For the user, they're a lot better since they will happen instantly,
> something that usually doesn't work with unstable. Yes, they do cause
> work for the release people and some maintainers are annoyed by the
> delay to get all affected packages and architectures in sync.
For which users is testing really usable?
Until recently there was no security support, and even I have yet to see
the first DSAs for testing (otherwise you'd force all users of testing
to daily upgrade their computers for getting security fixes).
Data loss bugs like #220983 are possible every day.
And it's a moving target, so if you upgraded computer A today and
computer B tomorrow they are installed differently and might have
software with different bugs.
Most people I know who run testing use it because stable is too
outdated.
It's true that Debian stable is too old and that Debian 3.1 will be the
second release after another where the X11 is already seriously outdated
at time of the release (with the usual problems with hardware support).
But these are release management problems and using testing is only a
workaround for these problems.
> > And RC bugs already fixed in unstable but not in testing need to be
> > tracked.
>
> When testing ist not partially frozen and the package runs fine in the
> archive, it'll migrate into testing automatically. Tracking is only
> required during a (partial) freeze.
Agreed.
But it's still a serious amount of work:
The sarge freeze was announced to start pretty soon, and there are
currently at about thousand packages in testing with a more recent
version in unstable.
Some of them might have RC issues fixed several months ago and some of
them might have never been reported in the BTS.
> > - Architectures have to be in sync due to testing.
>
> ... due to an upcoming release.
Or due to a library transition (that might be many months before the
start of the next freeze).
>...
> Regards,
>
> Joey
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
Reply to: