Preparing the next Release

[Martin Schulze <joey@infodrom.org>]

Hi,

in private Steve and I discussed some release issues, and we both
agreed that (a) our output is public and (b) it should happen in
public, so here it is dragged into the public.

Steve wrote:
> Broad categories of release-critical bugs that exist today:
> 
> - packages that FTBFS
> - security bugs
> - data loss bugs
> - packages that violate the DFSG as understood by the project today
> - bugs related to removal of obsolete libs
> - trivially broken maintainer scripts
> - packages that are badly put together to the point that the whole package,
>   or a major component of it, is unusable

Steve Langasek wrote:
> On Wed, Aug 30, 2006 at 07:44:27AM +0200, Martin Schulze wrote:
> > These bugs are at least to be investigated and maybe resolved in a
> > rather pragmatic way:
> 
> > > - packages that FTBFS
> 
> >   . on which architecture?
> 
> As long as it's a release architecture, does this matter?  Doesn't the
> security team still typically wait for a security update to be built on all
> archs before releasing it?

I'm not sure if it makes sense to reply to each of your paragraph,
since you're the one to decide anyway, not me.

However, I believe that such questions need to be raised and
investigated in time, and maybe some packages need to be dropped from
the release or from particular architectures.

With regards to security support, all packages that the security team
needs to support must to be compileable by the buildds for all
architectures they were released for.

When there is no cups for amd64 in the release, it does not matter
whether it FTBFS on amd64 or not, for example.

If a package FTBFS on one architecture, it (or rather an oder version)
must not be released with this architecture, though.  Being unable to
fix security bugs in such packages is one reason, potential license
violation by not shipping the proper source package would be another
reason not to do so.

While I would love to have the exact set of packages on all
architectures, I need to accept the fact that on some architectures
some packages just don't work or cause an FTBFS we are unable to fix
in time for the release.

> >   . how widely is it used?
> 
> Again, does this matter for questions of RCness?  (If you mean "is it ok to
> remove the package instead of fixing it", this is already a judgement call
> the release team makes.)

I believe that it does matter.  You're explanation in brackets imply
that you're already working as I suggested.

Fixing the problems should have a higher priority than removing a
package.  However, when either nobody cares about a fix or it is not
possible to fix the problem in time, a removal may be warranted.

> >   . how likely will we have to recompile it?
> 
> Do you have a formula for easily determining the probability of a security
> hole being discovered in any arbitrary package over a given period of time?
> If so, please share it!  Follow-up question, what is the threshold below
> which it's sufficiently unlikely for a security hole to be found that the
> security team volunteers to accept responsibility for the FTBFS bug as well
> in the event a security bug *is* found?

No, I don't have.  I could add a question about the possibility to run
remotely provided data by this package?  Whether the program runs
under the user id of the normal user or is privileged in any way.
It's very difficult to measure, I admit.

> > > - data loss bugs
> 
> >   . data loss always or only in some esoteric situations?
> 
> Do you think it should be ok for packages to lose users' data only /some/ of
> the time?  I don't, FWIW...

Well, we've already had packages that can potentially use data when
you use three quite uncommen switches, two undocumented commandline
switches, one broken config line and the system has three processors
with a permanent load of 53.

This is a bit exagerated, but I hope you'll get an idea of what I was
trying to express.  When the data los *can* happen in only certain
quite uncommon circumstances and the rest of the system and of the
package works fine, I'm not too sure we should delay the release to
get this problem fixed before.  Especially since the problem exists
for n months already without the hell being frozen.

Such problems usually warrant an inclusion in a point release, so can
be fixed after the release as well.

> >   . could we accept that this package has a bug and keep it buggy in etch?
> 
> What's the point of including it in a stable release if we know it'll eat
> user data?  Honestly, shouldn't we have more pride in our stable releases
> than that?

Please read above.  If it only happens or can happen in some very rare
situations, what's the point of delaying the release?  This could
still be fixed in a point release.

It's better not to release etch with such bugs, however, are they all
really worth being able to delay a release?

> > > - bugs related to removal of obsolete libs
> 
> >   . how widely is it used?
> >   . can the package be removed without hurting the overall system?
> >   . can the package be removed on the particular architecture?
> 
> Would you care to look at bug #370429 and render your own opinion on the
> impact of trying to remove it in advance of transitioning its
> reverse-depends?

Well, as a second step (after declaring it obsolet) it should be tried
to fix the dependencies and upload packages that don't depend on this
but the new version.

In this case this seems to be

  libsoup2.0-0               libsoup2.0-dev libsoup2.2-dev
  zapping
  ofx
  lynx
  libxmlsec1-gnutls          libxmlsec1-dev
  libggzcore7
  libggz2                    libggz-dev
  ggz-kde-games
  ggz-kde-client

                             libgnutls-dev
                             libgnutls-dev

Bug reports need to be written and maintainers be contacted.  I
remember that Smurf announced that he'd like to remove that library,
not sure about the other steps, though.

To mitigate the problem, at least ofx, lynx, ggz-kde-games and
probably ggz-kde-client could be temporarily removed from etch and
migrate back when the dependencies don't require TLS 11 anymore.  The
libraries would have to be investigated further.  (I'll do it if it
helps you.)

> > > - trivially broken maintainer scripts
> 
> >   since this is easy to solve, can the package be removed from etch
> >   and migrate back into it when it's fixed without hurting the system?
> 
> Generally.  But the question wasn't about how to resolve bugs of each of
> these classes, the question was whether it was a good idea to stop
> considering these bugs release-critical for the package.  Of course many of
> these packages can be removed from testing; that still takes release team
> time to accomplish.

Sure.  Sorry for misunderstanding you.

> > > and obsolete libs are priorities for the security team, and relaxing our
> > > stance on these issues would be bad for the overall security supportability
> > > of etch?  Do you think that any of the others should be ok to ship in a
> > > release?
> 
> > We may reach a date at which the release manages may have to sit down
> > and think about what to do with the remaining list of RC bugs.  One
> > way would be to reclassify and document them in the release notes and
> > to remove several packages from etch in order to get it in a
> > releasable state.
> 
> Removing RC-buggy packages from testing is an ongoing task.  I'm afraid that
> the nearly-constant bug count in testing factors this work in.

Ok.  (merely quoted to get it copied in public).

> Documenting bugs of this severity is IMHO a poor substitute for resolving
> them, either by actually fixing the bug or by removing the package.  That
> might be the decision for new RC bugs that show up in the last stages of the
> freeze, but I don't really think that helps us get to the point that we're
> /ready/ to freeze.

It's always better if these problems would be corrected.  However,
when this is not possible in time for whatever reason, it may be worth
to consider another means of dealing with them.

Speaking of being ready for a freeze, I would consider this date
generally reached already.  Most of the packages are in a considerably
fine state.  There are still problems, though, but there will always
be problems anyway.  When the installer is working fine on all release
architectures and the kernel and udebs have been migrated into testing
as well, we already have a *pretty* *good* system.

> > > As for ignoring RC bugs for etch, I don't expect that to be true for many
> > > bugs.  A larger number of bugs on the RC bug list simply don't meet the
> > > criteria for RC bugs and should be downgraded -- that's not anything that
> > > requires the release team's imprimatur to fix up, either, but I still find
> > > myself fixing up quite a few bug states every day, too.
> 
> > Hmm, maybe it's not clear to the average developer that they are
> > permitted to upgrade/downgrade arbitrary bugs as well, especially if
> > they're not assigned to a package they maintain.
> 
> I think the BTS documentation does say you shouldn't fiddle with bugs
> (closing them, etc) on packages you don't maintain.  But then, it doesn't
> exempt the release team either. ;)

Evil release guys. :P

Regards,

	Joey

-- 
Have you ever noticed that "General Public Licence" contains the word "Pub"?