[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

architecture-specific release criteria - requalification needed



Hi all,

It has been discussed for a while already.  While we have release
criteria for packages, up until now we don't have any for the
architectures.  However, decisions made about an architecture affect
both our users (and developers) and the release cycle much more than
decisions about an individual package.

For that reason, we discussed in multiple meetings, together with porters,
ftp-masters and other people more than once how the criteria should
look.  Also, there was more than one discussion on debian-devel. [1, 2]

In essence, the requirements that are being established exist to ensure
that the port is in good enough shape and sufficiently well-supported
that:
* our users will benefit from the architecture's presence in a
  release,
* the architecture will give our users the same support and
  stability as any other architecture in the stable release,
* the architecture's inclusion doesn't negatively impact other
  architectures or the release process as a whole.

These criteria do _not_ control addition of an architecture to unstable,
but rather apply to architectures which the ftp-masters have accepted
into unstable and are targetting testing and the next stable release.
In other words, an architecture that fails these criteria can still be
part of unstable.  And, of course as always, the release team can grant
exceptions - which is of course an exception, and we need to be
convinced that this is the right action.

Over the next two months, all of the current release architectures will
need to requalify for inclusion in etch according to the following
criteria.  See the bottom of this mail for details.

Now, looking more into details, the criteria are:

|  * Availability:
|     The architecture needs to be available for everybody, i.e.
The reason for this should be obvious

|     it must be available without NDAs and
Same for this. We're about free software.

|     it must be possible to buy machines on the market.
The reason should be obvious: Our users should be able to use the
architecture.

Of course, it is known that there will be some timespan where a product
line is end-of-lifed by its hardware manufacturer, but still useful to
users (and still possible to buy the product).  So, we also don't directly
drop the architecture on the floor, but on the other hand, there will be a
day where the port's porters and Debian cannot longer give the port that
much support that we can do stable releases with it - and even a day when
we cannot longer support that port at all.

|  * Developer availability: The architecture must have a
|    developer-available (i.e. debian.org) machine that contains the
|    usual development chroots (at least stable, testing, unstable).

This criterion is there so that any developer can actually find out what the
issue is if his package fails to work on a specific architecture.  Of
course, when adding a new architecture, there will be a time without a
stable release, and there will be some special arrangement how such a
machine can be provided without having even some packages in testing.  But
that's not meant as a no-go, as long as we are quite optimistic that adding
the new machine will actually work in time.


|  * Users: The architecture needs to prove that developers and users
|    are actually using it. Five Developers needs to certify in that
|    they're actively developing on this architecture, and it needs to
|    be demonstrated that at least 50 users are using the platform. We
|    are counting users, not machines; e.g., one s390-installation
|    with 50,000 users fullfils the user criterion just fine.

As already discussed multiple times, the "50 users" really means "50
individuals using that architecture".  Both criteria are there to make
sure that an architecture gets just enough usage so that
architecture-specific bugs are found in time.


|  * Installer: The architecture must have a working, tested installer.

Obviously, we need an installer. Though that doesn't say "debian
installer", we think that our users expect that there are not too many
different ways for them to install the released version of Debian etch one
day.


|  * Porters and Upstream support: There is support by the porters and
|    upstream. This is especially true for the toolchain and the
|    kernel.

Obviously, we cannot keep a port alive if there is nobody doing support for
it.  Of course, it is quite possible that Debian and upstream support is
done by the same persons.  And our experiences with support of gcc-4.0
on m68k have shown that it is possible to get such issues fixed, if the
porters are notified in time and are really interested in their port (and
if there are enough porters).


|  * Archive coverage: The architecture needs to have successfully
|    compiled the current version of the overwhelming part of the
|    archive, excluding architecture-specific packages.

Our back-of-the-envelope number for this criterion is 98%.  As pointed
out multiple times during recent discussions, we don't have a good way
to measure an architecture's compliance with this yet, but we'll work on
figuring that out; of course we will exclude hardware-specific packages and
buggy optional/extra packages with severe portability issues, but
porters must take responsibility for working with maintainers to fix
portability issues.



|  * Archive cleanliness: All binary packages need to be built from
|    unmodified sources (i.e. from the source found in the
|    ftp archive), and all binary packages need to be built by Debian
|    developers.

That's already part of the normal rules. Just repeated for clearness
purposes.


|  * Autobuilder support: 
|    The architecture is able to keep up with unstable

This is obviously needed. If the architecture cannot keep up, there is no
way to support it in a stable release.

|    with not more than two buildds,

That is one of the most discussed criteria. As mentioned previously [2],
there is a nontrivial cost to each buildd, which increases super-linearly;
there have been cases in the past where this resulted in ports with many
autobuilders slacking when updates were necessary.

When reviewing the past however, m68k as the architecture with the most
autobuilders isn't performing too bad regarding the availability of the
autobuilders.  So, there is the chance for m68k to get grandfathered in
for this clause.  However, we expect that they explain why the higher
numbers of buildds they use are not as bad increasing the maintenance
overhead.


|    has redundancy in the autobuilder network,

This is the "it needs to have N+1 buildds" - just in case some buildd has
hardware failures or whatever else.  History told the release team that
redundancy is really necessary.  No one in the release team wants to be 
in the position of tracking where a box in Europe is just located, and
proding some developer in that country to pick the box up, because that
box has become the largest blocker of the next stable release.


|    keeps their autobuilders running for 24x7,

Of course, autobuilders can have hardware maintainence.  But the
autobuilders need to be able to run 24x7, and the need to be generally
up all the time (and thanks to the redundancy above, there should always
be an autobuilder currently running).


|    has autobuilders acceptable for security support.

If we want to do security support, that of course needs to be there.


|  * Veto powers: Security team, system administrators and release team
|    must not veto inclusion.

This is also a much discussed criterion.  But - though it is never intended
to be used, there is basically no way how we can make it without it.
Please see the previous mail [2] for further reasonings.  Of course, before
the veto power is used, the relevant team needs to send a mail to the
developers at large and to the porters and explain what concerns they
raise, and only if the concerns are not addressed appropriate after
reasonable time, the veto power can be used.




So, of course the question is:  Which of the current architectures do
fulfill this set of requirements?  To get this answer, and because we
know there are currently architectures which do *not* meet the
requirements, all architectures will need to be requalified.

We added an overview page about the release criteria on
http://release.debian.org/etch_arch_criteria.html and on the
requalifying on http://release.debian.org/etch_arch_qualify.html.

Porters, please feel free to prove us compliance of your architecture with
the remaining issues (that are all as of today :) - or rather, please start
on this, as this needs to be done soon.  We will follow up when we made some
progress, so that you all see what parts and architectures are more
problematic before any decision is done.  We hope to finish this in the
next two months.

And of course, as always: If there are any issues that you think aren't
addressed properly, please feel free to contact
debian-release@lists.debian.org - but remember: that address is not a
discussion list, so if you rather want to discuss, select a more
appropriate place for that.


Cheers,
-- 
Andi Barth
Debian Release Team
http://release.debian.org/


[1] <20050314044505.GA5157@mauritius.dodds.net>
[2] <20050821015824.GT16930@country.grep.be>

Attachment: signature.asc
Description: Digital signature


Reply to: