A new arch support proposal, hopefully consensual (?)
Hello,
This is an attempt to do a vancouver-counter proposal in such a way that would
be acceptable to all, including the folk who was at the vancouver meeting.
Please be resonable when we post here, refrain from agressive behavior, and
provide argumentation to your proposed solutions.
We have three separate issues under discussion :
1) The mirror network, and bandwidth/mirror space considerations.
=> this one has been amply discussed, and the solution is not really
controversial.
2) the includsion of minority arches in the stable release and testing
infrastructure.
3) stable security updates.
I will not extend to point 1), as i think it is resonable to not mirrror
everything all over for a very limited usage. The main point under discussion
will be that we should encourage a more smart mirroring technic, which would
allow per-mirror selection without duplicating arch:all packages.
Maybe we can propose three mirroring ways : rsync of tier1 arches, rsync of
tier2 arches, smart-per-arch mirroring. Mirror operator would be free to chose
any of the above solutions, and everyone would be happy.
The main problem is in 2) and 3), and especially 2), as i didn't really see
any of the security team discussing issues about it here.
I also want to say that this proposal will take as pre-requisite the following
things :
a) we will try to support as many arch as possible, as long as the porter
team for said arch is ready to make the effort to make it possible. Debian
without arch support is not really debian as we all know it, so let's try to
stay ourselves.
b) the tier1-team (release/ftp-master team) should not place an unwaranted
burden on the porters, but should not themselves be over-burdened by the
port effort.
c) as long as possible, we should try to keep both the testing
infrastructure and stable releases for all arches.
d) the debian infrastructure is not the sole property of the mainstream
arches, nor of the dtp-master team, but belong to the whole project,
including to the different porter team.
I have as to yet failed to find a complete description of the problems at
hand, many should be listed in old postings, or something, but let's try to
list them here, i will list a few, please complete by those i forgot.
- the mirror bandwidth is problematic.
- the testing scripts don't scale well on so many architectures.
- the autobuilders need to be reactive enough to build for RC bugs and
security issues.
- the release-team is not scaling well to the workload involved by all those
arches.
- same for the security team.
- unstable snapshots are not a viable solution, even if stable is not
acceptable, all ports need at least some testing solution.
I have already made some proposals in the past, but this is an effort to
clarify it, get people involved in a fresh thread, and to bring this
discussion forward in such a way that we can get a resonable discussion at the
helsinski debconf'05 meeting.
My proposal was to keep a sort-of-testing going for all ports, in such a way
as to allow syncing of the minority ports with the testing, and make a stable
release possible, but without hindering the release of tier1 arches. It works
as follow :
1) all uploads happen on a single archive (located on ftp-master), which
would be unstable.
2) the tier1 arches all release in sync, using the existing testing script,
and handled by the release-managers. The testing script can be located on
another machine if needed, depending on the load.
3) tier1 releases happen when the release manager is happy with them, without
any interaction of the minority arches.
Ok, this is the easy part, and also what the vancouver-proposal included, the
difference comes in how the minority-arches are handled, and my proposal is a
'including' proposal, while the vancouver-proposal was 'excluding'.
4) each non-tier1 arches will have its own testing infrastructure, which
would take both unstable and testing in account.
5) packages are built out of unstable into an arch-specific unstable binary
repository on a separate machine (altough many minority arches may share this
infrastructure).
6) the per arch testing script will migrate packages from arch-unstable into
arch-testing with the common criteria's as the testing script do today, with
two differences : the packages must be in tier1 testing also, and it can take
care of per-arch RC bugs (that is bugs that are none-RC for tier1, but are RC
on this arch).
7) the porter team has the possibility to providing arch-specific overrides
to solve the issue of a package not passing from unstable into testing due to
a tier1-specific RC bug or whatever. Should be used sparingly though.
8) the port team can do testing-proposed-updates or
arch-testing-proposed-updates for arch fixes which are hold back in unstable
for unrelated reason.
=> this is the most flakey entry of this plan, i am not entirely sure how the
exact mechanism would work, discussion accepted.
Ok, this would take care of having a testing infrastructure, which altough it
would not hold on the tier1 testing, will still try to be synced with it,
depending on the archive speed.
The next step is the release process.
9) tier1 arches decide to release as ususal. At this point the tier2 arches
decide individually what they want to do, if they are ready for a release, or
don't want to try for it.
10) if they want to release, they will either freeze unstable, or fork it or
whatever, and will work with their copy of testing to stabilize it with regard
to the released tier1 arch, and make uploads to stable-proposed-updates (or a
separate stable-arch-proposed-updates) for the sole reason to make
arch-stable releasable. Uploads should be minimalistic and only needed to fix
arch-specific breakage, and tested on tier1 arches before being accepted in
stable-proposed-updates. It is the responsability of the porters (or a
porter-support-team), to do this testing.
11) if they succeed in this, those updates can be made part of a future
stable point release.
Now to the security setup, which is only handling stable releases and work
through stable-proposed-updates or something similar, as well as the
stable-security stuff. The vancouver proposal has said on the behalf of the
security team, that the problem in doing security updates may be twofold,
correct me if i am wrong :
a) security builds need to build in a finite amount of time, in order to not
delay security updates past their end-of-embargo time. Notice that the
embargo time is often more than a week or something such though.
b) arch-specific security issues need someone to investigate fix and build.
c) security work needs NDA to get advance warning of security issues.
I don't think there are other issues. The vancouver proposal just drops
security for non-tier1 arches, without further posibility.
The counter proposal would be :
1) tier1 arch are supported by the security team, and handled as usual. The
announcement goes out without waiting for tier2 arches.
2) each port which want to do security upgrades has to have a
security-representative, which would be under NDA and have the possibility
of getting access to the security info during the embargo, and being able to
do the security build well in advance of the announce date.
3) at security announcement, the security team does provide info about all
the ports which made the build, and inform that builds are upcoming on
slower arches that are building it.
That should be make everyone involved happy.
What is needed to make this happen ? We need (have) :
1) the tier1-team, comprising ftp-masters, release-managers, security-team.
Those would work as usual, but with fewer arches, and make a decision of
dropping/promoting arches from/to tier1. There may be some initial setup
cost, but it should not be all that high, and other folk can help if needed
without being part of the tier1-team.
2) each arch needs to provide :
- a buildd network able to build the packages.
- one machine handling the per-arch testing script, and other overhead.
- one arch-release-assistant (or team), with power to handle the
arch-specific testing script and be able to follow the issue.
(see bill's proposal for details on this).
- one arch-security representative, which will be able to get early access
to the embargoed security issues, and build the security fix if the
tier1 - security can't handle those.
3) the tier1-team could be adjoined a port-support-team, with would handle a
certain number of issues common to all arches.
4) the tier1-team will *NOT* make live difficult for the porters, and keep
considering them when they make decision, even though they may not have
other choice than go with some port-harming solutions.
Well, that is my proposal upto now, a bit different from the one i posted
previously, but i believe many think it a good idea to try to solve this issue
positively, and think like me that debian without the many arch support, or at
least trying for it, has partly lost his soul.
I would like to have feedback on this from :
1) the vancouver-proposal authors.
2) the other members of the upcoming tier1-team, who are not mentioned in
the vancouver document.
3) the porters.
Since obviously without input and agreement of all of them, there is no chance
of solving this. And please be positive about your input, as we are all in
this together and want to solve it, don't we ?
Friendly,
Sven Luther
Reply to: