It was a nice surprise finding several huge threads waiting for me after my
return from VAC. I had browsed the archives while at Steve McIntyre's annual
BBQ (thanks Steve, it was great!), but amazing how much mail had been
written since then.
Both personally and in my role as D-I release manager I am less than happy
with some points in the discussion and the way it currently seems to be
heading.
This mail will probably run quite long, but as I had over two weeks to
catch up with and as the installer has a fairly central role in this whole
issue, I hope you will bear with me.
I will start with an alternative GR proposal based on the one from aj. After
that I will give both a personal view and reaction as D-I release manager
on the discussion so far and sketch the solution I envision for after Etch.
As the most contentious issue seems to be sourceless firmware and as that
most affects the installer, I will mostly concentrate on that.
On Tuesday 05 September 2006 09:44, Anthony Towns wrote in
<[🔎] 20060905074404.GA8506@azure.humbug.org.au>:
> It therefore seems to me as though we're going to be failing to meet the
> social contract again, and as a consequence I think we should seriously
> reconsider whether the change we made in 2004 was the right one. So I'd
> like to propose the following course of action for consideration:
I rather like the basic idea behind this proposal and I applaud the personal
commitment the DPL is willing to make on this issue, however I feel it is
wrong to do things in this order: we should not rush a change in the SC in
order to get a release out on schedule only to have to discuss another
change in the SC immediately after the release.
I would suggest to not decide on a), b), c) and d) now but rather shelve
those until after the release and instead do something like:
The project acknowledges that a lot of progress has been made with regard
to the removal from the distribution (main) of "software" that could be
considered non-free given the current wording of the Social Contract.
However, in some cases for valid reasons, this work is not finished and
requiring this to be finished before the release of Etch would result in a
serious delay of the planned release.
There are also indications that a significant group of people within the
project feels that the current Social Contract does not meet the best
interests of the project in that the current wording is too restrictive and
that a limited and conditional inclusion/support of some types of
"software" should be possible. Example: support for loading sourceless
firmware during installation.
The Debian Project resolves that:
(a) The inclusion in main of sourceless firmware and support in Debian
Installer is not a release blocker for the release of Etch.
(b) For the release of Etch, the Release Managers are given discretion
to waive RC issues in other cases where the letter of the Social
Contract is currently not being met, provided there is no regression
relative to the Sarge release and that waivers are done consistently
and with proper consideration of past resolutions (e.g. GDFL) and
work already done on other (comparable) packages.
(c) Following the release of etch, the Debian Project Leader shall:
i. ensure that the Debian community has a good understanding
of the technical and legal issues that prevent the Debian
Free Software Guidelines from being applied to logos and
firmware in a manner that meets the needs of our users;
ii. ensure that project resources are made available to
people working on addressing those issues;
iii. keep the Debian community updated on progress achieved
in these areas.
(d) Following the release of etch, the Debian Project as a whole shall
reopen the question of which commitments should be codified in the
project's Social Contract. This shall include both an online
consultation with Debian developers, users, Debian derivatives and
the free software community, and a public in-person discussion at
DebConf 7 in Edinburgh in honour of the 10th anniversary of the
original publication of the Social Contract on the 4th of July 1997.
----
I have changed (c).iii to not refer explicitly to Debconf as I feel progress
should be reported and discussed with the whole Debian community, not just
at Debconf. Others have made similar comments. I see no problems with
organizing a discussion at DebConf.
I also fixed a typo, added "developers" and removed "and debate" in (d).
These changes have already been OKed by aj.
With this wording this proposal could perhaps be an alternative to Frederik
Schuller's proposal (<20060830210654.GA8675@mail.lowpingbastards.de>) with
as main difference that:
- his proposal assumes there is consensus about the DFSG/SC and merely
postpones work on the implementation, while
- this proposal aims to structurally revisit the whole subject after the
release of Etch and leaves more room for more liberal eventual solutions.
Key point of this proposal is: do not decide now, focus on releasing Etch
and have a proper discussion about the SC and consequences/implementation
of options after the release.
Note that my proposal also allows for reaffirmation of the current SC: it
leaves the issue open. The only cost is some time, but as the discussion
will take place straight after the release and a lot of groundwork has
already been laid, that should not hinder implementation in the Etch+1
timeframe.
COMMENTS ON THE DISCUSSION SO FAR
=================================
My rough summary:
- (almost) everybody agrees that non-free drivers don't belong in main;
- (almost) everybody agrees that sourceless firmware at least needs to be
distributable before any kind of support can be considered;
- most people agree that Etch should not be delayed for a solution to the
sourceless firmware issue;
- a fair number of people (though a percentage is hard to estimate) seem to
feel that the current Social Contract is too restrictive when it comes to
some types of files, forms of documentation and sourceless firmware;
- probably a larger number feels that we should not kill the project by
scaring away users with hardware that depends on sourceless firmware and
is willing to consider solutions for that while still making the projects
preference for firmware _with_ source clear to users and others.
There is a weird mix of policy/politics and implementation in this
discussion, especially by the people in favor of resolution of the
firmware issue before Etch. IMO d-vote is _not_ the correct platform to
discuss implementation, that should be done on either d-kernel/boot or
d-devel, but there has been a lack of real discussion there (except for one
short recent thread started by Wouter Verhelst on d-boot).
Also, the proposed implementations are at the level of a rough idea without
any details worked out or research done regarding feasibility.
As indicated above, I also feel that the pressure of the Etch release is
tempting people too hurry decisions without really discussing the issues and
alternatives, which can only lead to poorly drafted ballots with a bias to
one solution or another (as IMHO the past ballots on the SC change and the
waiver for Sarge have been).
IMO it will be a real challenge to formulate a coherent GR ballot from
everything that is currently on the table (oh dear, I've just added to the
mess...), or no longer on the table.
Also, a depressingly small number of people have really participated in the
discussion. One reason is that not that many people are subscribed to
d-vote. The people who have participated tend to be the ones that are very
outspoken and on extreme ends of the discussion. Thus reading the thread
probably gives a very poor indication of what the project as a whole would
like to see happen.
Some statistics to give an indication (this is based on my local mailbox
of d-vote for Aug/Sept for subjects containing "source" or "firmware" with
only some irrelevant subthreads deleted).
In total 465 posts from approximately 80 different persons.
87 From: Sven Luther <sven.luther@wanadoo.fr>
33 From: Thomas Bushnell BSG <tb@becket.net>
30 From: Marco d'Itri <md@Linux.IT>
29 From: Steve Langasek <vorlon@debian.org>
23 From: Anthony Towns <aj@azure.humbug.org.au>
16 From: MJ Ray <mjr@phonecoop.coop>
18 From: Nathanael Nerode <neroden@fastmail.fm>
13 From: Manoj Srivastava <srivasta@debian.org>
10 From: Josselin Mouette <joss@debian.org>
In addition to these heavy posters, about 15 people contributed 4 to 9
mails. The full archive from master will give some different numbers but I
would be very surprised if the overall picture will be any different.
My impression was that numerically there were *more* people _in favor_ of
a more liberal approach (or at least of voting those proposals), but that
they found it sufficient to second the proposals, sometimes with short
comments, and not participate in the discussion.
The position of the d-i team has so far been presented by Joey Hess and
there have been some contributions from other d-i team members (Yoe,
p2-mate, bubulle). Joey has given an overview of the impact of removing
firmware from main on the installer with an estimate of required effort.
The position of the kernel team in this is interesting. AFAIK the kernel
team _is_ unanimous in the proposal to postpone the firmware issue until
after Etch (which has consistently been Sven's position in the discussion).
However, from discussions on IRC and private conversations with several
kernel team members I know that the personal opinions of its members vary
wildly. At least one is radically opposed to removing firmware from main
and a few others are much more open to further discussion or favor a much
more liberal long term solution than proposed by Sven. IMO it is a real
pity that not more kernel team members have spoken up in the discussion.
I was appalled at the suggestion that surfaced in the discussion that the
kernel or d-i team have actively refused to work on or delayed a solution
for the firmware issue in time for Etch. The facts:
- both the kernel and d-i team consist of volunteers, so the basic rule is:
members are masters over their own time and effort;
- in general there has been good coordination and cooperation between the
kernel and d-i team;
- the kernel team has done a major job on the integration of kernel source
for all arches for 2.6 _and_ making the switch from initrd to initramfs
while keeping support in kernel-package; a lot of time and effort has
gone into this and as this was unavoidable to support upstream releases it
took precedence over the firmware issue;
- between Sarge and Etch there has been major development on the installer
even though the number of active team members has shrunk somewhat, mainly
because the installer had reached maturity with the Sarge release; a lot
of the work had to be done to keep up with changes in the kernel and
environment; other major projects: integration of base-config into the
installer, graphical installer and partman-crypto (last two mostly by
"new" members in the team);
- implementation of a solution for firmware/non-free drivers in d-i has
been discussed but consensus was that there was not much point in working
on it while there was no separation in the kernel;
- when that was done a change in the archives was requested to support
loading udebs from contrib/non-free (and experimental); this was not
picked up for a long time which led to loss of motivation and time;
- the person implementing the kernel part of the firmware solution happens
to also be the person most suited to make some necessary changes in the
installer;
- the first separation in kernel packaging happened only recently; at that
time it was really already too late for a solution in time for Etch in
the installer.
VISION FOR THE FUTURE
=====================
My _personal_ preference at this point would be a solution as
etch-a-sketched below. I have seen several people suggesting elements from
this during the discussion.
- Creation in the archive of a new, separate section ("restricted"?) for
packages that can be used for new installations but are not suitable for
main (alternatively they could be left in main but specially tagged in
the control file). Main reason: you not only need such packages during
installation, but also for the installed system; loading such packages
from contrib/non-free would imply that these sections have to be added by
default to the sources list for these users which is undesirable given the
aims of the project [1].
- Allow inclusion of these packages in installer images and on installation
media.
- During installation, present a message to the user every time such a
package is actually used; user has to acknowledge.
- Also add support for loading non-free/third party drivers into the
installer, but _without_ including such drivers in installer images or on
installation media.
These are invasive changes and require much more thought and careful
implementation in kernel packaging, the installer and debian-cd; the
required support in the archive will probably also not happen overnight ;-)
I am strongly opposed to suggestions of creating a "non-free" version of the
installer besides the regular "free" version. This would lead to an
explosion of the number of images, a waste of bandwidth and mirror space,
severely complicate building, testing and release management and will only
confuse users who will probably only know they need the non-free version
after a failed installation.
[1] This is also why I object to MJ Ray's amendment in
<[🔎] 20060905122545.1178DF6CBE@nail.towers.org.uk> as it codifies the solution
without having checked its practical implications.
STANDPOINT AS D-I RELEASE MANAGER
=================================
Frankly I am amazed that some people so easily dismiss Joey Hess' estimate
of the effort required for proper implementation of support for firmware
(and non-free drivers). There are probably only two people who come close
to his level of overall understanding of the installer (no, I'm not one of
them, I rank myself about fifth).
The installer is a complex beast with loads of subtle dependencies, space
issues and a tight interdependency with debian-cd. My experience with any
structural change is that there is always unexpected breakage that only
shows sometime _after_ the next release.
Besides that there are a load of issues that people fail to consider when
proposing solutions:
- the way non free kernel udebs are currently built (as by-product of the
build of the deb) does not fit with d-i release management and the skew
we habitually see near a d-i release in unstable between the kernel
version the installer uses and the kernel images in unstable; it would
be better to create the udebs using kernel-wedge;
- if firmware is used in the installer, the installer also has to make sure
the corresponding regular package is installed for the installed system;
so, as already indicated above, a solution needs to be found for both the
installer _and_ the installed system, preferably without forcing the user
to add _all_ of contrib/non-free in his sources list just because he/she
needs one sourceless firmware file;
- adding separate non-free installer images requires extra mirror space and
changes in BYHAND processing and thus needs approval from FTP-masters;
- adding separate non-free CD images would mean a substantial hit on the
debian-cd (mirror) infrastructure and is currently probably not even an
option.
The current status of the installer is that, after the recent Beta 3
release, it is basically frozen for new development (with a few exceptions
agreed on before the release of Beta 3). There is also quite a long list of
RC and minor issues, polishing and testing to be done and we still have the
upgrade from 2.6.17 to 2.6.18 coming. I estimate we will have at least 3 RC
releases in the run up-to 4 December.
There is currently no support whatsoever for loading firmware in the
installer (as so far there has been no need for it).
So, my formal position as D-I release manager has to be:
"We will not accept any structural changes to support loading firmware in
the installer (not from main nor from elsewhere),
- if the release plan for Etch remains at 4 December 2006;
- unless both Joey Hess and I, after careful review of a finished and
tested proposed solution, decide that 1) it provides an acceptable
solution for all installation methods and architectures, 2) it poses no
risk of regressions or delays in the run-up to the release."
The initiatives started during the discussion on d-vote to implement some
kind of support are appreciated, but they are too late for Etch!
In my personal opinion they are also premature given the controversy that
exists over the question if firmware should really be treated as totally
non-free or rather be supported to some extend.
Thanks for hearing me out,
FJP
Attachment:
pgpjKrA_GRgWD.pgp
Description: PGP signature