[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Proposal - Defer discussion about SC and firmware until after the Etch release

It was a nice surprise finding several huge threads waiting for me after my
return from VAC. I had browsed the archives while at Steve McIntyre's annual 
BBQ (thanks Steve, it was great!), but amazing how much mail had been 
written since then.

Both personally and in my role as D-I release manager I am less than happy
with some points in the discussion and the way it currently seems to be

This mail will probably run quite long, but as I had over two weeks to
catch up with and as the installer has a fairly central role in this whole
issue, I hope you will bear with me.

I will start with an alternative GR proposal based on the one from aj. After 
that I will give both a personal view and reaction as D-I release manager 
on the discussion so far and sketch the solution I envision for after Etch.
As the most contentious issue seems to be sourceless firmware and as that
most affects the installer, I will mostly concentrate on that.

On Tuesday 05 September 2006 09:44, Anthony Towns wrote in 
> It therefore seems to me as though we're going to be failing to meet the
> social contract again, and as a consequence I think we should seriously
> reconsider whether the change we made in 2004 was the right one. So I'd
> like to propose the following course of action for consideration:

I rather like the basic idea behind this proposal and I applaud the personal 
commitment the DPL is willing to make on this issue, however I feel it is
wrong to do things in this order: we should not rush a change in the SC in
order to get a release out on schedule only to have to discuss another 
change in the SC immediately after the release.

I would suggest to not decide on a), b), c) and d) now but rather shelve
those until after the release and instead do something like:

The project acknowledges that a lot of progress has been made with regard
to the removal from the distribution (main) of "software" that could be
considered non-free given the current wording of the Social Contract.
However, in some cases for valid reasons, this work is not finished and
requiring this to be finished before the release of Etch would result in a
serious delay of the planned release.

There are also indications that a significant group of people within the
project feels that the current Social Contract does not meet the best
interests of the project in that the current wording is too restrictive and
that a limited and conditional inclusion/support  of some types of
"software" should be possible. Example: support for loading sourceless 
firmware during installation.

The Debian Project resolves that:

    (a) The inclusion in main of sourceless firmware and support in Debian
        Installer is not a release blocker for the release of Etch.

    (b) For the release of Etch, the Release Managers are given discretion
        to waive RC issues in other cases where the letter of the Social
        Contract is currently not being met, provided there is no regression
        relative to the Sarge release and that waivers are done consistently
        and with proper consideration of past resolutions (e.g. GDFL) and
        work already done on other (comparable) packages.

    (c) Following the release of etch, the Debian Project Leader shall:
          i.   ensure that the Debian community has a good understanding
               of the technical and legal issues that prevent the Debian
               Free Software Guidelines from being applied to logos and
               firmware in a manner that meets the needs of our users;
          ii.  ensure that project resources are made available to
               people working on addressing those issues;
          iii. keep the Debian community updated on progress achieved
               in these areas.

    (d) Following the release of etch, the Debian Project as a whole shall
        reopen the question of which commitments should be codified in the
        project's Social Contract. This shall include both an online
        consultation with Debian developers, users, Debian derivatives and
        the free software community, and a public in-person discussion at
        DebConf 7 in Edinburgh in honour of the 10th anniversary of the
        original publication of the Social Contract on the 4th of July 1997.


I have changed (c).iii to not refer explicitly to Debconf as I feel progress 
should be reported and discussed with the whole Debian community, not just 
at Debconf. Others have made similar comments. I see no problems with 
organizing a discussion at DebConf.
I also fixed a typo, added "developers" and removed "and debate" in (d). 
These changes have already been OKed by aj.

With this wording this proposal could perhaps be an alternative to Frederik
Schuller's proposal (<20060830210654.GA8675@mail.lowpingbastards.de>) with 
as main difference that:
- his proposal assumes there is consensus about the DFSG/SC and merely
  postpones work on the implementation, while
- this proposal aims to structurally revisit the whole subject after the
  release of Etch and leaves more room for more liberal eventual solutions.

Key point of this proposal is: do not decide now, focus on releasing Etch 
and have a proper discussion about the SC and consequences/implementation 
of options after the release.

Note that my proposal also allows for reaffirmation of the current SC: it 
leaves the issue open. The only cost is some time, but as the discussion 
will take place straight after the release and a lot of groundwork has 
already been laid, that should not hinder implementation in the Etch+1 


My rough summary:
- (almost) everybody agrees that non-free drivers don't belong in main;
- (almost) everybody agrees that sourceless firmware at least needs to be
  distributable before any kind of support can be considered;
- most people agree that Etch should not be delayed for a solution to the
  sourceless firmware issue;
- a fair number of people (though a percentage is hard to estimate) seem to
  feel that the current Social Contract is too restrictive when it comes to
  some types of files, forms of documentation and sourceless firmware;
- probably a larger number feels that we should not kill the project by
  scaring away users with hardware that depends on sourceless firmware and
  is willing to consider solutions for that while still making the projects
  preference for firmware _with_ source clear to users and others.

There is a weird mix of policy/politics and implementation in this
discussion, especially by the people in favor of resolution of the
firmware issue before Etch. IMO d-vote is _not_ the correct platform to
discuss implementation, that should be done on either d-kernel/boot or
d-devel, but there has been a lack of real discussion there (except for one
short recent thread started by Wouter Verhelst on d-boot).
Also, the proposed implementations are at the level of a rough idea without
any details worked out or research done regarding feasibility.

As indicated above, I also feel that the pressure of the Etch release is
tempting people too hurry decisions without really discussing the issues and 
alternatives, which can only lead to poorly drafted ballots with a bias to 
one solution or another (as IMHO the past ballots on the SC change and the 
waiver for Sarge have been).

IMO it will be a real challenge to formulate a coherent GR ballot from
everything that is currently on the table (oh dear, I've just added to the
mess...), or no longer on the table.

Also, a depressingly small number of people have really participated in the
discussion. One reason is that not that many people are subscribed to
d-vote. The people who have participated tend to be the ones that are very
outspoken and on extreme ends of the discussion. Thus reading the thread
probably gives a very poor indication of what the project as a whole would
like to see happen.

Some statistics to give an indication (this is based on my local mailbox
of d-vote for Aug/Sept for subjects containing "source" or "firmware" with
only some irrelevant subthreads deleted).
In total 465 posts from approximately 80 different persons.
     87 From: Sven Luther <sven.luther@wanadoo.fr>
     33 From: Thomas Bushnell BSG <tb@becket.net>
     30 From: Marco d'Itri <md@Linux.IT>
     29 From: Steve Langasek <vorlon@debian.org>
     23 From: Anthony Towns <aj@azure.humbug.org.au>
     16 From: MJ Ray <mjr@phonecoop.coop>
     18 From: Nathanael Nerode <neroden@fastmail.fm>
     13 From: Manoj Srivastava <srivasta@debian.org>
     10 From: Josselin Mouette <joss@debian.org>
In addition to these heavy posters, about 15 people contributed 4 to 9 
mails. The full archive from master will give some different numbers but I 
would be very surprised if the overall picture will be any different.

My impression was that numerically there were *more* people _in favor_ of
a more liberal approach (or at least of voting those proposals), but that
they found it sufficient to second the proposals, sometimes with short
comments, and not participate in the discussion.

The position of the d-i team has so far been presented by Joey Hess and 
there have been some contributions from other d-i team members (Yoe, 
p2-mate, bubulle). Joey has given an overview of the impact of removing 
firmware from main on the installer with an estimate of required effort.

The position of the kernel team in this is interesting. AFAIK the kernel 
team _is_ unanimous in the proposal to postpone the firmware issue until 
after Etch (which has consistently been Sven's position in the discussion).
However, from discussions on IRC and private conversations with several 
kernel team members I know that the personal opinions of its members vary 
wildly.  At least one is radically opposed to removing firmware from main 
and a few others are much more open to further discussion or favor a much 
more liberal long term solution than proposed by Sven. IMO it is a real 
pity that not more kernel team members have spoken up in the discussion.

I was appalled at the suggestion that surfaced in the discussion that the
kernel or d-i team have actively refused to work on or delayed a solution
for the firmware issue in time for Etch. The facts:
- both the kernel and d-i team consist of volunteers, so the basic rule is:
  members are masters over their own time and effort;
- in general there has been good coordination and cooperation between the
  kernel and d-i team;
- the kernel team has done a major job on the integration of kernel source
  for all arches for 2.6 _and_ making the switch from initrd to initramfs
  while keeping support in kernel-package; a lot of time and effort has
  gone into this and as this was unavoidable to support upstream releases it
  took precedence over the firmware issue;
- between Sarge and Etch there has been major development on the installer
  even though the number of active team members has shrunk somewhat, mainly
  because the installer had reached maturity with the Sarge release; a lot
  of the work had to be done to keep up with changes in the kernel and
  environment; other major projects: integration of base-config into the
  installer, graphical installer and partman-crypto (last two mostly by
  "new" members in the team);
- implementation of a solution for firmware/non-free drivers in d-i has
  been discussed but consensus was that there was not much point in working
  on it while there was no separation in the kernel;
- when that was done a change in the archives was requested to support
  loading udebs from contrib/non-free (and experimental); this was not
  picked up for a long time which led to loss of motivation and time;
- the person implementing the kernel part of the firmware solution happens
  to also be the person most suited to make some necessary changes in the
- the first separation in kernel packaging happened only recently; at that
  time it was really already too late for a solution in time for Etch in
  the installer.


My _personal_ preference at this point would be a solution as 
etch-a-sketched below. I have seen several people suggesting elements from 
this during the discussion.

- Creation in the archive of a new, separate section ("restricted"?) for
  packages that can be used for new installations but are not suitable for
  main (alternatively they could be left in main but specially tagged in
  the control file). Main reason: you not only need such packages during
  installation, but also for the installed system; loading such packages
  from contrib/non-free would imply that these sections have to be added by
  default to the sources list for these users which is undesirable given the
  aims of the project [1].
- Allow inclusion of these packages in installer images and on installation
- During installation, present a message to the user every time such a
  package is actually used; user has to acknowledge.
- Also add support for loading non-free/third party drivers into the
  installer, but _without_ including such drivers in installer images or on
  installation media.

These are invasive changes and require much more thought and careful 
implementation in kernel packaging, the installer and debian-cd; the
required support in the archive will probably also not happen overnight ;-)

I am strongly opposed to suggestions of creating a "non-free" version of the 
installer besides the regular "free" version. This would lead to an 
explosion of the number of images, a waste of bandwidth and mirror space, 
severely complicate building, testing and release management and will only 
confuse users who will probably only know they need the non-free version 
after a failed installation.

[1] This is also why I object to MJ Ray's amendment in 
<20060905122545.1178DF6CBE@nail.towers.org.uk> as it codifies the solution 
without having checked its practical implications.


Frankly I am amazed that some people so easily dismiss Joey Hess' estimate
of the effort required for proper implementation of support for firmware
(and non-free drivers). There are probably only two people who come close
to his level of overall understanding of the installer (no, I'm not one of 
them, I rank myself about fifth).

The installer is a complex beast with loads of subtle dependencies, space
issues and a tight interdependency with debian-cd. My experience with any
structural change is that there is always unexpected breakage that only
shows sometime _after_ the next release.

Besides that there are a load of issues that people fail to consider when
proposing solutions:
- the way non free kernel udebs are currently built (as by-product of the
  build of the deb) does not fit with d-i release management and the skew
  we habitually see near a d-i release in unstable between the kernel
  version the installer uses and the kernel images in unstable; it would
  be better to create the udebs using kernel-wedge;
- if firmware is used in the installer, the installer also has to make sure
  the corresponding regular package is installed for the installed system;
  so, as already indicated above, a solution needs to be found for both the
  installer _and_ the installed system, preferably without forcing the user
  to add _all_ of contrib/non-free in his sources list just because he/she
  needs one sourceless firmware file;
- adding separate non-free installer images requires extra mirror space and
  changes in BYHAND processing and thus needs approval from FTP-masters;
- adding separate non-free CD images would mean a substantial hit on the
  debian-cd (mirror) infrastructure and is currently probably not even an

The current status of the installer is that, after the recent Beta 3
release, it is basically frozen for new development (with a few exceptions
agreed on before the release of Beta 3). There is also quite a long list of
RC and minor issues, polishing and testing to be done and we still have the
upgrade from 2.6.17 to 2.6.18 coming. I estimate we will have at least 3 RC
releases in the run up-to 4 December.
There is currently no support whatsoever for loading firmware in the 
installer (as so far there has been no need for it).

So, my formal position as D-I release manager has to be:
"We will not accept any structural changes to support loading firmware in
the installer (not from main nor from elsewhere),
- if the release plan for Etch remains at 4 December 2006;
- unless both Joey Hess and I, after careful review of a finished and
  tested proposed solution, decide that 1) it provides an acceptable
  solution for all installation methods and architectures, 2) it poses no
  risk of regressions or delays in the run-up to the release."

The initiatives started during the discussion on d-vote to implement some
kind of support are appreciated, but they are too late for Etch!
In my personal opinion they are also premature given the controversy that
exists over the question if firmware should really be treated as totally
non-free or rather be supported to some extend.

Thanks for hearing me out,

Attachment: pgpCyJwsCmlLp.pgp
Description: PGP signature

Reply to: