Re: Proposal - Defer discussion about SC and firmware until after the Etch release
On Tue, Sep 12, 2006 at 01:47:18AM +0200, Frans Pop wrote:
> Some statistics to give an indication (this is based on my local mailbox
> of d-vote for Aug/Sept for subjects containing "source" or "firmware" with
> only some irrelevant subthreads deleted).
> In total 465 posts from approximately 80 different persons.
> 87 From: Sven Luther <sven.luther@wanadoo.fr>
> 33 From: Thomas Bushnell BSG <tb@becket.net>
> 30 From: Marco d'Itri <md@Linux.IT>
> 29 From: Steve Langasek <vorlon@debian.org>
> 23 From: Anthony Towns <aj@azure.humbug.org.au>
> 16 From: MJ Ray <mjr@phonecoop.coop>
> 18 From: Nathanael Nerode <neroden@fastmail.fm>
> 13 From: Manoj Srivastava <srivasta@debian.org>
> 10 From: Josselin Mouette <joss@debian.org>
> In addition to these heavy posters, about 15 people contributed 4 to 9
> mails. The full archive from master will give some different numbers but I
> would be very surprised if the overall picture will be any different.
What do you hope to gain by this ?
> My impression was that numerically there were *more* people _in favor_ of
> a more liberal approach (or at least of voting those proposals), but that
> they found it sufficient to second the proposals, sometimes with short
> comments, and not participate in the discussion.
>
> The position of the d-i team has so far been presented by Joey Hess and
> there have been some contributions from other d-i team members (Yoe,
> p2-mate, bubulle). Joey has given an overview of the impact of removing
> firmware from main on the installer with an estimate of required effort.
In which he has already been proved wrong once, by Nerode, Goswin and Wouter.
> The position of the kernel team in this is interesting. AFAIK the kernel
> team _is_ unanimous in the proposal to postpone the firmware issue until
> after Etch (which has consistently been Sven's position in the discussion).
> However, from discussions on IRC and private conversations with several
> kernel team members I know that the personal opinions of its members vary
> wildly. At least one is radically opposed to removing firmware from main
> and a few others are much more open to further discussion or favor a much
> more liberal long term solution than proposed by Sven. IMO it is a real
> pity that not more kernel team members have spoken up in the discussion.
Please stop those ad-hominem attacks. You already accused me in the past of
having personal agendas and stuff like that, and you have been on a campaign
to discredit me as a kernel team member because you disagreed with me since
last fall or so.
It is in fact true that a few of the kernel team members don't see it as
important to remove the non-free firmware from main, and are indeed opposed to
it. You make it seem as if i am trying to force the issue on them, while my
position, and also the position of the kernel team as seen in Frederik's
proposal is to avoid the issue until after etch, and then see.
Please stop this diffamatory tactics against me, and if you can't avoid
bashing me in this discussion, maybe you should go back to some more vacations
or something.
> VISION FOR THE FUTURE
> =====================
>
> My _personal_ preference at this point would be a solution as
> etch-a-sketched below. I have seen several people suggesting elements from
> this during the discussion.
>
> - Creation in the archive of a new, separate section ("restricted"?) for
> packages that can be used for new installations but are not suitable for
> main (alternatively they could be left in main but specially tagged in
> the control file). Main reason: you not only need such packages during
> installation, but also for the installed system; loading such packages
> from contrib/non-free would imply that these sections have to be added by
> default to the sources list for these users which is undesirable given the
> aims of the project [1].
How would this be different from a more generic distributable but non-free
section, as i have been proposing ?
> - Allow inclusion of these packages in installer images and on installation
> media.
> - During installation, present a message to the user every time such a
> package is actually used; user has to acknowledge.
Indeed, exactly my own proposal.
> - Also add support for loading non-free/third party drivers into the
> installer, but _without_ including such drivers in installer images or on
> installation media.
What Wouter has been working on.
> These are invasive changes and require much more thought and careful
> implementation in kernel packaging, the installer and debian-cd; the
> required support in the archive will probably also not happen overnight ;-)
>
> I am strongly opposed to suggestions of creating a "non-free" version of the
> installer besides the regular "free" version. This would lead to an
> explosion of the number of images, a waste of bandwidth and mirror space,
> severely complicate building, testing and release management and will only
> confuse users who will probably only know they need the non-free version
> after a failed installation.
This is indeed not the case. There is no reason why the *ADDITION* of a few
non-free .udebs to either an image or a CD will cause any kind of major
breakage. The initrd images are rather samll, and on all arches using
initramfs ramdisks, can be simply cat'ed at the end of the free one. This is
the position held by Wouter and Goswin also, if you don't believe me.
> STANDPOINT AS D-I RELEASE MANAGER
> =================================
>
> Frankly I am amazed that some people so easily dismiss Joey Hess' estimate
> of the effort required for proper implementation of support for firmware
> (and non-free drivers). There are probably only two people who come close
> to his level of overall understanding of the installer (no, I'm not one of
> them, I rank myself about fifth).
>
> The installer is a complex beast with loads of subtle dependencies, space
> issues and a tight interdependency with debian-cd. My experience with any
> structural change is that there is always unexpected breakage that only
> shows sometime _after_ the next release.
Indeed. Such breakage is the sign of poor planing and implementation though,
or maybe due to the huge complexity of the d-i design in see.
> Besides that there are a load of issues that people fail to consider when
> proposing solutions:
> - the way non free kernel udebs are currently built (as by-product of the
> build of the deb) does not fit with d-i release management and the skew
> we habitually see near a d-i release in unstable between the kernel
> version the installer uses and the kernel images in unstable; it would
> be better to create the udebs using kernel-wedge;
Indeed, which is why we have urged for the .udeb kernels to be generated from
the same source as the kernel ones, which is something you have strongly
opposed. This would solve this issue, as the kernel team will be able to
synchronize both the free and non-free .debs and .udebs.
> - if firmware is used in the installer, the installer also has to make sure
> the corresponding regular package is installed for the installed system;
> so, as already indicated above, a solution needs to be found for both the
> installer _and_ the installed system, preferably without forcing the user
> to add _all_ of contrib/non-free in his sources list just because he/she
> needs one sourceless firmware file;
Indeed. The natural solution would be for the kernel .debs and .udebs to be
generated from the same source package. Also, my proposal of separating the
.udeb archive into a kernel only .udeb section as well as the normal
non-kernel .udeb section would greatly help here. It will also represent an
economy in build time and maybe even image size, to be able to build the d-i
image only once, and then add the kernel .udeb archive for each flavour in a
separate way. Remember initramfs images are gzipped cpio archives, so you can
just expand on them, and you can just add such an initramfs archive at the
back of the kernel. Mmm, i wonder if this could not be a most elegant
solution, in which the initramfs is split in two, the kernel modules being
added to the kernel, and the rest of d-i going in a separate initrd image.
This would be most elegant.
> - adding separate non-free installer images requires extra mirror space and
> changes in BYHAND processing and thus needs approval from FTP-masters;
> - adding separate non-free CD images would mean a substantial hit on the
> debian-cd (mirror) infrastructure and is currently probably not even an
> option.
The idea is to simply add a netinst or businesscard cd which includes the
non-free images and udebs. This would mean an additional media change for
those using the non-free images, but appart from that is the best compromise
with regard to extra mirror space, and beats a 'remasterizing of the CDs
need'.
> The current status of the installer is that, after the recent Beta 3
> release, it is basically frozen for new development (with a few exceptions
> agreed on before the release of Beta 3). There is also quite a long list of
> RC and minor issues, polishing and testing to be done and we still have the
> upgrade from 2.6.17 to 2.6.18 coming. I estimate we will have at least 3 RC
> releases in the run up-to 4 December.
> There is currently no support whatsoever for loading firmware in the
> installer (as so far there has been no need for it).
But Goswin and Wouter have been working on it, with the full knowledge of
JoeyH. Also, the ql2xxx firmware is no more included in the mainline kernel,
in favour of a separate archive, and the in-kernel ones are marked as obsolet
or something, and the qlogic firmwares have been uploaded and accepted into
non-free (including .udebs ?). So, there is clearly a need for this for the
etch timeframe.
> So, my formal position as D-I release manager has to be:
> "We will not accept any structural changes to support loading firmware in
> the installer (not from main nor from elsewhere),
> - if the release plan for Etch remains at 4 December 2006;
> - unless both Joey Hess and I, after careful review of a finished and
> tested proposed solution, decide that 1) it provides an acceptable
> solution for all installation methods and architectures, 2) it poses no
> risk of regressions or delays in the run-up to the release."
Yeah, more of the heavy-handed-borderline-dictatorship approach to leading d-i :)
> The initiatives started during the discussion on d-vote to implement some
> kind of support are appreciated, but they are too late for Etch!
> In my personal opinion they are also premature given the controversy that
> exists over the question if firmware should really be treated as totally
> non-free or rather be supported to some extend.
>
> Thanks for hearing me out,
I hope you will extend the same courtesy to me too.
Still hurting,
Sven Luther
Reply to: