Re: Etch timeline is unrealistic because non-free firmware is NOT being dealt with
On Sun, Aug 13, 2006 at 03:47:13AM +0200, Goswin von Brederlow wrote:
> > The whole concept is of an extreme fragility, prone to break again and again,
> > cause lot of work for all involved, who become irritable, and bash on you when
> > you even mention it.
>
> I did it when working on the amd64 D-I for sarge. I found it quite
> trivial to do, a matter of minutes. In fact a hell of a lot simpler
> and way faster than getting the linux-2.6 source package to do
> something for me.
Naturally, amd64 being one of the mainstream arches the d-i team cares about.
Now, please provide patches for resurecting the apus flavour, and then we can
speak again.
>
> The kernel-wedge lists do break from time to time but they are simple
> to adjust and quick to rebuild.
>
> And another advantage with kernel-wedge: You can easily take your
> (maybe even prebuild) custom kernel and create udebs from it. I would
> hate to have to add the SuSe or RH patch sets to the linux-2.6 source
> to build kernel udebs for hardware that requires their patches.
This could be easily be solved by adding the module info to the source package
in some way, or even integrate them into kernel-package.
> > This, in my book, is not an example of good software engineering, and any
> > proposal to help improve this should be worth considering.
>
> Still not convinced. You do have some points but I see more drawbacks
> and problems than advantages.
At least you had the honestity to discuss it, while other rejected it out of
hand, with patronizing and aggresive language.
> >> Is the space taken up by the installed modules your actual argument
> >> for having finer grained module udebs?
> >
> > Nope, but then i told you that in the first mail already :)
>
> Then you have presented no argument for splitting the module udebs and
> several against.
Several ?
> You do have raised some points of improvement for kernel-wedge. Like
> automaticaly adjusting to changed .config files or some other magics
> you mentioned. And some points for building module udebs from within
> linux-2.6 source although I don't agree with you there.
Why not ? It is the only way we can bring down the delay for security builds
with abi change to the minimum time, and the less human intervention.
> > The d-i team only needs to handle a list of the modules it want to include in
> > this or that ramdisk, and all will work automatically.
>
> Which has the same problem as the kernel-wedge configuration has now
> when splitting the modules. When a module gets added or removed the
> list has to be adjusted.
Err, no, because the list would be handled in the debian-installer package or
its devel tree in the svn repo, and not inside kernel-wedge.
Notice that we already have the mapping of module .udeb packages to d-i images
in d-i proper, which needs to be done anyway in addition to the
kernel-wedge/linux-xxx-di work.
Furthermore with the current situation, and given that not all modules are
built the same on all arches, and that binary size vary per arch, you end up
in an unholy mess when trying to build space constrained ramdisks, like the
floppy ones for example.
> > Compared to the current situation, where the kernel porters have to update the
> > config files, then build the kernel .deb, they have to pass NEW, then the d-i
>
> Still has to happen.
Indeed, but that is the only part that needs to be done, not the rest.
> > porters update the the kernel .udeb packages (one per arch), and the list of
>
> Which is mostly just a new upload.
WRONG. it is at best an upload *PER ARCH*, done by *ONE DIFFERENT PERSON AT A
DIFFERENT TIME* for each arch.
Furthermore, the build, failure, check what the heck kernel module has changed
name or dissapeared, or whatnot, is a *ROYAL PAIN*, and very much time
consuming., especially on slow arches with lot of flavours.
Furthermore, given that you need to install the kernel image package, you
can't even start to autobuild that without hosing the autobuilders.
> > modules which need to go into each ramdisk flavour, not mentioning that the
>
> Which is pretty constant from my experience since the modules are
> already split right into udebs.
They are split into the right .udebs for the x86 and other mainstream cases
the kernel-wedge maintainer cares about.
> > ramdisk package list has no support for per-flavour module selection, and you
> > have to end up with stuff like the netboot64 list, which has as sole usage to
> > add the ibm power hypervisor and virtualization modules, all an ugly mess.
>
> Something to improve. No argument for or against your proposal.
Because you miss that my proposal makes this whole issue obsolet and
non-existent.
> > So, the new proposal would mean the porters decide which config options (and
> > thus which modules), should be built, as well, in conjunction with the d-i
> > folk, which of those modules may be usefull for d-i, disks, networks, input
> > devices, display devices, the decision is pretty easy. The d-i team only needs
> > to handle a single list of modules per ramdisk, which can be determined at d-i
> > image built time, and not need to upload kernel wedge and the individual
> > kernel .udeb packages for each arch, just because you want to add a single
> > module you didn't think of the week before.
>
> All modules will be build for a kernel usualy. At least a ton more
> modules than D-I needs. So you need an extra config file that lists
> the modules that D-I needs, which is the list kernel-wedge now has.
Indeed. The difference is that the knowledge about this is disseminated in at
least 3 different places, and that the kernel-wedge maintainer cares about no
more than 3-5 arches compared to the 12 or so official debian arches (at least
the powerpc case repeteadly broke every so often).
The reasoning behind this analysis is that the kernel porter need to make the
decision about each new config option anyway, that the kernel has good
infrastructure in place for separating that choice in between debian-global
choices, per-arch choices, per variant (historically called subarches), and
per flavour.
The decision on what modules to offer as .udebs for d-i is rather trivial,
once the decision on how to group them in .udebs is out of the way, and the
kernel team is as well if not better placed to take that decision. You need
the network modules, the disk modules, the display modules and the input
modules, and another set of assorted modules needed for each machine to run
correctly, like the fan control modules on apple hardware for example.
Then the d-i team can cherry pick among those .udebs and put into each flavour
those they need, being able to do different variants for different hardware
much easier than the current step which takes a couple of week or so.
> You didn't eliminate the need for a manualy tuned list of modules to
> put into udebs. You just shifted the problem from A to B.
Indeed, i split the problematic into two different cases, and assigned those
cases to the people most competent in solving them, in a way which cuts the
requirement for human intervention to the minimum, both time wise and with
regard to the number of actors, as well as simplifying the whole issue by an
order of magnitude.
The cost is small, the only drawback would be the relative augmentation of the
number of .udebs packages, but not the global size of them, and a little work
on the infrastructure side, which is out of our control.
Even the package size augmentation is minimal, since you said yourself that on
most mainstream machines, bandwidth and memory size are perfectly able to
handle it, and on those arches which have problems with memory size, most
modules will probably be builtin, and thus the amount of needed .udebs is also
minimal.
All, in all, it is an almost perfect solution to the issues at hand, elegant,
and with provision for almost every issue involved, it is a global solution to
make everyone's work easier, allow for more timely abi bumping uploads, both
for security and normal development, which would allow to avoid abherations,
like sarge shipping with a known remote security exploit, which is when i
started proposing ideas about creating a common kernel package for all arches,
as well as having hierarchical config file, instead of a bunch of single ones,
and already then i was flamed to death for even proposing such ideas, which is
probably when this witch hunt against me started, since i have been under
attack always since then.
But i think we can agree that nobody regrets the current state of the kernel
packages, nor of the single day uploads, which has brought debian out of the
always-months-outdated kernel era into a more modern and responsive one.
> And then you duplicated the problem because D-I still needs a list of
> modules to put into each ramdisks that they have to adjust for every
> change in the kernels. And you need some meta udebs like net-drivers
> that depend on all network driver modules for easy selection. That is
> exactly the list of modules in kernel-wedge. You didn't eliminate the
> kernel-wedge config files from D-I.
kernel-wedge is not inside d-i, it is an external tool which has nothing to do
with d-i.
As a proof, please do an apt-get source debian-installer, and search for that
list there.
> So now you have both kernel team and D-I team having to handle the
> list of modules for the installer.
I already replied to this repeteadly.
> >> No it isn't. The GPL only requires that you have the source and that
> >> is available via the linux-tree package even for past versions no
> >> longer in the archive.
> >
> > Yeah, but what about cases where old kernel versions are used by d-i while the
> > kernel team already got ride of those ? There linux-tree is of no help to you.
>
> You must mean upstream versions because for debian revisions
> linux-tree (or rather linux-patch-debian-2.x.y) has exactly those
Indeed.
> patches needed. And for upstream versions the
> linux-tree/source/patch-debian packages get renamed and all that has
> to be done by ftp-master is not to remove them before the kernel udebs
> are updated.
Ah, but you did miss the fact we have moved to a single kernel source package,
without the version information in it, around a year ago.
> >> >> Since I showed that new sections have only drawbacks you now suddenly
> >> >> propose building udebs in the linux-2.6 source.
> >> >
> >> > Suddenly, i have been saying exactly that since january or earlier, go read
> >> > the archive, i was even going to implement it, when i was hit by a barrage of
> >> > hatefullness and kicked out of d-i, and threatened by Frans if i ever dared
> >> > work on it.
> >>
> >> I'm talking about this thread.
> >
> > I think the world at large knows that i want the kernel .udebs to be built
>
> I didn't.
Google is your friend.
But you are right, i need to formalize this proposal, and make a larger
diffusion of it.
Friendly,
Sven Luther
Reply to: