Re: Etch timeline is unrealistic because non-free firmware is NOT being dealt with
Sven Luther <sven.luther@wanadoo.fr> writes:
> On Sat, Aug 12, 2006 at 10:32:54PM +0200, Goswin von Brederlow wrote:
>> > Indeed. The proposal to split the packages file in a per flavour kernel
>> > repository comes directly from the need to counterbalance this augmentation of
>> > the number of packages.
>>
>> So instead of having 5 module udebs for my ONE generic kernel I get
>> 200? For say amd64 it would be a big step back.
>
> Indeed, but smaller ones.
>
>> How many archs with flavours are we talking about anyway? I think m68k
>> has 3 flavours and ppc some and every other archs has a generic
>> flavour or drivers buildin already.
>
> I know that the more exotic ones have many flavours. Also, the amount of
> packages will be proportional of the non-builtin drivers. POwerpc has
> currently 5 d-i flavours (or would have if the new d-i powerpc people did they
> work correctly), powerpc, powerpc64, powerpc-miboot, prep and apus.
So the worst case is 5 flavours while most archs have only one. Your
solution would cut the number of kernel udebs by at most a factor of 5
while you want to increase by a factor of 40. That is still a net loss
by a factor of 8 for 5 flavours.
>> So why then do you want to split module udebs?
>
> Because the current way of doing it is problematic. The d-i folk don't want to
> give control of it to the kernel team, and the new proposal handled they
> keeping the same and even better control of how to build the ramdisks, while
> at the same time building the module .udebs from the common kernel source,
> thus saving around 2 weeks of time the d-i folk need to adjust to a new abi or
> version, thus making for more timely kernel security updates, and making the
> work of the divers security teams easier.
Who controls the udebs has nothing to do with splitting the module
udebs into smaller chunks. You could split them while D-I has control
of them or have them build by the kernel team and not split them up
further or do both. The two issues are independent of each other.
>> You just agreed that downloading the module udebs is inconsequential
>> given their size. You agreed that more udebs increases the Index files
>> which is bad.
>
> Err, let's say we have 5 flavours like on powerpc, and each would have 200+
> little module .udebs, which gives us Package numbers of 1000+. Worse if there
> are more flavours. This is the part that joeyh objected to this plan of
> spliting modules because of the fact that d-i loads three in memory copy of
> the Packages file, which in turn prompted the idea of per-flavour
> repositories.
>
> If you look at the whole idea though, you find out that it is a really neat
> solution to this problem, it cares for all the technical hurdles, and is
> elegant and nice, and if you throw in the part that can be automated, it saves
> work for everyone involved.
Say you do get your per flavour split despite increasing the number of
kernel modules worst arch has to deal with by a factor of 8 and for
most archs a factor of 40.
Can you imaging the poor users of a low-mem system going through the
list of 200+ components to pick out the right modules to download?
Neat?
> In my book, this makes it a good idea, or at least something that should be
> tried, and not something you have to menace the implementator so he doesn't
> dare pursue it.
>
>> So all I'm left with is that you don't end up with as much modules in
>> the ramdisk. Something I find irelevant unless you talk about the
>> low-mem flavour.
>
> And the fact that it is much easier to update config option and add and remove
> new modules doesn't count. Naturally, you don't handle kernel .udebs. I did,
> and it was a royal pain in the ass to handle those, it took me hours, for
> stuff that was already done 10 times for the other flavours. And even a few
> days ago, the powerpc64 flavour was still broken with the 2.6.17 kernels,
> because some random module listed in the list of modules was not present.
>
> The whole concept is of an extreme fragility, prone to break again and again,
> cause lot of work for all involved, who become irritable, and bash on you when
> you even mention it.
I did it when working on the amd64 D-I for sarge. I found it quite
trivial to do, a matter of minutes. In fact a hell of a lot simpler
and way faster than getting the linux-2.6 source package to do
something for me.
The kernel-wedge lists do break from time to time but they are simple
to adjust and quick to rebuild.
And another advantage with kernel-wedge: You can easily take your
(maybe even prebuild) custom kernel and create udebs from it. I would
hate to have to add the SuSe or RH patch sets to the linux-2.6 source
to build kernel udebs for hardware that requires their patches.
> This, in my book, is not an example of good software engineering, and any
> proposal to help improve this should be worth considering.
Still not convinced. You do have some points but I see more drawbacks
and problems than advantages.
>> Is the space taken up by the installed modules your actual argument
>> for having finer grained module udebs?
>
> Nope, but then i told you that in the first mail already :)
Then you have presented no argument for splitting the module udebs and
several against.
You do have raised some points of improvement for kernel-wedge. Like
automaticaly adjusting to changed .config files or some other magics
you mentioned. And some points for building module udebs from within
linux-2.6 source although I don't agree with you there.
...
> The d-i team only needs to handle a list of the modules it want to include in
> this or that ramdisk, and all will work automatically.
Which has the same problem as the kernel-wedge configuration has now
when splitting the modules. When a module gets added or removed the
list has to be adjusted.
> Compared to the current situation, where the kernel porters have to update the
> config files, then build the kernel .deb, they have to pass NEW, then the d-i
Still has to happen.
> porters update the the kernel .udeb packages (one per arch), and the list of
Which is mostly just a new upload.
> modules which need to go into each ramdisk flavour, not mentioning that the
Which is pretty constant from my experience since the modules are
already split right into udebs.
> ramdisk package list has no support for per-flavour module selection, and you
> have to end up with stuff like the netboot64 list, which has as sole usage to
> add the ibm power hypervisor and virtualization modules, all an ugly mess.
Something to improve. No argument for or against your proposal.
> So, the new proposal would mean the porters decide which config options (and
> thus which modules), should be built, as well, in conjunction with the d-i
> folk, which of those modules may be usefull for d-i, disks, networks, input
> devices, display devices, the decision is pretty easy. The d-i team only needs
> to handle a single list of modules per ramdisk, which can be determined at d-i
> image built time, and not need to upload kernel wedge and the individual
> kernel .udeb packages for each arch, just because you want to add a single
> module you didn't think of the week before.
All modules will be build for a kernel usualy. At least a ton more
modules than D-I needs. So you need an extra config file that lists
the modules that D-I needs, which is the list kernel-wedge now has.
You didn't eliminate the need for a manualy tuned list of modules to
put into udebs. You just shifted the problem from A to B.
And then you duplicated the problem because D-I still needs a list of
modules to put into each ramdisks that they have to adjust for every
change in the kernels. And you need some meta udebs like net-drivers
that depend on all network driver modules for easy selection. That is
exactly the list of modules in kernel-wedge. You didn't eliminate the
kernel-wedge config files from D-I.
So now you have both kernel team and D-I team having to handle the
list of modules for the installer.
[whining removed]
>> No it isn't. The GPL only requires that you have the source and that
>> is available via the linux-tree package even for past versions no
>> longer in the archive.
>
> Yeah, but what about cases where old kernel versions are used by d-i while the
> kernel team already got ride of those ? There linux-tree is of no help to you.
You must mean upstream versions because for debian revisions
linux-tree (or rather linux-patch-debian-2.x.y) has exactly those
patches needed. And for upstream versions the
linux-tree/source/patch-debian packages get renamed and all that has
to be done by ftp-master is not to remove them before the kernel udebs
are updated.
>> >> Since I showed that new sections have only drawbacks you now suddenly
>> >> propose building udebs in the linux-2.6 source.
>> >
>> > Suddenly, i have been saying exactly that since january or earlier, go read
>> > the archive, i was even going to implement it, when i was hit by a barrage of
>> > hatefullness and kicked out of d-i, and threatened by Frans if i ever dared
>> > work on it.
>>
>> I'm talking about this thread.
>
> I think the world at large knows that i want the kernel .udebs to be built
I didn't.
[more whining removed]
MfG
Goswin
Reply to: