Re: Etch timeline is unrealistic because non-free firmware is NOT being dealt with
Sven Luther <sven.luther@wanadoo.fr> writes:
> On Sun, Aug 13, 2006 at 03:47:13AM +0200, Goswin von Brederlow wrote:
>> > The whole concept is of an extreme fragility, prone to break again and again,
>> > cause lot of work for all involved, who become irritable, and bash on you when
>> > you even mention it.
>>
>> I did it when working on the amd64 D-I for sarge. I found it quite
>> trivial to do, a matter of minutes. In fact a hell of a lot simpler
>> and way faster than getting the linux-2.6 source package to do
>> something for me.
>
> Naturally, amd64 being one of the mainstream arches the d-i team cares about.
No they didn't since amd64 was not official or even liked. It didn't
even exist before I cloned the i386 config and adjusted them to fit.
> Now, please provide patches for resurecting the apus flavour, and then we can
> speak again.
>
>>
>> The kernel-wedge lists do break from time to time but they are simple
>> to adjust and quick to rebuild.
>>
>> And another advantage with kernel-wedge: You can easily take your
>> (maybe even prebuild) custom kernel and create udebs from it. I would
>> hate to have to add the SuSe or RH patch sets to the linux-2.6 source
>> to build kernel udebs for hardware that requires their patches.
>
> This could be easily be solved by adding the module info to the source package
> in some way, or even integrate them into kernel-package.
Into the SuSe/RH source? Not likely. And kernel-package doesn't deal
with prebuild binaries.
>> > This, in my book, is not an example of good software engineering, and any
>> > proposal to help improve this should be worth considering.
>>
>> Still not convinced. You do have some points but I see more drawbacks
>> and problems than advantages.
>
> At least you had the honestity to discuss it, while other rejected it out of
> hand, with patronizing and aggresive language.
>
>> >> Is the space taken up by the installed modules your actual argument
>> >> for having finer grained module udebs?
>> >
>> > Nope, but then i told you that in the first mail already :)
>>
>> Then you have presented no argument for splitting the module udebs and
>> several against.
>
> Several ?
>
>> You do have raised some points of improvement for kernel-wedge. Like
>> automaticaly adjusting to changed .config files or some other magics
>> you mentioned. And some points for building module udebs from within
>> linux-2.6 source although I don't agree with you there.
>
> Why not ? It is the only way we can bring down the delay for security builds
> with abi change to the minimum time, and the less human intervention.
I don't see security for the installer as a top priority. It is not
like people will burn their daily security patched D-I image. It is
not like the security team does do security releases for D-I. For
stable that seems to get totaly ignored.
D-I in testing/sid, especially the daylies, are for developing. Not to
run a secure and stable system. The have bugs, they have security
flaws, they might not work at all. That is what unstable is.
>> > The d-i team only needs to handle a list of the modules it want to include in
>> > this or that ramdisk, and all will work automatically.
>>
>> Which has the same problem as the kernel-wedge configuration has now
>> when splitting the modules. When a module gets added or removed the
>> list has to be adjusted.
>
> Err, no, because the list would be handled in the debian-installer package or
> its devel tree in the svn repo, and not inside kernel-wedge.
Moving the problem from A to B. Doesn't matter what svn repository it
is in.
> Notice that we already have the mapping of module .udeb packages to d-i images
> in d-i proper, which needs to be done anyway in addition to the
> kernel-wedge/linux-xxx-di work.
>
> Furthermore with the current situation, and given that not all modules are
> built the same on all arches, and that binary size vary per arch, you end up
> in an unholy mess when trying to build space constrained ramdisks, like the
> floppy ones for example.
That just means that you need per arch lists of modules. And
kernel-wedge has that. Having each module as udeb doesn't change the
fact that you have a unholy mess what modules to put in the image.
>> > Compared to the current situation, where the kernel porters have to update the
>> > config files, then build the kernel .deb, they have to pass NEW, then the d-i
>>
>> Still has to happen.
>
> Indeed, but that is the only part that needs to be done, not the rest.
>
>> > porters update the the kernel .udeb packages (one per arch), and the list of
>>
>> Which is mostly just a new upload.
>
> WRONG. it is at best an upload *PER ARCH*, done by *ONE DIFFERENT PERSON AT A
> DIFFERENT TIME* for each arch.
>
> Furthermore, the build, failure, check what the heck kernel module has changed
> name or dissapeared, or whatnot, is a *ROYAL PAIN*, and very much time
> consuming., especially on slow arches with lot of flavours.
A factor of 10000 or so faster than building the linux-2.6 source. As
combined time the kernel-wedge time is neglible.
> Furthermore, given that you need to install the kernel image package, you
> can't even start to autobuild that without hosing the autobuilders.
Last I use kernel-wedge, and this is ~2 years ago now, you could build
all kernel udebs on a single arch with the help of a little wrapper
script. Did somone break that?
Rewrite that script.
>> > modules which need to go into each ramdisk flavour, not mentioning that the
>>
>> Which is pretty constant from my experience since the modules are
>> already split right into udebs.
>
> They are split into the right .udebs for the x86 and other mainstream cases
> the kernel-wedge maintainer cares about.
Even if they are not split right you don't change what udebs go into
the ramdisk. You fix the split itself (which you already have to do,
that work is already accounted for). So this part of the job is a NOP.
>> > ramdisk package list has no support for per-flavour module selection, and you
>> > have to end up with stuff like the netboot64 list, which has as sole usage to
>> > add the ibm power hypervisor and virtualization modules, all an ugly mess.
>>
>> Something to improve. No argument for or against your proposal.
>
> Because you miss that my proposal makes this whole issue obsolet and
> non-existent.
No it doesn't. You still have to list the modules that get into the
ramdisk. That list doesn't magically build itself just because you
parse .config.
>> > So, the new proposal would mean the porters decide which config options (and
>> > thus which modules), should be built, as well, in conjunction with the d-i
>> > folk, which of those modules may be usefull for d-i, disks, networks, input
>> > devices, display devices, the decision is pretty easy. The d-i team only needs
>> > to handle a single list of modules per ramdisk, which can be determined at d-i
>> > image built time, and not need to upload kernel wedge and the individual
>> > kernel .udeb packages for each arch, just because you want to add a single
>> > module you didn't think of the week before.
>>
>> All modules will be build for a kernel usualy. At least a ton more
>> modules than D-I needs. So you need an extra config file that lists
>> the modules that D-I needs, which is the list kernel-wedge now has.
>
> Indeed. The difference is that the knowledge about this is disseminated in at
> least 3 different places, and that the kernel-wedge maintainer cares about no
> more than 3-5 arches compared to the 12 or so official debian arches (at least
> the powerpc case repeteadly broke every so often).
I only konw about one place: kernel-wedge.
Where else?
> The reasoning behind this analysis is that the kernel porter need to make the
> decision about each new config option anyway, that the kernel has good
> infrastructure in place for separating that choice in between debian-global
> choices, per-arch choices, per variant (historically called subarches), and
> per flavour.
>
> The decision on what modules to offer as .udebs for d-i is rather trivial,
> once the decision on how to group them in .udebs is out of the way, and the
> kernel team is as well if not better placed to take that decision. You need
> the network modules, the disk modules, the display modules and the input
> modules, and another set of assorted modules needed for each machine to run
> correctly, like the fan control modules on apple hardware for example.
Sure. That part is easy. So easy it is merged into kernel-wedge
because D-I can just as well decide that too.
> Then the d-i team can cherry pick among those .udebs and put into each flavour
> those they need, being able to do different variants for different hardware
> much easier than the current step which takes a couple of week or so.
Which is what kernel-wedge does now. You still have that decision at
exactly the same spot you have it now. You just moved it from a
kernel-wedge upload to a d-i upload. Ok, 1 upload instead of 12 Some
progress. But still, all 12 ports have to change D-I first before the
upload. So in fact you haven't saved anything but just made every
single arch block the next D-I upload instead of their own
kernel-wedge.
>> You didn't eliminate the need for a manualy tuned list of modules to
>> put into udebs. You just shifted the problem from A to B.
>
> Indeed, i split the problematic into two different cases, and assigned those
> cases to the people most competent in solving them, in a way which cuts the
> requirement for human intervention to the minimum, both time wise and with
> regard to the number of actors, as well as simplifying the whole issue by an
> order of magnitude.
You have split the work from one team to two. You have not reduce the
work. One could say you have double the work because now twice as many
people have to look at every change.
A new module gets added, kernel team has to decide if it becomes an
udeb, D-I team has to look again and decide where to put it. And so
on.
> The cost is small, the only drawback would be the relative augmentation of the
> number of .udebs packages, but not the global size of them, and a little work
> on the infrastructure side, which is out of our control.
>
> Even the package size augmentation is minimal, since you said yourself that on
> most mainstream machines, bandwidth and memory size are perfectly able to
> handle it, and on those arches which have problems with memory size, most
> modules will probably be builtin, and thus the amount of needed .udebs is also
> minimal.
>From what I understand all your proposal does is shift work around,
splits it up, fragment it. I don't see any work reduction.
On the other hand I see drawbacks for prebuild kernel images, the DAK,
the ram requirement for anna/udpkg (a factor of 8-40 in tripple
replica of the index files) and most of all the user when selecting
modules.
So I have to agree with the D-I team. It isn't a viable proposal.
> But i think we can agree that nobody regrets the current state of the kernel
> packages, nor of the single day uploads, which has brought debian out of the
> always-months-outdated kernel era into a more modern and responsive one.
I do. It is a mess that nobody outside the kernel team understands.
The old way was way easier to grasp. Maybe not better but simpler.
Especialy the incompatibility with make-kpkg to build custom kernels
sucks. You can't build any kernel that requires an *-extra patch with
make-kpkg.
MfG
Goswin
Reply to: