Re: Etch timeline is unrealistic because non-free firmware is NOT being dealt with
On Sat, Aug 12, 2006 at 10:32:54PM +0200, Goswin von Brederlow wrote:
> > Indeed. The proposal to split the packages file in a per flavour kernel
> > repository comes directly from the need to counterbalance this augmentation of
> > the number of packages.
>
> So instead of having 5 module udebs for my ONE generic kernel I get
> 200? For say amd64 it would be a big step back.
Indeed, but smaller ones.
> How many archs with flavours are we talking about anyway? I think m68k
> has 3 flavours and ppc some and every other archs has a generic
> flavour or drivers buildin already.
I know that the more exotic ones have many flavours. Also, the amount of
packages will be proportional of the non-builtin drivers. POwerpc has
currently 5 d-i flavours (or would have if the new d-i powerpc people did they
work correctly), powerpc, powerpc64, powerpc-miboot, prep and apus.
> > Also, do you really need a gazillion of not-for-your-hardware modules
> > installed ?
>
> All my hardware (that includes m68k) has absolutley 0 problems with
> that. They will be installed in D-I and they will be installed on the
> finished system. No point breaking my neck to get less installed
> during D-I.
>
> >> you want (save space). As for saving download bandwith, saving 1MB of
> >> udebs compared to the 500-2000MB debs of a normal install is quite
> >> incosequential. You might be optimizing at the wrong end here.
> >
> > Indeed.
>
> So why then do you want to split module udebs?
Because the current way of doing it is problematic. The d-i folk don't want to
give control of it to the kernel team, and the new proposal handled they
keeping the same and even better control of how to build the ramdisks, while
at the same time building the module .udebs from the common kernel source,
thus saving around 2 weeks of time the d-i folk need to adjust to a new abi or
version, thus making for more timely kernel security updates, and making the
work of the divers security teams easier.
> You just agreed that downloading the module udebs is inconsequential
> given their size. You agreed that more udebs increases the Index files
> which is bad.
Err, let's say we have 5 flavours like on powerpc, and each would have 200+
little module .udebs, which gives us Package numbers of 1000+. Worse if there
are more flavours. This is the part that joeyh objected to this plan of
spliting modules because of the fact that d-i loads three in memory copy of
the Packages file, which in turn prompted the idea of per-flavour
repositories.
If you look at the whole idea though, you find out that it is a really neat
solution to this problem, it cares for all the technical hurdles, and is
elegant and nice, and if you throw in the part that can be automated, it saves
work for everyone involved.
In my book, this makes it a good idea, or at least something that should be
tried, and not something you have to menace the implementator so he doesn't
dare pursue it.
> So all I'm left with is that you don't end up with as much modules in
> the ramdisk. Something I find irelevant unless you talk about the
> low-mem flavour.
And the fact that it is much easier to update config option and add and remove
new modules doesn't count. Naturally, you don't handle kernel .udebs. I did,
and it was a royal pain in the ass to handle those, it took me hours, for
stuff that was already done 10 times for the other flavours. And even a few
days ago, the powerpc64 flavour was still broken with the 2.6.17 kernels,
because some random module listed in the list of modules was not present.
The whole concept is of an extreme fragility, prone to break again and again,
cause lot of work for all involved, who become irritable, and bash on you when
you even mention it.
This, in my book, is not an example of good software engineering, and any
proposal to help improve this should be worth considering.
> Is the space taken up by the installed modules your actual argument
> for having finer grained module udebs?
Nope, but then i told you that in the first mail already :)
> ...
> >> You can already do that. Just because the modules are not copied into
> >> udebs at the end does not mean you can't pretend. Just imagine they
> >> get copied and do all your magic. After that you have a set of modules
> >> and the depmod file for them that D-I uses to split modules into
> >> udebs.
> >
> > "the depmod file that D-I uses to split modules into .udebs". You must be
> > living in another planet than me. Right now the modules are split manually, in
> > a stupid painstakingly repetitive and time-consuming way. And you can't even
> > per-arch/subarch override properly the default choice done in kernel-wedge.
>
> Last I used it, and that is some time ago, the dependencies where
> automatically checked on build. Maybe someone broke that but that
> could be repaired easily enough.
The dependencies, yes, with a manual override which you have to use in
mysterious cases. The actual module list is fully manually handled, in 2
places for each arch, and if you happen to not build a module which is present
in the kernel-wedge list, you are hosed. This is why there is no apus flavour
anymore.
> ...
> >> the kernel-team. Currently building udebs from the linux-2.6 source
> >> would add another indirection to the build process, not remove one.
> >
> > Another indirection ? It would mean that in a single upload, all kernel
> > related .debs and .udebs are built. Furthermore, it allows to autobuild the
> > kernel, which is not currently possible for the .udebs, which means 10+ times
> > human intervention, uncoordinated uploads, uncoordinated NEW handling, random
> > delays for each arch, and that the porters are left alone with random breakage
> > introduced by untested changes of kernel-wedge, except for the pet arch of the
> > d-i leaders.
>
> The indirection is that every change in the module split needed by the
> D-I team would have to through the kernel-team and would require a
No, because you have finegrained control. You build a dependency tree where
the nodes are individual CONFIG file option you deem important for d-i, and
the tool automatically create .udebs including all the modules linked to that
config option (if there are more than one), and all dependencies are set
right.
This would be the sole province of the kernel team, who handles the kernel
patches and configuration files.
The d-i team only needs to handle a list of the modules it want to include in
this or that ramdisk, and all will work automatically.
Compared to the current situation, where the kernel porters have to update the
config files, then build the kernel .deb, they have to pass NEW, then the d-i
porters update the the kernel .udeb packages (one per arch), and the list of
modules which need to go into each ramdisk flavour, not mentioning that the
ramdisk package list has no support for per-flavour module selection, and you
have to end up with stuff like the netboot64 list, which has as sole usage to
add the ibm power hypervisor and virtualization modules, all an ugly mess.
So, the new proposal would mean the porters decide which config options (and
thus which modules), should be built, as well, in conjunction with the d-i
folk, which of those modules may be usefull for d-i, disks, networks, input
devices, display devices, the decision is pretty easy. The d-i team only needs
to handle a single list of modules per ramdisk, which can be determined at d-i
image built time, and not need to upload kernel wedge and the individual
kernel .udeb packages for each arch, just because you want to add a single
module you didn't think of the week before.
> full source upload with days of buildd time. Which is quite insane
> just to move a module from package A to package B so the floppy image
> doesn't overflow while things get worked out.
Indeed, which is why my proposal is so neat, and obsolet that kind of
thinking. And it is clear by the above that you have not yet understood the
full extent of this proposal, after what, 5 or so mail exchanges about the
subject, but you seem more interested to proof that i am wrong, than to try to
understand.
Notice also that i proposed the creation of a single linux-2.6-udeb
package, which would contain all arches, at debconf, as an intermediate
proposal, and that this was dismissed out of hand.
> >> > Compare this to the current situation, favored by the installer team, probably
> >> > just to oppose me, where every porter has to redo the module selection job by
> >> > hand, where none of them are informed about the problems faced by the others,
> >> > where a slight indisponibility of one of the porters result in thee brokeness
> >> > of the related build, and accusations of lazyness by the d-i leaders, and you
> >> > see how this would be dissatisfying, and how the d-i leaders in their
> >> > pettiness see my technical proposals as an attack against their supremacy.
> >>
> >> The same can be said to happen for the kernel-team. Changing
> >> responcibility to another team does not magicaly solve problems. I
> >> somewhat feel that you just say all this (moving udebs into the
> >> linux-2.6 source) because you were kicked out of D-I.
> >
> > No, the kernel team builds from a single package, for all arches, so there
> > needs to be coordination between the maintainers/porters, we care about
>
> Which in no way garanties that there is such coordination. The need
> for coordination in the D-I team is just as great as in the kernel-team.
yeah, the difference being that :
1) the kernel team is able to coordinate all arches, while the d-i team
accuses porters of laziness, and goes into full bashing campaign against
individual porters who dare say it could be made better an other way.
2) the d-i team resist the idea of moving all kernel .udebs in a single
source package, because then they would be forced to fix all arches, while
just uploading x86 and a few pet arches like they do now, and let the other
arches bitrot, especially when the porter is known to be away or otherwise
busy.
> > other-people arches, and we never accused people of being lazy and other ugly
>
> Well, YOU do and you are part of the kernel team.
Please quote me where is said that the porters where lazy ? Notice also that i
willingly investigated issues on arches i have no relationship to or knowledge
whatsoever.
> > things. We managed to do 1-day uploads for all arches except m68k (because of
> > the way upstream works there).
>
> I would rather have 7-day uploads and have the kernel DFSG free and
> distributable. Two things that it is not and known since before sarge
> strictly speaking.
These are two different issues. The 1-day upload needs packaging work and
coordination, while the DFSG free and distributable issue needs upstream kind
of work and legal work.
Also notice that i approached broadcom over the tg3 licence, while everyone
said that i was crazy and they would never reply me, and thanks to the work i
started and later Andres Salomon and others completed, the tg3 driver is now
distributable (but still non-free) in both debian and upstream.
So, i am impatiently awaiting your patches to solve this issue.
> > Err, they break d-i compatibility every so often anyway, and having kernel modules
> > without the corresponding kernel .debs is a GPL violation anyway.
>
> No it isn't. The GPL only requires that you have the source and that
> is available via the linux-tree package even for past versions no
> longer in the archive.
Yeah, but what about cases where old kernel versions are used by d-i while the
kernel team already got ride of those ? There linux-tree is of no help to you.
> >> Since I showed that new sections have only drawbacks you now suddenly
> >> propose building udebs in the linux-2.6 source.
> >
> > Suddenly, i have been saying exactly that since january or earlier, go read
> > the archive, i was even going to implement it, when i was hit by a barrage of
> > hatefullness and kicked out of d-i, and threatened by Frans if i ever dared
> > work on it.
>
> I'm talking about this thread.
I think the world at large knows that i want the kernel .udebs to be built
from linux-2.6, why do you think Frans got me quicked out ? Because he
considered that me even suggesting such was an inadmissible affront to his
authority.
> >> So far two totaly unrelated features while you try to make it look
> >> like the later supports the former.
> >
> > Because you fail to understand the whole picture, it seems such to you, please
> > read the past discussions on the topic, and try to understand the proposal
> > with an open mind.
>
> You assume knowledge about past discussions. You should have included
> a link to your proposal when you started mentioning features of it the
> first time.
Google for it, you can do this as well as i.
> ...
> > Maybe, bvut do you really believe that what they did is right ? Would you
> > really be able to work with such persons without feeling ashamed of yourself
> > for having said nothing ? Or even like some encouraged them ?
>
> The only thing I believe is that you acting wrong NOW and you keep
> acting wrong NOW. I wasn't involved in your D-I incident and I refuse
> to be drawn into it.
So, then let me rant, and ignore my rant, instead of prolonging the issue as
you did in those past 5 or so mails. You clearly are interested in it, because
you keep aluding to it, just remove what you don't like and that would be the
end of it.
> > As for perception, anyones perception of me is anyway hosed, and half of
> > debian if not more has me in their killfill, and just wish i would go away and
> > be silent, so what have i too lose ? That they kick me out ? Let it come to
>
> You loose any chance of getting back in or anyone listening to you.
and do you really think people would some day say, hey, let's listen to sven
again, they will just add me to the killfill and forget about it.
> And just to let you know, from now on I will ignore all mails that
> include the D-I incident or you being mistreated. If you can't keep
Then why do you mention it again and again and again ?
> to the technical discussion then I'm sorry but I want to do some
> productive work and not wallow in self pitty (be that justified or not
> I simply don't care).
Well, the problem is that this incident (or series of incident ranging over
more than a year), are intimely related to the technical issues discussed
here. And any solution to this problem will have to take into account the
irrational and vengeful opposition of the d-i leadership to this idea, only
because i dared propose it, and they felt menaced in their leadership.
Friendly,
Sven Luther
Reply to: