Re: What's Debian's /usr/src policy
Hi,
I know I am raking up ancient mud, but I have just come back
and am catching up with my mail while waiting to here from the
french, sedish, and german consulates.
As to the reasons for packaging up kernel sources. I offer:
a) Most users are encouraged to compile their own kernels,
espescially if there are hardware problems (probing for a
non-existent hardware freezes the machine), or to get a faster,
smaller, less error prone kernel configured to the local machine.
Espescially for new users, it makes sense for Debian to package
known good kernel versions, and make them easy to access.
Experienced Linux folk download from the horses mouth, but
novices find being able to use dpkg-ftp or dft to get kernel
sources very comforting. A deb file is justified since there are
mechanisms in place to get .deb files to users as seamlessly as
possible.
Since compiling a kernel is what most Debian users are
exhorted to do, no matter what novices the are in Unix systems,
this separates the kernel image package from the rest of our
packages, where end user compilation is the exception rather than
the rule.
This reason is enough not to discontinue the practice for
purely aesthetic reasons.
b) There have been, and may again be, patches applied and tested by
the kernel image maintainer that add stability/debian specific
features to a kernel, or to backpatch desred deatures from e
newer kernel that by itself is too unstable to get into Debian
(Yes, I think we are about a higher quality distribution). The
deb file is the most efficient way of disseminating these
improvements.
>>"Dale" == Dale Scheetz <dwarf@polaris.net> writes:
Dale> I'm sorry, but my experience has been very different. I too have
Dale> tried both. Maybe my timing has been poor, but both times I
Dale> tried there was a mismatch between the kernel-package package
Dale> and the kernel-source package and after much futsing around
Dale> resulted in failure.
Could you please file bug reports? kernel-package should be
totally independent of where you got the sources from. I can only say
there must be something very --- different (was that polite
enough?) about your setup. I routinely compile kernels with that, and
there are novices on debian-user that also have reported success. I
no longer think that technical deficiencies in kernel-package hold
water.
Dale> Kernel source should be provided the way we provide source for
Dale> all other packages. You should be able to unpack it with
Dale> dpkg-source -x and run dpkg-buildpackage and build a kernel to
Dale> your spec and construct a .deb package from that result. (Note,
Dale> I have no real problem with a kernel-binary package, although
Dale> personally I would have no use for it, I can understand others
Dale> point of view in wishing to "manage" their kernels with dpkg)
See above for why I think kernel source packages are different
from other packages; and if only experieced developers were the
target audience for kernel compilations, I would agree with you.
Every debian user knows how to install .deb files --either
from a CDROM, or from a ftp site -- and there are fromtends and
convenience packages to ease that task. using dpkg -x is way more
complex.
>> Why does libc6 depend on kernel-header ?
>> >
>> > I don't know either, but it is on the top of my list of things I
>> need to understand as the new maintainer. It was my understanding
>> that the way we deal with kernel headers was supposed to free the c
>> library from the headers. I don't know that anything has changed in
>> that reguard. I'll let you know what I find asap.
>>"Christoph" == Christoph Lameter <chris@waterf.org> writes:
Christoph> I want to be able to change the kernel-headers a program is
Christoph> compiled with. Certain tools (especially in 2.1.X) are
Christoph> dependant on a certain kernel version. Nothing wrong with
Christoph> providing the default as 2.0.33 f.e. (why 32??).
Very few programs really need specific kernel headers. For the
few that do, use -I/usr/src/kernel-headers-2.0.33.
libc development package need to include files found in
kernel-sources to insulate most systems from the vagaries of bleeding
edge kernel sources (you *do* need to include these header files,
since one can't even include <errno.h> without having a valid
<linux/errno.h>)
There is a FAQ I seem to be posting continually arguing for
not having libc development use any old (or new, or random) kernel
headers; and libc5 did include a static set of kernel include files.
Unfortunately, the kernel header files are getting to be quite
architecture dependent, and hence if libc development packages
continued to include kernel headers explicitly, we would need
different headers for different architectures, resulting in
libc6-dev-i386, libc6-dev-m68k, et. al.
The current solution is to make the libc development package
depend on one particular, stable, well tested, *supported* kernel
headers, namely version 2.0.32, and let the kernel-headers package
handle architecture dependancies (which it had to anyway).
This is a working technical solution to having libc
development package contain/depend on a well known static set of
kernel headers (insulating the vast majority of programs that are not
closely tied to kernel version specific internal data structures),
while allowing the kernel headers to vary from architecture to
architecture, and still allowing device driver authors from having
any set of kernel headers they want on the machine through the simple
artifice of adding a -I flag to the compilation flags.
I think this solution works, and is an elegant solution.
Dale> This exaplanation is inadequate as well. While shrinking the
Dale> size of a diff file helps the developer who pays by the
Dale> byte/minute for access, it creates a problem for the user which
Dale> is not necessary. Forcing the coupling of two packages via
Dale> depends when the two packages are only used together makes
Dale> installation and upgrades one step more complex than is
Dale> necessary in a way that adds to package bloat when it isn't
Dale> necessary. (BTW it's libc6-dev that is coupled to
Dale> kenrel-headers)
I don't get this. You mean a well marked dependency is still
too complex? Despite the reasons given above?
Dale> Still unconvinced,
Sorry. I did my best. (I am including the FAQ mentioned above,
in hopes this sheds additional light)
manoj
--
I think we're all Bozos on this bus.
Manoj Srivastava <srivasta@acm.org> <http://www.datasync.com/%7Esrivasta/>
Key C7261095 fingerprint = CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E
$Id: README.headers,v 1.3 1997/06/25 07:33:27 srivasta Exp $
This is the Debian GNU/Linux prepackaged version of the Linux kernel
headers. Linux was written by Linus Torvalds
<Linus.Torvalds@cs.Helsinki.FI> and others.
This package was put together by Simon Shapiro <Shimon@i-Connect.Net>, from
sources retrieved from directories under
ftp.cs.helsinki.fi:/pub/Software/Linux/Kernel/
This package contains the Linux kernel header files (also contained
in libc5-dev packages).
This document contains comments from Linus Torvalds (made in
an ``off-the-cuff'' personal email) to help clarify the rationale
behind the Debian way of handling symlinks, but this should not be
seen as an official policy statement by Linus. I'm attaching a
disclaimer in his own words.
The only reason that Linus's message is quoted in here is that
he can explain the technical reasons with far more lucidity than I
can, and now that I have permission to include his mail, I am
removing most of my far less facile efforts in that regard.
----------------------------------------------------------------------------
>> "David" == David Engel <david@sw.ods.com> said on Mon, 24 Feb 1997
>> "Linus" == Linus Torvalds said on Mon, 24 Feb 1997
David> Hi Linus,
David> No matter how well we try to explain ourselves, the symlinks issue
David> keeps coming up. Would you mind if we used your message below in
David> our responses?
Linus> Sure. Don't make it "the word of God" - please point out that
Linus> it was a off-the-bat personal reply to a question concerning
Linus> this, and while I'm more than happy to have the email
Linus> circulated it shouldn't be seen as a "official" document in any
Linus> way..
Linus> Linus
---------------------------------------------------------------------------
The headers were included in libc5-dev after a rash of very
buggy alpha kernel releases (1.3.7* or something like that) that
proceeded to break compilations, etc. Kernel versions are changed
far more rapidly than libc is, and there are higher chances that
people install a custom kernel than they install custom libc.
libc6 includes it's own version of /usr/include/linux and
friends form the beginning (that is, this is no longer a Debian only
feature, the upstream version has moved to this scheme as well).
>> "Linus" == Linus Torvalds said on Wed, 22 Jan 1997:
Linus> The kernel headers used to make sense exporting to user space,
Linus> but the user space thing has grown so much that it's really not
Linus> practical any more. The problem with Debian is just that they
Linus> are different, not that they are doing anything wrong. That
Linus> leads to differences between the distributions, and that in
Linus> turn obviously can result in subtle problems.
Linus> As of glibc, the kernel headers will really be _kernel_
Linus> headers, and user level includes are user level
Linus> includes. Matthias Ulrich did that partly because I've asked
Linus> him to, but mainly just because it is no longer possible to try
Linus> to synchronize the libc and the kernel the way it used to
Linus> be. The symlinks have been a bad idea for at least a year now,
Linus> and the problem is just how to get rid of them
Linus> gracefully. Personally, I'm counting on glibc, which we are
Linus> already using on alpha.
Linus> Just to give you some idea of exactly why the includes really
Linus> can't be handled by simple symlinks: the main problem is
Linus> version skew. Lots of people want to upgrade their library
Linus> without affecting the kernel, and probably even more people
Linus> want to be able to upgrade their kernel without affecting their
Linus> compilation environment. Right now doing that has been
Linus> extremely fragile.
Linus> Just to give _one_ example of why the symlinks are bad: NR_OPEN
Linus> and "fd_set". I have had no end of problems making NR_OPEN
Linus> larger in the kernel, exactly _because_ of the damn
Linus> sym-links. If I just make NR_OPEN larger (the right thing to
Linus> do), the problem is that people with old libraries will now
Linus> compile against a header file that doesn't match the library
Linus> any more. And when the library internally uses another NR_OPEN
Linus> than the new program does, "interesting" things happen.
Linus> In contrast, with separate header files, this doesn't make any
Linus> difference. If I change NR_OPEN in the kernel, the compilation
Linus> environment won't notice UNTIL the library and associated
Linus> header files are changed: thus the user will continue to compile
Linus> with the old values, but because we'll still be binary
Linus> compatible, the worst thing that happens is that new programs
Linus> won't take advantage of new features unless the developer has
Linus> upgraded his library. Compare that to breaking subtly.
Linus> NR_OPEN is just _one_ example, and actually it's one of the
Linus> easier ones to handle (because the only thing that really makes
Linus> much of a difference when it comes to NR_OPEN is the fd_set
Linus> size - but it certainly bit some people). Another major problem
Linus> is name-space pollution: the POSIX/ANSI/XOpen rules are not
Linus> only complex, but they are actually contradictory too. And the
Linus> kernel header files really can't reasonably support all of the
Linus> intricacies very cleanly.
Linus> One specific example of why we want separate header files for
Linus> libraries and kernel is offered by glibc: Matthias wanted to
Linus> have a "sigset_t" that will suffice for the future when the
Linus> POSIX.1b realtime signals are implemented. But at the same time
Linus> he obviously wants to be able to support programming on
Linus> Linux-2.0 and the current 2.1 that do not have that support.
Linus> The _only_ reasonably clean way to handle these kinds of
Linus> problems is to have separate header files: user programs see a
Linus> larger sigset_t, and then the library interaction with the
Linus> kernel doesn't necessarily use all of the bits, for
Linus> example. Then later, when the kernel support is actually there,
Linus> it's just a matter of getting a new shared library, and voila,
Linus> all the realtime signals work.
Linus> The symlink approach simply wouldn't work for the above: that
Linus> would have required everybody who uses the library to have a
Linus> recent enough kernel that whatever magic all the above entails
Linus> would be available in the kernel header files. But not only
Linus> don't I want to pollute the kernel header files with user level
Linus> decisions, it's actually possible that somebody wants to run
Linus> glibc on a 1.2.x kernel, for example. We _definitely_ do not
Linus> want him to get a 32-bit sigset_t just because he is happy with
Linus> an old kernel.
Linus> Anyway, this email got longer than intended, but I just wanted
Linus> to make clear that the symlinks will eventually be going away
Linus> even in non-Debian distributions. Debian just happened to do it
Linus> first - probably because Debian seems to be more interested in
Linus> technical reasons than any old traditions. And technically, the
Linus> symlinks really aren't very good.
Linus> The _only_ reason for the symlinks is to immediately give
Linus> access to new features in the kernel when those happen. New
Linus> ioctl numbers etc etc. That was an overriding concern early on:
Linus> the kernel interfaces expanded so rapidly even in "normal"
Linus> areas that having the synchronization that symlinks offered was
Linus> a good thing.
Linus> However, the kernel interfaces aren't really supposed to change
Linus> all that quickly any more, and more importantly: the technical
Linus> users know how to fix things any way they want, so if they want
Linus> a new ioctl number to show up they can actually edit the header
Linus> files themselves, for example. But having separation is good
Linus> for the non-technical user, because there are less surprises
Linus> and package dependencies.
Linus> Anyway, something like the patch that David suggested will
Linus> certainly go in, although I suspect I'll wait for it to become
Linus> "standard" and the glibc first real release to take place.
Add to that the fact that few programs really need the more
volatile elements of the header files (that is, things that really
change from kernel version to kernel version), [before you reject
this, consider: programs compiled on one kernel version usually work
on other kernels].
So, it makes sense that a set of headers be provided from a
known good kernel version, and that is sufficient for compiling most
programs, (it also makes the compile time environments for programs
on Debian machines a well known one, easing the process of dealing
with problem reports), the few programs that really depend on cutting
edge kernel data structures may just use -I/usr/src/linux/include
(provided that kernel-headers or kernel-source exists on the system).
Most programs, even if they include <linux/something.h>, do
not really depend on the version of the kernel, as long as the kernel
versions are not too far off, they will work. And the headers
provided in libc5-dev (and libc6-dev) are just that.
libc5-dev is uploaded frequently enough that it never lags too
far behind the latest released kernel. libc6 has totally disconnected
the included headers from kernel headers.
There are two different capabilities which are the issue, and
the kernel-packages and libc{5,6}-dev address different ones:
a) The kernel packages try to provide a stable, well behaved kernel
and modules, and may be upgraded whenever there are significant
advances in those directions (bug fixes, more/better module
support, etc). These, however, may not have include files that
are non-broken as far as non-kernel programs are concerned, and
the quality of the development/compilation environment is not the
kernel packages priority (Also, please note that the kernel
packages are tied together, so kernel-source, headers, and image
are produced in sync)
b) Quality of the development/compilation environment is the priority
of libc{5,6}-dev package, and it tries to ensure that the headers it
provides would be stable and not break non-kernel programs. This
assertion may fail for alpha kernels, which may otherwise be
perfectly stable, hence the need for a different set of known-good
kernel include files.
--
TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to
debian-devel-request@lists.debian.org .
Trouble? e-mail to templin@bucknell.edu .
Reply to: