Re: Cleaning up kernel's headers for userspace

[Tim Yamin <plasmaroo@gentoo.org>]

Hi David,

> The Linux kernel makes a complete abortion of exporting stuff to
> userspace in its headers. I'm trying to clean this up -- I've
> implemented a 'make headers_install' in the kernel tree which copies out
> selected headers which are supposed to be user-visible, and runs them
> through unifdef if appropriate.

Yeah, this has been an ongoing problem for us; a lot of apps (badly written
or not we don't really care, end result is users want things to compile)
require linux/ often with interesting collisions with glibc depending on
include order.

The problem is multi-layered:

* Apps need kernel headers
* Headers need to be "complete" (in the sense that, if something such as
  a struct requires another kernel struct, it all needs to be there).
* Apps also need glibc and including both leads to plenty of fun. I find
  it amusing how the glibc headers include <linux/...> and yet when we
  sent some patches to solve the issue we got told that glibc must not
  do so by upstream ;)

> I've also added a 'make headers_check' which currently only does a
> primitive check that the exported headers don't try to include anything
> which _wasn't_ selected for inclusion, but hopefully can be extended to
> check that each header is compilable on its own, and check for namespace
> pollution, etc. 

So, I might be misunderstanding but what that does right now is just
grok the headers and strips out anything inside __KERNEL__. Yeah, that
works. But I think it's a bit redundant. If it's inside #ifdef
__KERNEL__ it shouldn't be a problem anyway because thanks to the wonders
of a pre-processor your user-app just never even sees it and IMO one
should keep things in the headers for completeness (problem solving is
easier that way).

The reason we go with this approach is transparency; rather than
stripping things out the "Gentoo way" of doing them is to play with the
jigsaw pieces. Some things need to be __KERNEL__ that aren't. For
others it's the opposite. Here and there we do namespace collision
fixes some of which happen via automated sed-fu (e.g. u8 -> __u8).

This is done with a patchset on top of the "raw" headers, ofwhich we
then do a raw copy (no export magic). Being a source oriented distro it
obviously makes sense for us to keep things in a way we can debug
them most easily if things go wrong.

The other problem is the vastness of our architecture support - we
do from x86 to m68k and all those headers need to work -- hence
hand-picking really never was seen as the viable option as it means
a *lot* of work between upstream releases. Our current way you just
fix a few conflicts with the patches have when you move from say 2.6.16
to 2.6.17, and run test-cases to see if any new gremlins have
appeared.

> I've also committed a few fixes so that the PowerPC kernel actually
> passes the 'make headers_check'. This is all in a git tree at
> git://git.infradead.org/~dwmw2/headers-2.6.git -- browsable at
> http://git.infradead.org/?p=users/dwmw2/headers-2.6.git 
> 
> As maintainers for glibc and/or kernel headers in various distributions,
> I'd like to solicit your feedback, and possibly also your assistance.
> Does this look like it's going to address your needs and give you
> something which you can just drop into /usr/include/ and use as it is?
> If not, what's wrong with it and what can we do about it?

Personally I think the approach is problematic since you'll end up
with the same problem as that with linux-libc-headers: basically, you
get a header tree and the diff between that and "raw" tree is pretty huge
and it needs a lot of work to maintain (for example, if the architecture
maintainers decide to add asm-foo/blah.h which requires asm-foo/baz.h you
now need to include both and make sure things play nice...)

Just exporting all the headers raw but properly fixing the #ifdef
__KERNEL__ logic is the easiest solution in my opinion, after all, that's
what the whole __KERNEL__ system is supposed to be there for ;) -- and
when things do break one can just fix that specific problem.
 
> Note that I'd rather not have a discussion about moving user-visible
> headers into a separate directory right now. As long as we mark them for
> export as we do at the moment, that's the interesting part. Linus has
> already vetoed a suggestion that we move the public headers into a kabi/
> directory, and I'd like to go back to him with that suggestion _after_
> we've cleaned it all up and hopefully abolished the use of '#ifdef
> __KERNEL__', in the hope that he'll change his mind. We'll have a
> clearer idea of how we might want to organise things under kabi/ once
> we've cleaned it all up and we know what we have left, anyway.

I agree with Linus' decision here -- having two sets of headers is a real
pain to maintain. You've got to add things to two files and it means double
the work. The undef solution fixes this somewhat but you still end up with
the same end-result: a filtered tree which is quite different from the "raw"
tree.

Look back at 2.4 -- everything worked with the raw set of headers because
maintainers took an active interest in fixing headers for userspace with
one easy-to-maintain set. What I think that needs doing for 2.6 is starting
to go through the all the crap that's built up over the months and making
the "#ifdef __KERNEL__"'s do their job properly.

Anyway, my $0.02 -- if I sound like the grumpy gnome then just ignore me
but this solution wouldn't really float our boat as things stand :)

HTH.