Re: Cleaning up kernel's headers for userspace

[David Woodhouse <dwmw2@infradead.org>]

On Mon, 2006-04-24 at 20:18 +0000, Tim Yamin wrote:
> That raises the question though, how does one pick what headers to ship and
> not ship? Not ship headers that are totally __KERNEL__ or some other scheme?

I've done only a rough cut so far. I've started by dropping headers
which are totally __KERNEL__, and I've dropped headers which have _no_
__KERNEL__ because it's obvious that they're all kernel-private stuff.
And I've dropped asm/atomic.h, of course, because anyone using that was
totally broken.

I'm less interested in the details of which headers we include and
exclude, and I'm more interested that we have a consensus and that we
get a list which is agreed upon upstream, and which people can rely upon
across distributions and architectures.
> 
> Right, so I probably misunderstood. Thanks for clearing this up and
> bringing in the role of arch maintainers: if they're responsible for
> maintaining their own little bit rather than an external repo then
> and external maintainers that nag then yes, of course, it'll only better
> the situation.

Not just arch maintainers. Everyone who touches user-visible stuff, if
they're working on sockopts, ioctls, etc. 

> > That hasn't happened, though. And a large part of that is because the
> > problems aren't _obvious_. Hence the tools which make the problems
> > obvious by removing the __KERNEL__ and the unneeded files.
> 
> Well, you get a compile failure either way? Unless your muppet has done
> #define __KERNEL__, of course, which is just a silly thing to do. I think
> part of the confusion is that you're looking at this from an angle of a
> kernel dev (i.e. "is this header kosher or do we have a problem") and
> I'm looking at it from a distro point of view ("if there's a header
> problem it'll show up regardless and then we can fix it").

No, the angle of the kernel developer is just to ignore the problem
because it doesn't bother us at all. I maintain the glibc-kernheaders
package in Fedora, and I'm definitely thinking of this from that point
of view. The point in 'make headers_install' is to _make_ the kernel
developers care about it, and to help highlight problems as soon as
possible. I don't want problems only showing up after we've shipped the
headers from a certain kernel, and certainly the current situation isn't
encouraging anyone to clean the crap up. Anyone who _has_ attempted to
do so hasn't made any progress in getting it merged back into Linus'
tree.

> Yeah, that made sense. The question is do the public bits get shipped by
> upstream or generated on-the-fly from the mixed ones with a Makefile
> command (once everything is cleaned up and "working")?

I don't mind. If the 'make headers_install' ends up being just 'cp -a'
that's fine by me. If we do end up with a kabi/ directory, that's what
would happen. And then we could perhaps drop the make target altogether,
although we might want to keep 'make headers_check'.

> Thanks for rephrasing -- and yes, our goal does seem more or less the
> same now after sorting the confusion out, which is a good thing. It would
> be nice to have a unified set of headers shipped by upstream which just
> works again -- I think the major hurdle is to get people to start fixing
> the breakages; once most of them are fixed then ensuring new problems
> don't crop up isn't so bad.

OK, thanks.

> I assume right now you're just gathering comments -- how do you plan on
> proceeding with this afterwards?

I've got headers_install and a simple headers_check implemented in the
git tree I referred to. I've got a few minor fixes in there too, so that
'make headers_check' passes for ARCH=powerpc. Once other architectures
work too, I'll drop them into Fedora and see who screams.

Ideally, I'd like to proceed by gathering small incremental header fixes
in the -mm tree and then in Linus' kernel. I'm happy enough to track
those in a git tree of their own, since they don't really depend on the
tree with the new make targets. I can give accounts to any janitor-types
who are going to make a habit of submitting such patches, and we can
have a shared tree on git.infradead.org.

Alternative suggestions are welcome -- maybe Andrew would like
individual header cleanup patches rather than a git tree? But I doubt
that.

My current tree is at git://git.infradead.org/~dwmw2/headers-2.6.git and
is browsable at http://git.infradead.org/?p=users/dwmw2/headers-2.6.git

It's not really supposed to be pulled upstream by Linus, although it
would be fine for Andrew to include in the -mm kernels. We should
probably put the header fixes into a separate tree.

-- 
dwmw2