Re: Bootstrappable Debian - proposal of needed changes
On Tue, Jan 15, 2013 at 07:18:40PM +0100, Johannes Schauer wrote:
> Besides bootstrapping, these build profiles could also be used for
> embedded builds, and to allow for changed buil-deps when cross-building.
On a related point, see:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=695287
I've been thinking, and talking to Wookey, about how to do this sanely.
The best idea I have is for those packages for which we have -triplet
versions (gcc-x.y et al, pkg-config, and so on) to have an extra control
field called something like "Cross-Tool-Available: yes". sbuild and
dpkg-checkbuilddeps could then use the presence of this field to know
that they need to translate "Build-Depends: gcc-4.6" into
"Build-Depends: gcc-4.6:native, gcc-4.6-arm-linux-gnueabihf" when
cross-compiling to armhf, say. Given my recent sbuild patches, it's
very easy for sbuild to do this once we have agreement on how it should
detect when to do so.
This is cleaner than any of the other options I've come up with: it
doesn't require hardcoding a list of "toolchain packages" that have
special cross versions; it would allow us to stop having to shove
pkg-config-HOST into cross-build chroots; and it wouldn't require
dpkg-checkbuilddeps to violate layers by looking in the apt cache, at
least as long as the available file is up to date. Does it seem sane to
people?
> While this field is meant to make sure not to allow any profile built
> binary package to be uploaded to the archive, it can also be abused to
> only allow "some" build profiles to be uploaded. For example ubuntu
> might generally forbid profile built binary packages to be uploaded
> except for packages built with the "ubuntu" profile only.
I'm not sure we (Ubuntu) would actually want to do this, FWIW. It
wouldn't buy us much since all our binaries are required to be built on
our own buildds anyway. That said, the ability to have conditional
build-dependencies for Ubuntu and its descendant dpkg-vendors would be
valuable; but I'm wary of feature creep.
> 5. Cross-Builds field
> =====================
>
> For even further automation and also for quality assurance, we propose
> another new field for source packages which indicates whether or not
> this source package is supposed to be cross compilable.
I don't think we should do this. Instead, we should have a cross-buildd
that regularly tests whether packages are cross-buildable, and over time
we should expect packages considerably beyond just the base system to be
cross-buildable. I want to be able to take any package that I might
find on (for the sake of argument ...) an Ubuntu phone and expect that I
can cross-build it cleanly; I don't want cross-building to be this
strange thing that only works for a few exceptional packages, and I
think that having this field sets that expectation. The more packages
that just work, the easier it will be to do anything related to
cross-building.
This shouldn't be unrealistic, although it's certainly ambitious.
http://people.canonical.com/~cjwatson/cross/armhf/raring/ shows about
35% of Ubuntu main cross-building cleanly right now, and there are lots
of cross-build and multiarch patches sitting in the Debian BTS (I've
forwarded and/or uploaded everything I've done) which would improve the
state of a similar core-ish set of Debian source packages to something
similar. There are really only a handful of underlying causes
representing many of the remaining failures - and yes, I know these are
fairly hard (perl, python, gobject-introspection, ...) but they aren't
intractable.
> If Debian wants to incorporate the ability to being bootstrappable in its
> policy, then there *must* be some packages which are cross compiled for
> a minimal build system. Adding this header to those source packages
> would make it a bug if they do not actually cross compile. Without this
> header, cross compilation would be wishlist as usual.
This would be better done by way of an external list of the packages
that need to cross-build cleanly.
> Furthermore, cross compilation is one of the methods a porter can use to
> break build dependency cycles. If some packages carry this new field
> then not only could a porter decide quicker whether or not a source
> package is cross compilable, an algorithm could also incorporate this
> information to automatically break build dependency cycles for cross
> compilation.
While this is true, having stared at similar things myself in the past,
I've come to believe that it's a better use of time to just make
everything in sight cross-buildable than to spend lots of effort trying
to algorithmically decide what might work or not.
> If more automated bootstrapping of Debian is desired, then at least build
> profiles (1.) should be introduced.
I am strongly in favour of something occupying the general position of
build profiles; there are several otherwise-intractable cross-building
problems that will require something like them. I don't have much
interest in the specifics so will refrain from suggesting another colour
for the bikeshed. :-)
> The question remains of how many source packages would have to have the
> proposed new fields added to them to make a full bootstrap from zero
> possible. If the Gentoo USE flags were not too far off and assuming or
> tools do the correct thing so far, then:
>
> - the number of source packages that has to be modified with the new
> fields is at maximum 83 (there might be a smaller set).
https://wiki.ubuntu.com/CrossBuilding/BuilddChroot (thanks, Matthias)
suggests that we are not that far from being able to at least get to a
sensible minimal chroot with a modicum of hacking.
--
Colin Watson [cjwatson@debian.org]
Reply to: