Pure-mulitarch Cross Toolchains
+++ Matthias Klose [2013-01-16 15:37 +0100]:
> Am 15.01.2013 19:18, schrieb Johannes Schauer:
> > This mechanism also covers cross-compiler bootstraping. The eglibc, gcc,
> > and kernel packages already have the neceassary staged-build info, but
> > the build profiles (1.) part is also needed to specifiy the reduced
> > build-deps. The cross-toolchain bootstrap ceases to be a special case if
> > treated this way and just becomes packages to be built in stages using
> > the profiles mechansim, like many others in the base system (but for
> > build arch taregtting host, arch, rather than built for host-arch). See
> > the wiki article at [11].
> >
> > [11] http://wiki.debian.org/MultiarchCrossToolchains
>
> Please stop calling this "MultiarchCrossToolchains". It was already wrong when
> used in the Emdebian context, and on some GSOC pages. What you are apparently
> aiming for is a cross compiler re-using the target libraries built (natively).
The target libraries can be built natively or cross-built. Clearly
cross-building is necessary when the target architecture does not yet
exist (e.g. arm64). The point is that the library dependencies use the
multiarch file locations and mechanisms, rather than extra copies in
different locations.
I agree this is separate from multiarch path support in the
cross-toolchain for finding libraries and headers for non-toolchain
libs. What term would you like me to use for this cross-toolchain
arangement? multiarch-libraryCrossToolchains?
PureMultiarchCrossToolchains? FullyMultiarchedCrossToolchains?
Yes it's a bit confusing, because multiarch affects various aspects
of the toolchain. I'm happy to use a different name to avoid
confusion, this just seems to me to be the logical conclusion of
'mulitarching the cross toolchains' (see discussion below), so it
seemed a reasonable name.
> However this is not yet done,
It was done in Thibg's GSOC project for 4.7.2-4. Yes it needs more
work to properly integrate, test, and make bootstrappable, but we have a
working implementation (modulo keeping up with your amazing rate of
new uploads :-). (For which kudos, even if it does make it hard to
keep up!)
> requires dependencies on "foreign" architectures
It does.
> and only is a very special case,
This is where we seem to disagree. I think it should be the standard
case (for binary distro cross-toolchains).
> not helping for targets where these are not yet
> available (like for the arm64 bootstrap). Having a cross-toolchain just built on
> the host architecture (targeting the target architecture), should still be an
> option.
This is still an option - you just cross-build the target libraries as
part of the bootstrap. The wiki page gives the runes.
I am not convinced of the benefit in also maintaining toolchains with
these libraries installed twice in different locations, now that we
have multiarch to make this unnecessary. But I would
like to see much wider discussion of this issue so we can reach
conclusions as a project on what the default cross-toochain setup
should be, and what non-default methods are also worth maintaining in the
packaging.
The main disadvantage of making use of the multiarch libraries is the
need to keep libgcc, libstdc++ etc in version lockstep across
architectures. But this applies to multiarch in general for all
libraries, and is something we manage in Debian as part of being a
distro. It is something to carefully consider. Is the reduced
independence of the cross-toolchain library versions a fatal flaw?
The main advantage is that the code you build against is actually the
code the toolchain was built against, not some other code that happens
to be installed in the expected location. Also build-dependencies work
automatically without having to have 'equivs' packages for libgcc1,
libstc++6-dev etc until the real native ones are built. And having one
copy of libgcc1:<targetarch>, libstdc++:<targetarch> on the system
instead of two seems like good practice to me.
Another advantage is that for normal cross-toolchain builds you don't
need to do a 3-stage bootstrap build, because you already have the
target-arch libc, libgcc1, libstc++ etc available so you just need to
rebuild gcc without staging. Much quicker and simpler. This is
(fundamentally) how the emdebian toolchain builds worked for many
years.
Using multiarch libraries also greatly simplifies the gcc packaging
(if we are able to drop multilib equivalents as a result). Again we
disagree on this point.
I think we should be encouaging people to install arm-linux-gnueabi
and arm-linux-gnueabihf cross-compilers if they want to target both
those arches, not install one or the other, each of which is
multilibbed to output code for both, with a corresponding collection
of libgcc's. I realise that upstream is not yet convinced of the
benefits of this view of the world, but I think we should be using it
and demonstrating its effectiveness (assuming we believe that
multiarch is actually the right way to be dealing with multiple ABIs)
I am happy to be persuaded that this is madness with real examples of
why we can't/shouldn't do things this way (and I can see that
deprecating multilib support where it is well-established and code is
co-runnable (x86_64 and i386) is not going to happen), but I've been
doing it for years for arm and it seems sensible to me.
> Binutils, GCC, clang, and probably more packages in Debian are currently aware
> about the multiarch locations, including native and cross compilers. So please
> don't hijack this term for your very specific view on a cross compiler.
That is using multiarch, as opposed to being multiarched themselves. I
agree we should have terms to distinguish between these concepts to
avoid confusion. How about 'Multiarch-compatible' and 'Pure-multiarch'?
> The so called "cross-toolchain bootstrap" is overly complex and probably should
> not be handled by staged builds, but by building from a combined source tree.
It's a rare case where more than one stage is needed (both libc and
gcc have stage1 and stage2 before the final full build), but there is
no way to avoid this process for a new architecture. The question is
simply how it is implemented.
It can be done just the same as other packages using profile builds,
or it can be special-cased as is currently done for the ubuntu builds.
That works by each of binutils, linux, eglibc and gcc binary builds
producing a -source package, and having a special package
(<targetarch>-cross-toolchain-base) which uses
all of those sources to step through the binutils, linux stage1, gcc
stage1, eglibc stage1, gcc stage2, eglibc stage2, builds, using
dpkg-cross to munge library locations.
This latter works but I disagree with the assertion that it is simpler
than making these packages the same as others that have profile builds:
The normal source packages are used (instead of binary -source
packages), and the build-deps encode the profile/stage builds needed,
and their ordering. Then any tool that can order builds for
bootstrapping can now also do toolchain bootstrapping. Why is that not
a desirable goal?
As you can see if you've made it to the bottom of this long mail,
there are some quite involved questions about how we should be
implementing cross-toolchains in debian. I would love to have wider
input from the project on these questions, and am not sure what the
best forum for that is.
I guess 'this mailing list' for now. With final decision at debconf
once people have got their use-cases and arguments aired?
I have a vision of a 'fully multiarched' cross-toolchain, but the
toolchain maintainer doesn't really share it. Ultimately it's his job not
mine, and perhaps that is the clinching argument, but it would be good
to hear from some other people too.
Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/
Reply to: