[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#766708: Processed: Re: Bug#766708: breaks multiarch cross building

+++ Dimitri John Ledkov [2014-11-23 12:27 +0000]:
On 23 November 2014 at 11:23, Ron <ron@debian.org> wrote:
> On Sat, Nov 22, 2014 at 08:51:41PM +0000, Dimitri John Ledkov wrote:
>> On 22 November 2014 at 16:21, Ron <ron@debian.org> wrote:
>> > Dimitri wrote:

> """"
> The newly introdued mualtiarch cross building in jessie to a few packages:
> * cannot be build on standard debian buildds

Not yet, but all the code to do this exists. The necessary sbuild is
in Jessie. The necessary wanna-build patches are here:
(awaiting review and the stable release before potentially problematic
wanna-build changes are actually applied)

> * cannot build multilib toolchains

Correct (although this could possibly be fixed).

> * and thus resulting toolchains cannot rebuild non-trivial & core
> debian packages

There are _lots_ of debian packages that currently don't cross-build,
for lots of reasons. A tiny number of packages _need_ multilib to
build (SFAIK). So whilst this is an issue to consider, it is a fairly
minor one on the scale of things. The cross-toolchains remain a) the
only ones available in the archive and b) useful for building _most_
debian packages (and other stuff).

> These reasons have been pointed out to the people raising this bug
> report before. If anyone missed that for any reason, it is pointed out
> now.
> """"

As as has been said numerous times in this thread, no-one is
suggesting that the current cross-toolchains are immutable and can't
be improved/replaced, but until we can actually build the alternatives
(Have you fixed cross-toolchain-base for debian yet?) there is no good
reason to break what is currently working (and in fact having that
working _still_ isn't a good reason for breaking the
'mulitarch-multiarch' builds).

> I'm not sure if you are deliberately missing portions I've emails that I sent.
> > The people who have actually been working on this _in Debian_ are
> > well aware of what is not perfect about it and what extra work
> > remains for post-Jessie to make it more so, and they are actually
> > working on those things.
> >
> > They even have packages based on this uploaded to sid, so that the
> > other work on fixing those things can continue, e.g.:
> >
> > https://packages.debian.org/sid/gcc-4.9-arm-linux-gnueabi
> >
> This package is not build on Debian

Yes it is

> and cannot be ever rebuild on Debian.

Nonsense. Of course it can.

> And RC bug reports are filed.

And they will cease to be RC as soon as Jessie is released and
wanna-build updated. The work has been done.

> This concrete example is very
> good way to show that its upload is pre-mature. The reason is quite
> simple, that we do not have foreign architectures enabled on the
> builders, nor any easy way to enable them (being or has been fixed in
> sbuild),

Yes we do. Building these packages in the Sbuild in Jessie 'just
works' already. Try it:
sbuild -A -s -d unstable -c sid-amd64-sbuild cross-gcc-4.9-armhf

(this will fail immediately after a new gcc upload until the
cross-build-deps are built, because its build-deps are not available,
just as any other package would))

> however on-demand enabling foreign architectures will not be
> in place until only after one stable release where it is possible to
> do so and buildds getting upgraded to such release.

It will be (is!) in place in Jessie. It's had limited testing so there
may prove to be problems in practice, but our testing so far indicates
that it works fine. All the packages uploaded to unstable (and for
Jessie in my p.d.o repo) are built in standard jessie/unstable sbuild.

> > What nobody has explained to us is what problem is *fixed* by
> > gratuitously breaking this for existing users of Jessie.
> >
> The problem is introducing incomplete & broken things into archive.

'incomplete & broken' is not fair. They are quite new and we are
awaiting infrastructure changes to be applied by the buildd
maintainers, but that's not going to happen until after the stable
release now. But the packages themselves are now in quite good shape.

Unstable now contains cross-compilers for all the arches in Jessie
(for amd64), built using standard sbuild. Including cross-gcc-defaults
to add the wanted symlinks for all arches except mips (because mips
was lagging badly at the time of the original upload so I missed that
one out - this has just been corrected in cross-gcc-defaults 0.4,
currently in NEW).

They work to build all (most?) of the packages in Debian which are
cross-buildable at all and whose cross-build-deps are installable:
(e.g. test on 100 packages here:

Yes there is plenty of stuff that doesn't cross-build but that's not
because these toolchains are particularly 'incomplete', or
'broken'. You'd get the exact same failure set with a standalone
cross-toochain too, SFAIK.

Convenience 'crossbuild-essential' packages could be in unstable too, but the
maintainer (Doko) has only uploaded them to experimental so far.

> E.g. the above uploaded package into sid FTBFS on any release: sid,
> testing or stable.

Not for me on testing or unstable. How did you test?

> This is obsurd to call it as "actually working",
> and no changes to gcc-4.X packages will make cross-gcc-4.9-armel
> finally build from source on debian infrastructure.
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766625

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=766626 (armhf) is more
informative (I didn't copy my response to 6 identical bugs)

In summary, as explained above, applying the wanna-build patches will
fix that bug, and allow us to maintain more than one build arch.

Britney patches will also be needed to allow migration.

> >> >> Can we all settle and move these developments to experimental
> >> >> targeting for stretch, instead?

I don't see why we shoudn't have useful-if-imperfect cross toolchains
in unstable until there is something better available.

> > Yes, it's still a bit more awkward to build a toolchain than we all
> > would like, but this is still infinitely easier and cleaner than
> > anything we've ever had before.  What do we gain by denying people
> > the ability to easily use and experiment with that, in real life
> > use cases, while we work on improving the other things too?
> >
> multiarch-multiarch is infinitely easier but for a very narrow use
> case & very narrow timing.
> Not useful for re-bootstrap - as pre-existing bootstrap binaries are
> required (foreign multiarch debs).

Except that rebootstrap uses it for bootstrapping, so (surprisingly)
it is relevant there too. And it was breaking that which prompted this
bug, in fact.

> Not useful for new arch bootstrap - as pre-existing bootstrap binaries
> are required (foreign multiarch debs).

Granted. And I agree we should attend to that case (I will do so
myself in due course if no-one beats me to it). But 90% of
cross-toolchains users are not doing this, so the cross-toolchains are
useful to most people.

> In-archive toolchain maintenance is also akward - as all
> build-dependencies must be install-able and of all the same version
> across build & host architecture. This is often affected by
> architecture skew (and/or builder's speed - # of builders), but even
> worse release team often binNMUs things with a skew - that is binNMU
> select architectures instead of all of them. Then things are
> essentially permanently multiarch non-co-installable. Until a said
> package re-uploaded, or requested to binNMU to even things out.

multiarch binNMU skew is a problem that we should just fix in dpkg -
it affects lots more than just this.

yes, the multiarch build-skew delay is a problem in unstable. No-one
denies that. Build-automation can minimise it but not make it go away
entirely. The 'right' way to deal with it might be to teach
wanna-build about 'target-arch' builds and automatically do a set of
those on a new gcc source upload. This potentially allows both native
and cross toolchains builds to be generated from one upload (which is
what we really want) but without making cross-failures cause the
native build to be help up. We plan to investigate this possibility
more thoroughly - it may be completely crazy.

> "infinitely easier and cleaner" and a whole lot more fragile.

It's only 'fragile' insofar as gcc source changes break the
cross-toolchain builds (thus breaking automated rebuilds), such as the
change that prompted this bug. If you mean 'uninstallable for a time
in unstable after new gcc uploads' then yes, everyone agrees that's an
issue, but I don't see why it makes for a 'very narrow use case'. It
shouldn't be a problem in testing as everything should be built in
time and migrate together (modulo problems caused by new uploads
breaking cross-builds, e.g by having to carry a growing patchset over
the gcc-source due to uncooperative maintainer, which fails to apply).

As has been repeatedly stated, when the alternative (standalone) build
is actually working, then this argument about which build method is
better becomes a lot less sterile. Until then, you are just arguing to
block/make-life-difficult-for-maintainers-of what we do have.

Principal hats:  Linaro, Debian, Wookware, ARM

Reply to: