Re: Speedup when using qemu to cross-build packages.
Am 07.09.22 um 22:41 schrieb Johannes Schauer Marin Rodrigues:
If half the time people spend on hacks would be spent on a proper solution, so
much more could've been done already... I know, everybody can spend their free
time on the stuff they find fun but sometimes it's a bit frustrating...
Yes, fully agreed. But the same applies for so many things: If people
(and companies)
would only be less selfish, everyone would benefit. The only way to fix
that would
probably be to create some incentive to do it right (for everyone). How
this could be
achieved for Debian should imho be a widely discussed topic within the
community.
Cross-building is often the same. Every new architecture that comes along is
bootstrapped in a rush and everybody rewrites the same hacks over and over and
over again.
To me, the major problem is that cross-building seems to be not even a
second-class
citizen to package maintainers: When cross-building only the most basic
packages, i.e.
those being reported by "debootstrap --print-debs", less than a third
build sucessfully,
the rest fail for various reasons, many of them (e.g. bash, glibc,
apparmor, ...) even fail
already to install their required build dependencies!
Thus, if even most of the maintainers don't care, it very much looks to
me like
almost everyone already gave up on this. And considering that Debian has
some
20.000 source packages, it certainly would be almost impossible for that
to be fixed by
a single individual.
Considering all this, qemu-user really seems to be a rather attractive
solution: It's
slow, but it works! And if it could significantly be sped up, maybe even
close to a
native build, we may seriously ask whether cross-compiling is still
needed at all!
Kudos here to Helmut's rebootstrap which is absolutely the right way forward
even if the world thinks otherwise and rather repeats last year's dirty hacks
instead.
In an attempt to be at least a little bit constructive with this email: Your
best solution is probably to buy two new computers. You probably already have
an amd64 machine, so that problem is solved. A computer that can run arm64 can
also run armhf so you only need one machine to run both architectures natively.
The second new machine is one that can run mipsel. This will cost a bit of
money but if you buy the stuff second-hand it will not cost much and if you
really need to compile several hundred packages, the money might be worth the
time you will be saving because qemu indeed is very slow.
If instead, you want to continue with the hack you started, then it sounds that
what you want is the equivalent of what box64 does for amd64:
https://github.com/ptitSeb/box64 Essentially, this uses a mechanisms called
dynamic recompilation where a foreign architecture binary will be able to use
native architecture libraries:
https://box86.org/2021/07/inner-workings-a-high%e2%80%91level-view-of-box86-and-a-low%e2%80%91level-view-of-the-dynarec/
This is still hacky and it will still fail in many situations but there you go.
:)
Many thanks for the pointers, I'm certainly going to have a close look :-).
And if there would ever be a collective effort to get crossbuilding going in
general, I'd be happy to contribute wherever I can.
Thanks!
cheers, josch
Reply to: