Re: RFC: Patches for supporting cross-building
Hello Helmut,
Thank you so much for the detailed information!
On 2018/06/16 4:56, Helmut Grohne wrote:
Hi Kazuhiro,
On Fri, Jun 15, 2018 at 11:09:58PM +0900, Kazuhiro Hayashi wrote:
I'd like to share patches for 70 Debian packages with this ML.
https://github.com/meta-debian/debian-cross-patches
Thanks for your work!
The patches are intended to make the packages cross-buildable.
According to the previous summary [1], these packages are still failed
for cross-building.
Oh, a lot of time has passed since that summary and a lot has changed.
Present numbers are more like: 18% of Debian is cross buildable. That's
more than a fourfold increase.
OK, we will check the latest package state again and update the patches soon.
Here are several questions about the patches and future contributing
to Debian community:
* The target packages of above patches are selected based on
the information in 2016 [2].
Can I find any other latest cross-building results?
We're seeking a hardware sponsor for running a cross buildd. That's why
we don't have any public CI at the moment. I occasionally run cross
builds for myself now and have a pile of 3400 vaguely current build logs
here, but they are not published anywhere. I've just uploaded a one-shot
summary of these logs at
http://subdivi.de/~helmut/cross_report_20180614.html.
Thanks, helpful report for us to know what's going on.
Regarding the build environment, as the first step,
I guess it's better to have the similar build environment as yours locally,
properly recognize the issues you are facing in your reply to Baurzhan,
then consider the next action.
Currently, we are simply try package building with dpkg-buildpackage
enabling the target foreign architecture (armhf) in host environment.
At least, more formal build process is required to test cross-building packages.
e.g. Using sbuild/buildd, testing with multiple target architectures, etc.
Also, is there a sharable place for debian-cross developers
to update such data together? At least, the bug report number
should be shared here to avoid duplicated efforts.
I have tagged[1] all of my cross build bugs with user helmutg@debian.org
and usertag rebootstrap. More than 500 open bugs affect cross building
of more than 700 source packages. I'm ok with other people using that
usertag for cross build and cross bootstrap issues. I'm also ok with
moving to a different usertag, e.g. user debian-cross@lists.debian.org
usertag ftcbfs would make a lot of sense for build failures. Also
https://bootstrap.debian.net/cross_all.html uses user
debian-cross@lists.debian.org usertag cross-satisfiability to display
dependency issues.
I got it.
At this moment, it's easy for us to find the work-in-progress packages
by "User=helmutg@debian.org" and "Usertags=rebootstrap".
Is there any preference from you or any other contributor here?
Actually, other several developers are mainly created the patches.
When they post the patches to debian bug report,
"Usertags=rebootstrap" is still enough to easily share our efforts.
Also, I think "User=debian-cross@lists.debian.org" would make sense
because it makes easier for other developer who are interested in cross-building
to find the activities in this mailing list.
X-Debbugs-Ccing such patches to this list could make sense, but it would
vastly increase the mail volume. How do other participants feel about
doing so? Expect around 100 mails/month.
In my opinion, Ccing should be avoided because it might become harder
for some newcomers like me to find key discussions about cross-building
which are not for specific packages.
Instead, it would be great if there is a small summary somewhere
about such preferred labels (like User and Usertags) in patch submissions.
* I'm planning to send the patches to each package maintainer
through Debian bug report.
Let me briefly skim through semi-random entries of your patches
repository:
* analog: Breaks building with clang. Better patch filed as #812231.
These days we also have buildtools.mk, which does the right thing.
* blktrace: Already filed as #894836 and fixed in unstable.
* bmon: The approach is wrong. A proper fix can be found in #870864. By
using my approach, we can help share it with other efforts such as
yocto, ptxdist, etc.
* coreutils: Nice patch, because it builds from source. Please send it
to #721358 immediately and Cc me or this list.
* libxt: This one needs discussion. It is about detecting the behaviour
of malloc, which is impossible during cross building. Moving that
check into the packaging is a bad idea, because it means modifying
hundreds of packages. We already have a standard way for autoconf and
that is "cache variables". These can either be set as environment
variables or via a config.site file. For libxt it happens to be
enough to set xorg_cv_malloc0_returns_null to yes (for glibc, this is
wrong for musl). The cross-config package has a number of such
assignments but it is very out of date and even has some wrong ones.
My own cross building efforts remove the broken cross-config from the
equation and I use environment variables. Getting a central piece to
maintain such variables is certainly a good thing. Having individual
packages be that place is going to be a maintenance hell though.
Thus I NACK this patch.
* netkit-telnet: Your patch breaks the test. That snippet will always
compile, but the question is whether it runs. It would break native
builds.
* openhpi: The approach is wrong. A proper fix can be found in #899198
and the patch is applied in unstable already.
* pixz: Already filed as #883888 with a very similar patch.
Given this sample I suggest that you:
* Check which of your patches are still relevant to unstable. Given the
current speed, I don't think it makes sense to target older releases.
* Check which of your patches already have bug reports.
* Defer all cache variable patches (e.g. malloc/realloc non-NULL)
until there is consensus.
* Then repost your patch repository on this list.
Thank you for your detailed suggestion,
we are going to send the updated patches again.
Are there any standard ways for posting patches to support cross-building?
Ex: Common keywords in subjects, information which should be shown, etc.
With my filing of patches (and I've filed more than 1000 now), I've used
the term FTCBFS (fails to cross build from source, which mimics the
common FTBFS). So the subject tends to be "$sourcepackage FTCBFS:
$summary". Sometimes a failure is caused by another package. Then I've
tried to describe the problem (e.g. "mark $binarypackage Multi-Arch:
foreign" or "move $pkgconfigfile.pc to a multiarch location") and add
"Control: affects -1 + src:$affectedsourcepackage".Great helps for unified patch posting, we will follow the formatting.
Agreeing on a usertag (see above) would help here.
* My current focus is to create patches for support cross-building,
but it might be better to concentrate on other packages or topics
which are more important for debian-cross community.
Indeed, I think much of the low hanging fruit in plain FTCBFS is gone
now. There are a number of harder issues that nobody has tried to solve
yet. Let me try to give some examples that I've poked at, but not
succeeded yet:
* cross building perl. src:perl is now built from source in Debian, so
patching the upstream configure generator to not need try_run checks
for e.g. sizeof is possible.
* gcc-for-host. WIP at #666743, but I'm stuck.
* Reducing the toolchain bootstrap from 3 gcc stages to two. The glibc
developers indicated that this is no longer necessary and
build_many_glibcs.py seems to work with a two-stage bootstrap.
* cross building perl extensions.
* gobject-introspections (yocto and ptxdist gave up and use qemu).
* foreign package installation. I've brought DPKG_ROOT and
--force-script-chrootless into dpkg, but we need update-alternatives
and dpkg-divert to support it.
* non-glibc ports (e.g. musl-linux-any)
* fixing failures from https://jenkins.debian.org/view/rebootstrap/
(is arm64 important to anyone?)
* ...
All of these are hard problems that one isn't going to solve in a week.
Just making random packages cross buildable is substantially easier.
Some topics seem to be challenging but also important for my cases.
Many packages in http://subdivi.de/~helmut/cross_report_20180614.html
are unsatisfiable because of perl-base, is it related to the first topic?
BTW, https://jenkins.debian.org/view/rebootstrap/ returns 404 to me,
do I need to setup something to see it, or was the page already moved to another place?
For example, some kinds of working were already shown by Helmut [3].
Hah. Most of that is still current except for the numbers. Though we
made qt "just work" now and having buildtools.mk makes writing patches
easier.
It's important for me to cross build a part (not all) of Debian packages
which are often used in embedded systems, and also interested in
the difficulty of bootstrapping Debian. I'd like to know if
there are big issues to be solved related to my interests.
If the list above is insufficient, I can try digging up more. ;)
Now, we acquired enough to look at existing issues and go to next, thanks much.
We would require more detailed discussions later in other threads
which focus on specific issues.
Here is a project to achieve such goals by cross-building Debian sources
with Yocto Project build system [4]. It can cross-build everything
and provide high customizability, but metadata of packages needs to be
implemented from scratch, which takes high development & maintenance costs.
I fully agree that duplicating the metadata is a bad idea from a
maintencance pov. That's why I avoided that from the start. You actually
loose the whole qa (e.g. reproducible builds). Trying to compete with
Debian on high quality package maintenance isn't going to end well.
Another approach is to make Debian packages cross-buildable with existing
package metadata (i.e. debian/*), which is just now progressing
in this great community. The first things I'd like to do is to join
debian-cross then keep better contributions.
It's been progressing a lot for a long while now. You correctly observed
that the approach has downsides as well:
* Customizing packages is not that easy, but with build profiles now
well supported and the extension namespace (pkg.$source.$anything),> customizing packages with build profiles is a reasonable and
maintainable thing now. What kind of customizations do you need?
For example,
1. Applying small patches such as bug fixes, security fixes
which are not applied to upstream Debian
(We always should post such fixes to Debian or upstream,
but sometime we need to apply them locally until
the fixes are officially merged)
2. Changing configure/build options to drop dependencies that
bring a lot which are not essential functions for the specific targets
(For robustness, simplifying packages list, reducing footprint, etc.)
Some of such customizations, of course, require dependency changing.
3. Changing the base system; e.g. init
busybox is still usable for many targets which provide
basic hardware controls and quite simple application only.
e.g. meta-debian ([4] in my last post) provides busybox based tiny RFS as well as systemd.
I know that such environment is no longer Debian, but
it still keep a lot of benefits from Debian 'source code' (e.g. security fixes).
My another interest related to this topic is that Debian installer (udeb) is also
using busybox based system and still keep maintained as 'Debian'.
I think there still be several possibilities to make use of such
existing efforts in embedded world, but I don't have concrete idea
to make good relation between the two different worlds.
4. Compiling packages with CPU specific optimizations to improve performance
I guess https://wiki.debian.org/BuildProfileSpec could cover 2 in above at least.
* Small footprint is another difficult thing. The essential set is
slowly growing all the time and making it smaller is non-trivial. I
learned that the hard way when I made e2fsprogs non-essential and it
still is required. Essential is roughly 110MB. If you nuke
/usr/share/locale and /usr/share/doc and a few more, you can get it
below 70MB.
Yeah, minimizing the essential set is the most important and difficult issue.
Also, other non essential packages (service daemon, utils, etc.) are usually appended
into RFS then their big dependencies sometimes affect to the final footprint.
Recently, reducing footprint may become less important comparing
when H/W resources were poor, but it's still required in some limited cases.
(e.g. Installing into 64bit FlashROM)
Still I think that we have reached the turning point for all but the
smallest of embedded systems. Integrating with Debian means that you
pick from a large set of packages with immense QA and long support
windows. If you repackage the Debian's source code, you don't get much
of that beyond license documentation.
Hope this helps
Helmut
[1] https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=rebootstrap;users=helmutg@debian.org
Reply to: