Re: Some suggestions on the improvement of rebootstrap.sh
> 2025年11月26日 16:55，Helmut Grohne <helmut@subdivi.de> 写道：
>
> Hello Miao Wang,
>
> Would you mind conversing in public? Ideally, you'd bounce your mail to
> debian-cross@lists.debian.org such that it can be seen by others. Feel
> free to quote this mail in public.
>
I'm very happy to also post it in public.
> On Wed, Nov 26, 2025 at 12:59:47AM +0800, Miao Wang wrote:
>> I recently working on rebootstrapping a trixie stable release repo on
>> loongarch64, a.k.a loong64. I used your rebootstrap.sh as a base of may work.
>
> Thanks for reporting back.
>
>> During the process of rebootstrapping, I found that the outcomings of
>> rebootstrap.sh are not adequate for setting up a native building environment
>> on the target arch.
>
> That is to be expected sadly. Thanks for working on it and supplying
> your patches.
>
>> I made some changes to rebootstrap.sh, so that the outcoming repo of
>> rebootstrap.sh can be directly used with mmdebstrap to create a native
>> building environment on the target arch and with sbuild to cross build other
>> packages on the building machine.
>
> Generally, that's the direction I want rebootstrap to be.
>
>> I put the changes on [1] . Since my main target was full repo rebootstrapping
>> on a specific arch, though I did my best to make my changes arch-generic, I
>> only tested these changes on rebootstrapping loong64 on arm64. As a result,
>> I did not open a merge request directly.
>
> Providing access to a fork like this is great. I already looked into
> your changes and cherry-picked the first patch with additional changes.
Many thanks for your time to look into my changes.
>
>> The main additions I made is as follows:
>>
>> - Add gcc-defaults:target (in commit 10ab34a5), so that we can have the package
>> gcc for the target arch
>
> Since you based your changes on a relatively old state (which you had to
> for targeting trixie), I independently implemented this via
> https://salsa.debian.org/helmutg/rebootstrap/-/commit/b5b86db45848f081afbd5ad7a445aecb4deefb3c
>
>> - Add perl:target (in commit 10ab34a5), to fulfill the dependency of
>> libdpkg-perl:target, which is needed by dpkg-dev:target and by
>> build-essential:target
>
> I am aware that perl is missing. This is partially intentional. At
> present, it can be cross built for some architectures, because the perl
> maintainers include the results of a native Configure in the source
> package. This is not sustainable in the long run. Hence, I kept it out
> until Configure is improved to require fewer native checks.
>
>> - Add gcc:target (in commit 47622b09, b3be02ee, 2d614087), so that we can have
>> the native compiler in the target arch. This step should be the last step,
>> because before picking up the gcc:target packages, the library packages built
>> by the cross compiler should be removed. After the replacement, the path of
>> c++ headers in libstdc++-14-dev:target will be changed and the cross compiler
>> will not work anymore. Ada and D are also enabled for the cross compiler,
>> since these two languages are needed to build the native gnat and gdc.
>
> I am aware that the target gcc is missing. I kept it out, because cross
> building it outside a rebootstrap environment has several problems
> already. Please refer to
> https://crossqa.debian.net/src/gcc-12
> https://crossqa.debian.net/src/gcc-13
> https://crossqa.debian.net/src/gcc-14
> https://crossqa.debian.net/src/gcc-15
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92336
>
> I lack the capacity to properly support a compiler cross build in
> rebootstrap. Progress on upstream issues is needed to make this
> feasible. Sorry.
>
>> - Temporarily remove the sqv binary on the build system to prevent apt from
>> generating the dependency to sqv (in commit 4607feae 4cfc1ffb).
>
> I fear at some point, we'll rather have to build sqv.
Sigh, that will add a lot to the rebootstrap script.
>
>> - Add gcc-ports-defaults:native and crossbuild-essential:native
>> (in commit 5e802027), so that we can have the package gcc-<target-triple>
>> and crossbuild-essential-<target>, which are required by sbuild. Here, I
>> chose to always use the source package gcc-ports-defaults, no matter whether
>> the target arch is in the ports repo or the main repo, to prevent the
>> generation of gcc:native package.
>
> I do not want these changes. The ports split is not useful for
> rebootstrap. Instead, I adjust gcc-defaults to cover any chosen
> HOST_ARCH. This also is a change included after your fork.
> https://salsa.debian.org/helmutg/rebootstrap/-/commit/de79aae6dea1e0bb10396304cace47c1c3dc3210
> Likewise, crossbuild-essential-* is not something I am going to include.
> The way to go here is getting rid of it in favor of improving
> build-essential itself.
> https://bugs.debian.org/815172
>
>> - Add dwz:target, m4:target, gettext:target, which are needed by debhelper.
>
> I agree that these are needed.
>
>> With above changes, the final outcomes of rebootstrap.sh can satisfy the
>> dependency of apt:target, build-essential:target, and debhelper:target on the
>> target arch and crossbuild-essential-<target>:native on the build arch, so
>> that the outcomes can directly work with mmdebstrap to create a building
>> environment for the target arch can be used to automatically compile the first
>> batch of packages, the number of which is about 2000.
>
> That's pretty impressive.
>
>> During using the generated repo with sbuild on the build machine for cross
>> compiling, additional symbol link has to be created from
>> /usr/include/<target-triple>/c++/<gcc-ver> to
>> /usr/include/c++/<gcc-ver>/<target-triple> to make the cross compiler compiled
>> by rebootstrap.sh work again. With that symbol link created using
>> --starting-build-commands=, the outcomes can directly work with sbuild on the
>> build machine to cross build packages for the target arch.
>
> How this is supposed to work depends on the value of
> ENABLE_MULTIARCH_GCC. If it is enabled (default), then that directory
> should be searched and you likely found a bug. If it is disabled, the
> directory should not be searched and *-$ARCH-cross packages need to be
> installed.
The libstdc++-14-dev:target compiled during the cross compiling of native gcc
(gcc_4 in my script) put headers in /usr/include/<target-triple>/c++/<gcc-ver>
instead of /usr/include/c++/<gcc-ver>/<target-triple>.
/usr/include/<target-triple>/c++/<gcc-ver> is also the path to place headers
in the official native built libstdc++-14-dev, while
/usr/include/c++/<gcc-ver>/<target-triple> is the path in the
libstdc++-14-dev:target compiled during the building of cross gcc (gcc_3).
In my build, ENABLE_MULTIARCH_GCC is enabled (by default). I can see in your
patch_gcc_wdotap(), you deliberately patched the rules2 file and changed
with-gxx-include-dir. As a result, the cross compiler will not search
/usr/include/<target-triple>/c++/<gcc-ver>. The changed code snippet is
surrounded by ifeq ($(DEB_CROSS),yes), which evaluates to true only when
building a cross compiler, while false when cross building a native compiler
or fully native building. I believe the path of the headers in
libstdc++-14-dev:target compiled in gcc_4 is expected and can be correctly
searched by the cross compiled native compiler on the target platform.
The headers in libstdc++-14-dev:target compiled in gcc_3 can also be searched
by the cross compiler. However, the headers in libstdc++-14-dev:target
compiled in gcc_4 cannot be searched by by the cross compiler. That's why
the symbol link is needed to make the cross compiler back to working after
the native compiler is cross built.
>
>> I hope the above changes are useful for you to integrate into the current
>> rebootstrap.
>
> This definitely is useful, but it integrating it will take quite a bit
> more effort.
>
> Your first patch "support for bootstrap other repos" is included
> already. From there, I would like your commit messages to tell more
> about why those changes are necessary.
>
> "apt-get: disable GzipIndexes" is unclear to me. The setting is disabled
> by default. Why does it have to be disabled explicitly?
It's because it is enabled in docker images. As a result, I explicitly
disable it, whether it is enabled or not in the building environment.
>
> "disable libprelude" is a bit short, but one can figure out that audit
> was the only package that used libprelude and no longer does. Leaving it
> commented out does not help. I can refactor it myself.
>
> "remove assert_built gmp for gnutls28" is unclear to me, because
> gnutls28 still uses gmp during build. Admittedly, it is an indirect
> dependency of nettle-dev, but it still is needed and should be asserted.
I think assert_built nettle might be enough, since nettle b-d on gmp.
Only asserting direct dependency should be enough. However, keeping the
assertion also makes sense. It should not a major change.
>
> "adjust ffi" is likewise too short to tell anything, but the changes
> indicate what is happening here and why. I can refactor it myself.
>
> Later, "add gcc" needs to be split into several commits to become
> reviewable. A commit should not be adding more than one package to
> rebootstrap.
Since currently the target gcc will be kept out of rebootstrap, there
is no need to include this commit, since all the added packages in
this commit are solely to meet the dependency of a native gcc build.
>
> Would you happen to have the resources to rebase your branch on a
> current master and also improve the git history?
I'm very sorry to say that I currently have no spare time to test
against the rebased code. That's why I did not directly put up a
merging request.
>
> In any case, your branch provides significant inspiration for changes to
> include even if I am not going to take it as is.
Many thanks again for your time to reviewing my code.
Attached is the download link for my built outcomes, including the
full build log. The starting command is:
docker run --rm -it -v $(pwd)/rebootstrap:/rebootstrap -v $(pwd)/repo13:/repo --tmpfs /tmp:exec debian:trixie bash -c "rm -f /etc/apt/sources.list.d/debian.sources && echo 'deb http://<mirror>/debian trixie main' > /etc/apt/sources.list.d/default.list && echo -e 'Package: *\\nPin: release n=trixie\\nPin-Priority: 1000' > /etc/apt/preferences.d/default.pref && apt-get update && apt-get dist-upgrade --allow-downgrades --no-install-recommends -y && rm -f /etc/apt/preferences.d/default.pref && rm -rf /var/lib/apt/lists/* && /rebootstrap/bootstrap.sh HOST_ARCH=loong64 REPODIR=/repo DIST=trixie GCC_VER=14 DEB_BUILD_OPTIONS=\"\\\"nocheck noddebs\\\"\" MIRROR=http://<mirror>/debian 2>&1 | tee /repo/rebootstrap.log; exec bash"
Cheers,
Miao Wang
-----BEGIN PGP MESSAGE-----
Comment: GPGTools - https://gpgtools.org
hQIMA5YyR5men3fGARAAixQ5A1q2ig4hjL98w3g/NysZpMKQx7s7QmMlXd/NetCU
AGcpw65PlxnhAdQDJRwfQXK+noJIHS/k3NplzL7F0K01Kbl7rihSm6fDJ4uAO01m
18J6OUfH07DXuiijVr/eQovbDuLBJh2Vc5AiCLgGhXsoLXG1wjTtSqUfNiKazJqD
SiAmdg+pUKC3G+lZ4zpxLMkIXyjezT+GUJ6tMbKUvZRBdl9y1v/dfXJJJkbqCeht
RvMpEkrrJUnjYWKw1SvWoq5/mQyREjwibKQ8A5uq4XH2Vr85W7syf9q53yngK6Vo
6wh5Jfdn3zlbVZrtpDHaV6FUlgHojDgIfaXqTeFsrQziPJC81nuWJMfDfvqWwQW7
CbUNXV2DeMNv1v5j2gYG8H/iSxSYM7WcrW9m3l4ZDgqj40RCloVhfb+WDopRIBPG
v+jFE2EXif1RywCM55tLmagpUdGvZglvi+vJmDme3EKWUcJZK2j9oHHbG3Aaxmg5
23Xld/lqthwAyydlJn2hAxVJeSV3kBciLcUf4mxB9RkqpmPWs03It45pugnCnZz4
aZvE+iy30joKsKl3wf+jaBgSSA2MX9cxQSc4TD616K3VLwrbrnXLf8dG4/HIHjLx
UVBUBYfuEnD66LKYS03efPSkHthIOHQdhQQNA4l2yyABO1pGTHeupt6MC9yQOBbS
wJQBMPXUjxIjCDu/KOaIqXZUuAPw0+6OPafe/t/ZN+SuLKpviVQYBMKkjU9vEjPN
7BUK8uGxdaI2cgmISJbv6UUwHIjcgZNziE0SNKeMiMUBhQFym8NBqwLFVbDtsBD5
BqHLZamuGaWzafIYOyOwrRiDxvYIcc1T7aFZ7Kt65P6UOsDSmQoRZS1oGxCCG5oa
aIc7X9/NmIiByIJAh0IMRkYXo5FRX+c2ZhLuDr8jZ1qpQV4So+tQT26kS+ZBPPcl
rs7E/nOH8E6GIFgiLmBl82JSGQOrcO1JCA4Yl7AcydPDedfOXBZIOxPYJ3r+Vxg4
3MbCfRzwaghRbkDQoF5KljoGdlpwlXXwjhKEZ1GBXR75WNXjmrt/SPBc9uQStd4H
Gek8w/6T7rYYPQj6dwlRUk7/o/BrRusiy3HsVzbb8e41R/wy7+Dni6ljko5ifzed
n99HH7Ku
=f0pj
-----END PGP MESSAGE-----
Reply to: