Stretch bootstrap for sparc64 and sparc, optimized for ultrasparc3
Hi,
On Thu, Sep 7, 2017, at 11:19 AM, John Paul Adrian Glaubitz wrote:
> On 09/07/2017 10:30 AM, Tom Turelinckx wrote:
>> Not all of those may be necessary anymore, but I've been doing it like
>> that since squeeze and up to the current sid on dozens of machines, and
>> it works reliably: when the first disk fails, I am able to boot from the
>> second disk.
>
> On a sidenote: Would you mind enabling popcon for your sparc64
> installations,
> so we get more counts of people running Debian on sparc64 hardware?
Enabling it for my sparc64 installations wouldn't be very useful, as those are just two or three machines (V210, V240, T5140) with one or two LXC containers each, and they're only used for experiments, so they're not representative.
Enabling it for my sparc installations would cause quite a spike in the Wheezy deployments, as those are 15-20 machines (V120, V210, V240, V440) with ~4 LXC containers each. Those currently can't be upgraded because Sid is too volatile.
Because I want to move forward with upgrading some of those machines to a newer release, and replacing some of the Sun Fire-series hardware with SPARC Enterprise-series hardware, I'm working on bootstrapping Stretch for both sparc64 and sparc, preferably --with-cpu(-32/-64)=ultrasparc3 instead of ultrasparc.
Just bootstrapping Stretch 9.0 for sparc64 is relatively straightforward thanks to excellent work by Adrian Glaubitz and others: for most of the binary packages the correct version is readily available from snapshot.debian.org. I've built a repository of binary packages to create Stretch 9.0 for sparc64 that has either the correct version of each package or slightly older, up to build-essential and some additional packages such as the kernel.
>From this repository I can create a pbuilder basetgz that's very close to Stretch 9.0, and allows to build additional packages to bring the "very-close-to-9.0" repo to "really-9.0", as well as up to 9.1. I've also installed a physical machine using a fairly old sid cd where all package versions were older than Stretch 9.0, then upgraded using this repository, so I have a physical machine that's "very-close-to-9.0" but hasn't seen any packages "from the future" to run pbuilder on.
The only problem here is how to automatically select the correct binNMU for sparc64 for a given source package version, for Stretch 9.0. I think it can't be automated (correctly), so I verified it manually for the packages that I've done, but it's unrealistic to do so for the entire archive. Once a certain amount of packages has been made available through this method, it seems easier to start (automatically) recompiling additional packages from source, rather than (manually) pulling in the correct binNMU from snapshot...
And if I do start recompiling a large amount of packages, I intend to optimize them for ultrasparc3 rather than ultrasparc, based on this [1] remark by David Miller.
Bootstrapping Stretch for sparc is less straightforward, as the latest/last packages available from snapshot are from two years ago. Still, there is a large amount of packages available that is not terribly old. Thanks to excellent work by Helmut Grohne and others [2], it's also possible to cross-compile a recent version of the most important packages from source.
It took only minimal patches to get rebootstrap to work against Stretch 9.0. It finishes successfully, and I have repositories available for both sparc and sparc64. Unfortunately, build-essential is not (yet) fully complete: rebootstrap does not (yet) produce a native gcc. At jenkins.debian.net, builds considered successful finish in the same state, so I guess producing all the build dependencies to produce a native gcc is (currently) out of the scope of rebootstrap. I'm working on creating a useful native pbuilder chroot for sparc (similar to what I have for sparc64), either by pulling packages from rebootstrap into a chroot created from snapshot, or by pulling packages from snapshot into the cross-compiling chroot from rebootstrap.
I'm also investigating botch and dose to determine the optimal order for mass-building all the packages. Because so many packages are available, build dependencies will probably not be a problem, but by doing them in the optimal order, it might be possible to get correctly ultrasparc3-optimized versions of all packages in one or maybe two runs, and I think a random order might require more such runs.
Having Stretch repositories with a significant number of packages available for both sparc64 and sparc would open up a lot of opportunities for testing and actually using the port, and having them optimized for ultrasparc3 would allow useful performance testing on a broad range of currently relevant hardware. If I succeed in building the repositories, I hope to make them public.
Tom
> [1] https://patchwork.ozlabs.org/comment/927979/
> [2] https://wiki.debian.org/HelmutGrohne/rebootstrap
Reply to: