Hi!
On Fri, 2023-09-01 at 08:43:55 +1200, Michael Hudson-Doyle wrote:
Recently the topic of exploiting newer instructions without dropping
support for older machines has come up several times inside Ubuntu
engineering. I understand this topic has come up several times in the past
for Debian as well, but nothing has really come of it to date.
I also had a chat about this with Matthias Klose (CCed) around 2022-05.
I've spent a while thinking through the options and coming up with a design
and wrote some notes into a wiki page:
https://wiki.debian.org/ArchitectureVariants
I think we are already doing 1, 2 and 3. I agree 4 is just wrong. And
something like 5 is what I suggested to Matthias for Ubuntu when we
last discussed it as the best way to go about this.
I'm not sure I entirely agree with the requirements you set forth
though:
- I think such optimized builds might need to be done with "special
toolchains" (these could simply be wrappers over the host compiler
passing the appropriate flags via command-line or via specs or
similar, not necessarily full blown toolchains), passing these via
something like dpkg-buildflags seems currently unreliable, as I don't
think we have full coverage in packages (neither for all compilers
available)? Although it would be better as it would centralize the
management. (For reference this is in part how rpm handles this:
https://github.com/rpm-software-management/rpm/blob/master/rpmrc.in)
- Perhaps that's a limitation from the archive software side, but
requiring to place the binary packages in the same pool seems
rather restrictive (it forces different filenames for example).
- I guess it might be nice for the ISA to be passed down to the
dpkg tools, but I don't think this is strictly necessary? A
frontend like apt could also decide based on metadata in say the
Release file, although not having the actual installed package
metadata on whether it was a different ISA build or not would make
its job more inconvenient. In any case I don't have a big issue
with recording this via dpkg-gencontrol or similar if necessary.
On the specific implementation details:
- Changing the Architecture format (as in adding colons there) seems
like a non-starter, and I expect that would break lots of things
(I mean it could be done but I'm not sure it's worth it for this).
Recording this mostly as a hint than anything else, via another
field (if necessary at all) I think would be best.
- As covered in previous discussions, dpkg could (but I don't think
it's necessary) check whether the .deb is runnable on the current
hw, but that's tricky as chrootless installs need to be taken
into account, etc. It should certainly not be part of dependency
resolution.
- I'm not fond of having to change the binary package name format
either for this (name_version_arch.deb) even if at least dpkg
itself does not care (but I know other tools do care), and
depending on the format I'd expect things to break (this goes
back to the shared pool concern).
- If dpkg-architecture needs to be aware of this, then this might need
to be auto-detectable from just the current toolchain being used.
Some of the above problems could perhaps be avoided if we introduced
a concept of architecture aliases/ISAs (similar to what rpm has), which
would side-step the pool sharing issue, the binary package renaming,
etc. One big issue with this is that it requires for dpkg to have an
exhaustive table of all such aliases, and if there's ever a new alias
added, old dpkg versions need to be updated or they will not understand
what they match with. So this does not seem ideal either. So I guess this
is a variation over your proposal, but perhaps this could still be used
in specific contexts, say only at build-time (but not for dependency
relationships), for repo management (say binary-arm64v9/Packages.xz),
or binary package names where the field would specify the actual name
for the filename, say:
Architecture: arm64
ArchitectureIsa: arm64v9
or maybe better:
Architecture: arm64
ArchitectureIsa: v9
resulting in dpkg-deb generating:
binpkg_1.0-1_arm64v9.deb
but targeting arm64. I also think I prefer naming this explicitly as ISA
variants, if you will, than just architecture variants as that gives
way too much room (which perhaps we want, but then that has other
implications over compatibility), and for the field perhaps just Isa is
better, to avoid the implicit repetition of
ArchitectureInstructionSetArchitecture :), but that makes it less easy
to associate both as related.