An introduction to multiarch
Hi,
after listening to the "Multiarch round table" talk at Debconf I feel
that the talk was targeted at people already familiar with the subject
and jumped right in at full speed. Someone new to the idea was
probably lost in the first minute.
So I decided to write an introduction to the problem that focuses more
on the user side, why we want multiarch at all, what goals and
requirements are to be met and gives some pointers. I leave the
techincal details to the actual proposal. Hopefully that helps more
people to jump on the wagon and help. Note that this is to be taken
together with the proposal itself:
https://wiki.ubuntu.com/MultiarchSpec
Steve, please take a look at the pointers section 2 and 3 below for
things diverging from the wiki.
So here goes nothing:
An introduction to multiarch
============================
What do we have currently?
--------------------------
Most architectures in Debian (all but alpha) are capable of executing
binaries compiled for different ABIs (Aplication Binary Interface).
The difference can be the 32bit or 64bit instruction sets as present
on amd64 / emt64 CPUs.
Now we face the problem that some software is only available for some
ABIs. For example wine only runs in 32bit mode as it emulates the win32
ABI or Google Earth is only available in 32bit. So while nearly all of
the amd64 packages are 64bit some need to be 32bit or are only
available in 32bit.
The solution to that currently is bi-/tri-arch packages. That means
that libraries are build on one architecture for multiple ABIs. The is
a libbz2 package, a lib32bz2 and a lib64bz2 package. The later two are
compiled using /usr/lib32 and /usr/lib64 as libdir respectively. This
has some major drawbacks:
1) Bi-/tri-arch is limited to exactly those 2/3 ABIs.
2) Libraries need to be compiled multiple times with special cases for
each architecture depending on what ABIs each architecture has.
3) Only a verry few libs are compiled bi-/tri-arch. The majority of
32bit libraries needed on amd64 come out of the ia32-libs /
ia32-apt-get hacks which is verry verry ugly.
Where do we want to go? What is the goal of multiarch?
------------------------------------------------------
Overall multiarch tries to unify sveral different use cases into a
straight forward and simple solution avoiding special casing
architectures or multiple compilations.
- On amd64 we want to be able to install libraries and binaries from
i386 (32bit) and amd64 (64bit). E.g. run wine or google earth.
- On hppa, sparc, mips, mipsel, powepc and s390 we want to be able to
install 32bit for most things but also 64bit packages for some
special cases where a 64bit address space or 64bit registers
outweigh the cost of the larger instruction set and pointers.
E.g. run a 64bit postgresql using >4GB ram.
- For embedded use we want to be able to install arm libraries on for
example amd64 and cross-compile packages.
- On kFreeBSD we want to be able to install Linux binaries.
- For fun we want to be able to install ppc binaries on mips and run
them with qemu.
Design goals and requirements:
------------------------------
The current proposal fullfills the following goals and requirements:
- library packages from multiple architectures must be installable at
the same time
- binaries must be installable from multiple architectures but one at
a time is sufficient
- new debs must work on old Debian (for the one architecture)
- old debs must work on new Debian, esspecially 3rd party ones
- only change what must
- switching to multiarch must work in increments, idealy deb by deb
- dependencies must track where a matching ABI is required and where
the ABI is inconsequential.
Those garanty a smooth transition from uni-arch to multi-arch.
The following goal is delayed for a later time to get the ball rolling
but nothing in the proposal actualy stands in the way of it. It is
just left out as a time saver for the start.
- Developement packages from multiple architectures must be
installable at the same time. This is important for cross-compiling.
The multiarch proposal (some pointers)
--------------------------------------
Please read the multiarch specs in parallel. The below is ment as
helpfull hints acompaning the specs. The full multiarch specs can be
found on: https://wiki.ubuntu.com/MultiarchSpec
The proposal contains 4 major points:
1) Filesystem layout
For library packages from multiple architectures to be installable at
the same time there can be no files in the same location for different
architectures. On each architecture the location of files must be
unique. The choosen unqiue identifier in the proposal is the GNU
triplet (DEB_HOST_GNU_TYPE). This is unique for each Debian
architecture and even for things not (yet) in Debian like building
against uclibc.
The GNU triplet is introduced for every arch and it is always the
$(DEB_HOST_GNU_TYPE). There is no special casing to figure out the
--libdir setting for a particular architecture. This simplifies rules
files considerably.
2) Binary package control files
To track where in the dependencies the ABI must match and where not
the package management needs some help. This actualy involves 2
seperate things. One that can be implemented imediatly and one needing
a release cycle to take effect:
a) Add Multi-Arch field (same/foreign)
The Multi-Arch field is introduced into the depended on package and
governs how depending packages behave. That means Debian packages need
to be changed and then existing 3rd party debs like google earth will
become installable. The 3rd party debs itself does not need to change
to benefit from Debian going multiarch.
This covers all cases of dependencies between binaries and
libraries. It does not cover scripting languages with binary
extensions (perl, python, ...) where some dependencies must match the
ABI and some don't.
In particular all bi-/tri-arch cases currently supported are covered
by this.
b) Add foreign-arch relationship / Multi-Arch: allowed
Some packages can have both dependies that must match the ABI and
dependies where the ABI is inconsequential. For example a library
binding for perl (e.g. for gtk) contains binary data that must have
the same ABI as the perl interpreter. On the other hand a binary
package (not arch: all) may also contain architecture independent perl
scripts. A perl interpreter from any architecture will run them just
fine.
For this the proposal adds "Multi-Arch: allowed" and "Depends:
<package>:any" for this. The "Multi-Arch: allowed" is only there as a
saftey precaution so packages don't wrongfully use "Depends:
<package>:any".
Unfortunately this needs changes in the archive software (DAK) and the
package management. It will break compatibility with existing Debian
installation so it needs to do be introduced with a release cycle
between implementation and actual use.
Add-on:
The current implementation restricts the new dependency syntax to
"<package>:any". Earlier on there was also "<package>:<architecture>"
on the board but is now listed as unsuported by the
implementation. Hopefully this will come back. A use case for this
will be wine. It could look something like this:
Package: wine
Version: 1.0.1-2
Depends: wine-bin
Suggests: wine-bin:i386
Recommends: wine-bin:amd64
Conflicts: wine-bin (<< 1.0.1-2), wine-bin (>> 1.0.1-2)
The reason for this is that future wine will support both the win32
ABI and win64 ABI allowing executing 32bit and 64bit windows
applications. Wine is usefull with either 32bit or 64bit support (so
it depends on wine-bin for any architecture) but nearly all users want
the 32bit wine.
3) apt sources extension
Here I have to disagree with the proposal and I hope the proposal will
be adjusted. So here is my counter proposal to this:
The default architectures for a system will be configured via
Apt::Architectures in /etc/apt/apt.conf. But sometimes an entry should
not use the default. The existing Apt already supports the following
syntax:
deb [whatever you like] url suite component ...
I propose to use this instead of appending the architectures at the
end of the line. Also it should be possible to put other things in
there. So I propose to use
deb [key1=val1,val2,val3;key2=val4,val5;...] url suite component ...
Or as specific example:
deb [arch=i386] http://download.skype.com/linux/repos/debian/ stable non-free
This keeps the soruces.list syntax compatible with existing apt and
reuses the already existing parser code. A patch for this is already
in the BTS as it can be added imediatly and would be usefull for
ia32-apt-get.
Steve please consider adapting this syntax and update the wiki.
4) Implementation (migration)
For most users activating multiarch would be a simple matter of
putting
Apt::Architectures {"amd64"; "i386";};
in /etc/apt/apt.conf.d/multiarch (or any other apt conffile). This
could also be done with a "multiarch" package that could auto detect
possible architectures or ask with debconf.
Only a few users would need to adjust their sources.list with the
extended syntax in (3). Even of those few most will only need the
extended syntax to add new repositories that currently can not be
added.
Migrating ia32-libs, ia32-libs-gtk and amd64-libs will be tricky if
not impossible to automate. The release notes might have to say that
the user has to replace all packages depending on them with the
respective i386 or amd64 packages after enabling multiarch.
MfG
Goswin
Reply to: