Re: multiarchitecture binaries - technical obstacles?
(Multiply replies collapsed and addressed)
Years ago, NeXT modified GCC and the rest of the GNU tools to allow
them to produce multi-architecture binaries, so that a single binary
executable could run on both 68k and i386 platforms. They also had a
tool that could strip out hunks for unwanted architectures.
Most of the tools only have minimal changes to support fat builds above
and
beyond the necessary support for the Mach-O file format. For instance,
when you invoke
"cc -arch ppc -arch i386 -c test.c", the compiler backend is invoked
twice (one is a native
compiler, the other a cross compiler to x86). Each assembly source
files is assembled
separately (again using a native and a cross assembler), and then
lipo(1) is used to
join the two object files into a single Mach-O fat file (with a fat
header). Most of the heavy
lifting is just parsing the -arch flags and knowing the correct path to
invoke, and when to
call lipo(1).
The current Mac OS X kernel, when preparing to execve() a file, reads
in the Mach-O
header. If the header is a fat header, one level of indirection is
employed to find the
correct architecture and start executing the correct code.
If you are interested in where and how this code operates, you may want
to look at some
of the Darwin source code at <http://developer.apple.com/darwin/>.
'xnu' is the Mac OS X/Darwin
kernel, 'gcc3' is the gcc 3.1-based compiler used for Mac OS X 10.2,
'cctools' includes
ld(1), as(1) and lipo(1).
Maybe that code has decayed, and maybe it had problems ... but it sure
would be nice if it could be resurrected and used by us, so that
Debian became able to produce multiplatform .deb containing such
multi-architecture executables.
It is alive and well in Mac OS X. The entire Darwin x86 release was
cross compiled from Mac OS X 10.2.
Are you suggesting adding support to multi-architecture ELF files to
the Linux kernel, or just some
metadata in .deb files such that only executables of the correct
architecture are installed at install time?
Here are a few potential benefits: it would reduce the total volume of
the archive;
I assume you mean because a lot of non-executable information like
documentation and support files would not need to be replicated in a
bunch of arch-specific debs. Keep in mind that there maybe be issues
with non-executable data files that are endian-specific which would be
implicitly correct during native building but perhaps not for cross
building. I can't think of any examples of this offhand though.
eliminate separate i386/Pentium/686/MMX-optimized packages
Mach-O support generic architecture types as well as cpu subtypes. On
Mac OS X, I've created a fat
file with an object code for a G3 subtype and a G4 subtype. The kernel
executes the most specific subtype, so
a single executable does the right thing for each type. Arguably, this
is less efficient than using primarily G3
code and just doing (ifG4) { g4optimized() } else { g3optimized() },
since the majority of the code need not be optimized.
reduce the burden on the autobuilders;
Using cross building like this would indeed allow a farm of x86
machines to compile all architectures, instead of needing to support
lots of builder types, which may or may not be as stable as x86. This
is quite a bit different than traditional cross-compiling, which
generates code that can never execute on the host. With the Mac OS X
toolchain, since intermediate and final binaries have a native portion
as well as a cross portion, way more programs can be trivially cross
compiled. For instance, consider a program like XFree86 which compiles
lots of auxiliary tools to process the source, before actually invoking
the compiler. This would be difficult to cross compile without hacking
the build system to death to figure out what needs to be compiled
native and what should be cross-compiled.
reduce cross-architecture version-skew; catch
cross-architecture compilation problems earlier; ensure widespread
availability of binary packages for multiple architectures; make the
upcoming deployment of competing 64-bit CPUs from Intel and AMD less
painful for our users; and be a unique feather in Debian's cap
These would be pretty cool advantages. Depending on the maturity of
IA64 and x86-64 linux ports, it might be advantageous to defer using
native builders.
Doing this might reduce the archive size, but it would increase
bandwidth use on the mirrors, since either way the packages would be
bigger.
Not necessarily. You could build all packages fat, and then
post-process them to thin out to all the various support architectures
before uploading to mirrors. This way, you get the binary packages
built without version skew, but the traditional delivery method is
still used.
Shantonu Sen
Reply to: