cross-compiling the kernel
I'm sending these notes to the list because some people might be
interested, and some people might have advice for me about how I could
have done it better.
Building a SPARC kernel on an Intel machine
-------------------------------------------
I did this partly because my Intel machine (233 BogoMIPS CyrixInstead,
64 MB) can compile the kernel about ten times faster than my
SPARCstation 2 (40 BogoMIPS, 32 MB), and partly because I wanted to
see if I could get a cross-compiler to work.
I used egcs-1.1.2 and binutils-2.9.1.0.19a on an i686 running Debian
GNU/Linux 2.1, but with kernel 2.2.7. I took the header files from the
SPARC version of Debian 2.1.
First make a directory, henceforth referred to as $b, in a file system
with plenty of space for building a cross-compiler (about 150 MB):
$ b=/tmp/ccc ; mkdir -p $b ; cd $b
Unpack the source packages:
$ tar xzf binutils_2.9.1.0.19a.orig.tar.gz
$ bzcat egcs-1.1.2.tar.bz2
Rather than download egcs-1.1.2, what I actually did was apply the
patch from soon-to-be-renamed-Cygnus's FTP site to Debian's source
package:
$ tar xzf egcs_1.1.1.orig.tar.gz
$ mkdir xx ; cd xx ; ../egcs-1.1.1.orig/configure ; cd ..
$ mv -i egcs-1.1.1.orig egcs-1.1.2
$ cd egcs-1.1.2 ; bzcat egcs-1.1.1-1.1.2.diff.bz2 | patch -p1 -s
$ cd .. ; rm -rf xx
(Why did I configure egcs-1.1.1 before applying the patch? Because it
seems to be necessary. I haven't investigated why.)
Make a link to binutils from the egcs directory:
$ cd egcs-1.1.2 ; ln -s ../binutils-2.9.1.0.19a binutils ; cd ..
Get a local copy of the target machine's header files. I don't know
which ones you really need, so I just take the whole damn lot. If you
have NFS-mounted the SPARC's root directory at $SUNROOT:
$ mkdir headers ; cd headers
$ ( cd $SUNROOT/usr/include ; tar cf - * ) | tar xf -
$ cd ..
For general-purpose cross-compilation you might also want to get a
local copy of the target machine's standard libraries, but you don't
need them just for building the kernel.
Unless you change the default (using --prefix=/whatever with
configure) the build process will try to make some directories under
/usr/local/sun4-linux, so make this directory, and make it writable.
As root:
# mkdir /usr/local/sun4-linux ; chmod a+rwxt /usr/local/sun4-linux
Make an object directory, cd to it, configure and build:
$ mkdir objdir ; cd objdir
$ ../egcs-1.1.2/configure --target=sun4-linux --with-headers=$b/headers
$ make cross
(If things go horribly wrong and you want to make a fresh start, make
sure you clean out both objdir and /usr/local/sun4-linux.)
This first attempt to build chugs along happily for a long time, but
then grinds to a halt at this point:
_muldi3
as: option `-s' is ambiguous
This is because $b/objdir/gcc/xgcc is invoking the wrong "as". Enquire
where it's looking with:
$ $b/objdir/gcc/xgcc --print-search-dirs | grep ^prog
We can fix this problem by installing "as" by hand:
$ mkdir /usr/local/sun4-linux/bin
$ ln $b/objdir/binutils/gas/as-new /usr/local/sun4-linux/bin/as
$ make cross
But it gets no further:
/bin/sh: sun4-linux-ar: command not found
...
I fixed this by putting the new "ar" on the path thus:
$ PATH=$b/p:$PATH
$ mkdir ../p
$ ln -s ../objdir/binutils/binutils/ar ../p/sun4-linux-ar
$ make cross
This time it gets a bit further:
Configuring in sun4-linux/libio
...
checking whether the C compiler (... ) works... no
configure: error: installation or configuration problem: C compiler cannot create executables.
Judging by the "FIXME" in egcs-1.1.2/libiberty/configure, this is not
a new problem. I don't know the solution, but I note that the build
process failed while trying to build the runtime libraries; the
cross-compiler has already been built. I couldn't get "make install"
or "make install-cross" to work, but I could install some of the bits
by hand ("as" has already been installed):
$ ln binutils/binutils/ar /usr/local/sun4-linux/bin/ar
$ ln gcc/cc1 /usr/local/sun4-linux/bin/cc1
$ ln gcc/gcc-cross /usr/local/sun4-linux/bin/gcc
$ ln binutils/ld/ld-new /usr/local/sun4-linux/bin/ld
$ ln binutils/binutils/nm-new /usr/local/sun4-linux/bin/nm
$ ln binutils/binutils/objdump /usr/local/sun4-linux/bin/objdump
$ ln binutils/binutils/strip-new /usr/local/sun4-linux/bin/strip
(You have to install the programs where you said you were going to
because some of the executables run the others using file paths that
were compiled into them. I think this is a damn nuisance. In an
alternative universe, would it better if there were a mechanism by
which executables could locate files relative to themselves? That
might be good for configuration files too ...)
(I did try hacking egcs-1.1.2/libiberty/configure to get past the
configure error, but I then got conflicting types for sys_errlist in
libiberty/strerror.c:461, and more errors when I fixed that ...)
You might want to change the owner back to root, by the way:
# chown -R root:root /usr/local/sun4-linux
# chmod a=rX,u+w /usr/local/sun4-linux
Now I can compile a simple program, provided I don't try to link it:
$ /usr/local/sun4-linux/bin/gcc -c /tmp/prog.c -o /tmp/prog.o
Link it and run it on the target machine:
sun$ egcc prog.o -o prog
sun$ ./prog
Hello, world!
We can't link on the host machine because there aren't any libraries,
but we don't need any libraries for compiling the kernel, so we just
ignore that problem for now and unpack the kernel source instead:
$ cd $b
$ bzcat linux-2.2.7.tar.bz2 | tar xf -
$ cd linux ; patch -p1 -s < your_patches_that_linus_rejected
Edit the Makefile to read:
ARCH := sparc
CROSS_COMPILE = /usr/local/sun4-linux/bin/
Copy a .config from somewhere else, if you want, or "make xconfig",
noting with pleasure that it doesn't ask you what kind of Pentium you
have, then do the standard:
$ make dep
$ make clean
$ make
$ make modules
$ make INSTALL_MOD_PATH=$b/modules modules_install
Now you have a "vmlinux" and modules. Try them to see if they work ...
You might also try comparing the binaries with ones built natively. If
you used the same compiler version and configuration in each case the
binaries should be identical apart from the embedded date stamps.
Cross-compiling and linking
---------------------------
What about compiling and linking an ordinary C program on the host
machine? This is harder because you have to provide some target
libraries in /usr/local/sun4-linux/lib/, and getting linkers to work
seems to be a frequent problem even for veteran hackers.
Of course you could use your cross-compiler to build the target
libraries on the host machine, but I just copied them from Debian
SPARC.
A minimum set of libraries for compiling a statically linked "Hello,
world" program appears to be: crt1.o crtbegin.o crtend.o crti.o crtn.o
libc.a libgcc.a.
A minimum set of libraries for compiling a dynamically linked "Hello,
world" appears to be: crt1.o crtbegin.o crtend.o crti.o crtn.o
ld-2.0.105.so ld-linux.so.2 libc-2.0.105.so libc.so libc.so.6
libc_nonshared.a libgcc.a; ld-linux.so.2 is a link to ld-2.0.105.so,
libc.so.6 is a link to libc-2.0.105.so, and libc.so is a little script
containing some file paths which you must alter to make it work.
When the inevitable linking problems appear, it's worth adding
"-Wl,--verbose" as an argument to gcc. Then you can see what the
linker is trying to do. It's probably trying to link Intel libraries
with the SPARC binaries. Unfortunately, if the linker can't find what
it wants in /usr/local/sun4-linux/lib it goes and looks in /lib or
/usr/lib. Preventing it from stubbornly trying to link with
/lib/libc.so.6 can be quite trying at times but don't make the mistake
which my manager once made of moving that file out of the way
temporarily!
Exercise: See if you can build the complete Debian binary distribution
for SPARC, from source, on an Intel machine. Special credit will be
given for a robust and reusable solution for cross-building tetex and
emacs ...
Edmund <edmundo@rano.demon.co.uk>, 1999-05-10
Reply to: