"New" Installation Method of the Hurd, please tes, develop and comment
Hi,
On Fri, Jul 23, 1999 at 06:37:10PM +0000, Nuno Emanuel F. Carvalho wrote:
> > The stuff you need to play with it fits in 100 MB, but you need more for a
> > "full" installation (wow :). "full" means what we have, not what we will
> > have when we are ready (when we are ready, you will need 2 GB for a full
> > installation, but that's far away...)
>
> Well, 1GB it's enough ... for now :)
Absolutely. that's also the current upper limit for GNU Mach to grok
partitions.
> > But I don't recommend vm ware for two reasons: It is very slow, and it's
> > non-free.
>
> Slow, I can take it. non-free ... maybe someday it change. ;)
I doubt that this would be soon. VM Ware is a big succes for the company,
and they would be stupid to free it as they can make a lot of money out of
this innovation. That's okay, it's their code. For us, we should focus on
Free Software, right?
Instead struggeling with VM Ware, it would be nice if someone could
investigate if the booting-the-hurd-from-a-diskimage-inside-any-ext2fs does
work, as it was implemented by Roland some time back. This would have two
advantages:
* No repartitioning is required.
* We could ship a compressed disk image that is completely preconfiure. No
need to do any configuration with native-install or whatsoever.
Here is what Roland wrote about it back in Dec 1998:
I hacked together a new hurd feature that makes it possible to boot without
having a disk partition dedicated to the hurd. It's kind of a kludge, but
I threw it together quickly and it ought to do. I barely tested this at
all, but enough that other hackers should be able to muddle through.
You need the most-current hurd from cvs to get this code (there is a
snapshot on the way soon). Actually, I checked it in late enough last
night/this morning that I'm not sure if the anoncvs mirrors have it yet.
To compile the current hurd without errors, you'll need the current glibc
from cvs as of the wee hours this morning (for unrelated reasons--I was
just a'hacking last night); there will hopefully be a glibc-2.0.109 fairly
soon.
[Of course, our recent packages is all what is required -- MB]
My point here is to explain the code changes and roughly how you use the
new feature, so that other folks can hack on it and put together a coherent
set of instructions for general consumption. Incoherence commences.
There is a new option for bootstrap filesystems in diskfs (i.e. ufs and
ext2fs), --boot-command', which specifies a command to run instead of
init. This command can be another bootstrap filesystem, which runs with
the original bootstrap filesystem as its root directory.
The idea is that the bootstrap filesystem and the exec server are started
up as usual, but rather than accessing /hurd/init and all the hurd servers
through that filesystem, it can run another bootstrap filesystem program
and give it as its storage, access to a regular file in the original
bootstrap filesystem that contains a filesystem image for the inner
bootstrap filesystem.
An example, using ext2fs. You'll recall the lines in /boot/servers.boot
that usually start up the bootstrap filesystem and the exec server:
/hurd/ext2fs.static --bootflags=${boot-args} --host-priv-port=${host-port} --device-master-port=${device-port} --exec-server-task=${exec-task} -Tdevice ${root-device} $(task-create) $(task-resume)
/lib/ld.so.1 /hurd/exec $(exec-task=task-create)
For this to work, serverboot must be able to read /hurd/ext2fs.static and
/lib/ld.so.1, and that ext2fs.static must be able to read /hurd/exec and
the shared libraries it needs.
For using --boot-command, let's modify that boot script:
/hurd-install/hurd/ext2fs.static --bootflags=${boot-args} --host-priv-port=${host-port} --device-master-port=${device-port} --exec-server-task=${exec-task} -Tdevice ${root-device} $(task-create) $(task-resume) --chroot=/hurd-install --boot-command /hurd/ext2fs.static --bootflags=${boot-args} hurd-root.img
/hurd-install/lib/ld.so.1 /hurd/exec $(exec-task=task-create)
Here I assume you have a directory called /hurd-install on the partition
you're booting from; this can be, e.g. your cross-compilation install
directory on your linux system (i.e. where you did
"make prefix=/hurd-install install" after cross-building libc and hurd).
But you only really need a few files for this to work. serverboot reads
/hurd-install/hurd/ext2fs.static and /hurd-install/lib/ld.so.1, and then
the bootstrap filesystem is running on the real root partition. Notice we
give it a --chroot option, which makes it pretend the directory
/hurd-install is the root of the filesystem. Now it starts the dynamic
linker that serverboot loaded into memory, and its root filesystem is the
tree under /hurd-install provided by the bootstrap ext2fs. There it needs
to find /hurd/exec and all the shared libraries it uses (use ldd or objdump
--private-headers to figure it out); then we have an exec server. At this
point the bootstrap fs would ordinarily run /hurd/init.
Now --boot-command comes in. Note that the --boot-command switch must be
the last thing on the command line (after the bootstrap root device), and
all arguments following it (with or without dashes) become the command line
for the boot command. This command runs, as init would, with a root and
current directory provided by the bootstrap filesystem (that /hurd-install
directory). Here we have /hurd/ext2fs.static again, though it could as
well be /hurd/ufs.static (in theory you could use a dynamically-linked fs
program here, but I wouldn't recommend it).
This ext2fs process gets the same --bootflags argument, which tells it to
be a bootstrap filesystem. But rather than having a device as its storage,
it can use a regular file found in the root filesystem provided by its
parent (we don't give a -Tfile option because it's the default). Now this
second ext2fs reads its filesystem image from the file hurd-root.img (aka
/hurd-install/hurd-root.img). Noticing that it didn't get the magical
--host-priv-port and --device-master-port options from serverboot, it
decides it must be a boot command and so does the fsys_getpriv RPC on its
bootstrap port (as init would) to get the privileged ports from its parent.
Then it notices it didn't get the magical --exec-server-task option, and
does a normal lookup of /servers/exec in the root filesystem provided by
its parent, just as any non-bootstrap filesystem would do. For this
reason, your /hurd-install directory must contain a /servers/exec file; an
empty file will do, and it doesn't need any special setup.
Now we have a filesystem and it has a port to the exec server, and it
proceeds just like a vanilla bootstrap filesystem would: it loads
/hurd/init from the filesystem it provides itself, and init takes over and
completes the server bootstrapping procedure as normal.
In the penultimate step of the normal bootstrap dance, init sends the
fsys_init RPC to the filesystem that loaded it. This is our second-stage
bootstrap filesystem. Knowing it's a boot command, it registers its parent
task with proc server and then turns around and sends another fsys_init to
its parent, the original bootstrap filesystem. On receiving this
fsys_init, just as it would from init, the bootstrap filesystem registers
the exec server task with the proc server, and then sends it the exec_init
RPC with its proc and auth ports. This allows the exec server to get the
privileged ports from the proc server, and complete the bootstrap handshake
by registering itself with init. Et voila, off to the races.
A final wrinkle concerns the exec server. Recall that we reach the exec
server through its active translator setting on /servers/exec--but that was
the /servers/exec in the original root filesystem, which noone but the
secondary bootstrap filesystem ever had access to (and it abandoned those
directory ports in the final phase of bootstrapping the servers with init).
Since the filesystem caches its port to the exec server, the root
filesystem will continue to be able to execute programs. However, any
other filesystem will start with the new root directory, and look up
/servers/exec there to reach the exec server. So, it is now necessary to
have set a passive translator of /hurd/exec on /servers/exec in the inner
root filesystem, which was never necessary before. This passive translator
setting is harmless in the normal booting case because the boot-time active
translation of /servers/exec causes it to be ignored, so it will become a
normal part of hurd installation. (It would be possible to pass on the
active translator setting to the inner root's /servers/exec, but doing it
this way lets you use a newer or different exec server binary and it can
share the library text pages with the rest of the system.)
The savvy hacker already knows how to create a filesystem image in a file,
but here are some tips anyway. Always the first step is to decide how big
a filesystem you want (i.e. how big a partition you would use if you were
using a partition), and make a file that big:
# dd if=/dev/zero of=/hurd-install/hurd.img bs=1024k count=300
That's a 300MB filesystem, which is big enough for a full gnu-0.2
installation and some hacking space.
Next, create the filesystem. On the Hurd and Linux this is easy; for
ext2fs:
# mke2fs -o hurd /hurd-install/hurd.img
For ufs on the Hurd, and maybe this works on BSD:
# mkfs.ufs /hurd-install/hurd.img
On the Hurd, "mounting" the filesystem from the image is easy:
# settrans -a /mnt /hurd/ext2fs /hurd-install/hurd.img
to set just an active translator (mount it once), or replace -a with -p for
a passive translator (mounted on demand whenever you use /mnt).
On Linux, you have to use the "loop" device:
# losetup /dev/loop0 /hurd-install/hurd.img
On BSD, it's the "vnd" device:
# vnconfig -c /dev/vnd0c /hurd-install/hurd.img
If it turns you on, you can do the equivalent on the Hurd too:
# settrans -a /dev/whatever /hurd/storeio -Tfile /hurd-install/hurd.img
Then you can use /dev/whatever as a block device:
Linux # mount /dev/loop0 /mnt
BSD # mount /dev/vnd0c /mnt
Hurd # settrans -a /mnt /hurd/{ext2fs,ufs} /dev/whatever
Populate /mnt to your heart's content, and then unmount it:
# umount /mnt
and detach the device:
Linux # losetup -d /dev/loop0
BSD # vnconfig -u /dev/vnd0c
And you should be ready to go. (If you're trying this from a running Hurd,
you can just use `fsysopts /mnt -r; sync' to sync the data and then boot
the the sub-Hurd with `boot'.)
This is all in theory of course. When I tried to do heavy filesystem
writing through /dev/loop0, it crashed linux-2.0.36. Since I have a lot
more and faster spare disk than I have patience for linux bugs, I just used
a spare partition to populate a filesystem and then dumped the image from
the partition into a file.
I'll leave it to the rest of you to work out the kinks in this procedure.
Someone could also use this to set up and distribute a "miniroot"
filesystem image that could be loaded from your linux or bsd partition with
ext2fs/ufs and uncompressed into memory with -Tgunzip, to provide a minimal
hurd environment for installation before we get something pared down to
floppy size.
Note that you should in theory be able to stack multiple boot commands, for
a filesystem within a filesystem within a filesystem as many levels as you
want, so you can think up the possibilities. It's also probably possible
with a little more work to set up a slightly more populated trampoline boot
filesystem with passive translators like pfinet and nfs set so as to
approximate an "almost-diskless" network boot of the hurd, or other
shenanigans. (There are several more hacks necessary to make that work
though.)
I hope someone finds this useful.
Enjoy,
Roland
D'oh! I forgot the last part of my message.
Those of you who are *really* paying attention must be wondering about the
active translator setting on the outer root's /servers/exec. Previously
the exec server set itself as active translator on /servers/exec after
receiving exec_init. But exec_init doesn't come until the outer bootfs
gets fsys_init from the inner bootfs, after the servers are all running.
So now the exec server acts more like a normal passive translator: the
bootstrap filesystem sets the active translator on /servers/exec after
receiving the fsys_startup RPC from exec.
A wart in the code is that for --boot-command to be parsed properly by the
diskfs argp code, every bootstrap filesystem must use the ARGP_IN_ORDER
flag in its argp_parse call.
--
`Rhubarb is no Egyptian god.' Debian http://www.debian.org finger brinkmd@
Marcus Brinkmann GNU http://www.gnu.org master.debian.org
Marcus.Brinkmann@ruhr-uni-bochum.de for public PGP Key
http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Reply to: