Re: Help netbooting a diskless, [headless] system [solved/tips]
I got things working and made some discoveries along the way. Maybe I
should put this material on a wiki; any suggestions where?
I "cheated" in a couple of ways. I built the original system image and
initrd on another system with the same architecture as the target. And
I gave up on headless, connecting a keyboard and monitor/TV, though I
probably could have got by without that if I did everything right.
My original view that I could just point the diskless system at an ISO
image (exported over NFS) was mistaken because both the initrd and the
system need to be tweaked to work over NFS.
Also, the target system hardware/BIOS does not appear to support remote
management (as in power on/off, tweak BIOS, access early boot
process--one can use ssh once the system is up). I haven't even gotten
wake on LAN to work, though I have a feeling there's a way.
Relative to regular install, I needed to
1. Set BOOT=NFS and MODULES=netboot in the initramfs.conf that creates
the initrd.
2. Disable automatically bringing up the network interface, since it's
already up (and I think bringing it up screws up the NFS root).
3. Set kernel parameters for NFS root in my syslinux.cfg file
4. Modify fstab for NFS root and selected filesystems on tmpfs
5. Modify the resolver files so that the system knows where the
nameservers and gateways are.
Since the network does not come up in the regular boot sequence (because
it is already up) events that are usually triggered on network up do not
happen. In particular, the resolver does not get set and NFS mounts
never happen. For the latter I put
mount -a -t nfs
in /etc/rc.local--thanks to Bob Proulx.
Here's the startup sequence:
1. Power system on. PXE booting gets basic info from DHCP, and then
uses TFTP to access files.
2. The first file it gets is pxelinux.0 (because that is the file DHCP
says to use). pxelinux.0 come from the syslinux package.
3. Around here the syslinux code takes over. It asks for a
configuration file (via TFTP). It tries a series of names; I named the
config file with the UUID of the booting system to avoid the delay of
multiple searches.
The configuration file is governed by syslinux rules and syntax.
4. Finally the config file specifies a VM and initrd (still files to be
served by TFTP), and the linux kernel takes over.
For basic setup I used advice from
http://www.debian.org/releases/stable/i386/ch04s05.html.en
http://www.linuxquestions.org/questions/debian-26/how-i-did-it-diskless-netboot-with-debian-etch-468870/ (but the fstab recommendation is out of date)
http://johannes-bauer.com/linux/rescue/?menuid=3
http://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt
http://onesis.sourceforge.net/NFSroot-HOWTO.php
http://www.howtogeek.com/61263/how-to-network-boot-pxe-the-ubuntu-livecd/
LTSP also has clues, but it was not what I wanted, since I wanted to run
a complete system, not just an X server.
There are multiple places and ways to specify an NFS root; it may be
possible to do it just in DHCP.
I built the original filesystem, including the pxelinux.0, kernel, and
initrd files to be served by tftp, using a Debian installer in a VM.
I attempted to install the system directly onto the target system
remotely, but could not get that to work. It may be possible if you
preseed enough answers to get an ssh server up; I didn't do any
preseeding.
The target system was amd64 while my server is i386. I tried to
generate an appropriate filesystem using my server and debootstrap
--arch=amd64 --foreign. But the result didn't even have a kernel, and
certainly wasn't ready to be netbooted.
Particulars
ON THE SYSTEM USED TO GENERATE THE INITRD
<initramfs.conf>
MODULES=netboot
BUSYBOX=y
KEYMAP=n
COMPRESS=gzip
DEVICE=
NFSROOT=auto
<initramfs.conf>
ON THE SERVER
<dhcpd.conf (excerpt)>
# docs say next 2 are the defaults anyway
allow booting;
allow bootp;
host mythtv {
hardware ethernet xxxx; #myth box
fixed-address 192.168.40.62;
# added from tftpclient, though some looks as if it should be general
option routers 192.168.40.2;
next-server 192.168.40.2;
option routers 192.168.40.2;
option subnet-mask 255.255.255.0;
# next line is clearly a no-op since this path is oboslete and empty
option root-path "192.168.40.2:/mnt/amd64";
filename "pxelinux.0";
}
</dhcpd.conf>
<config file (name is UUID) for syslinux>
default htpc
LABEL htpc
MENU DEFAULT
KERNEL vmlinuz-3.2.0-4-amd64
APPEND root=/dev/nfs netboot=nfs nfsroot=192.168.40.2:/mnt/htpc initrd=initrd.img-3.2.0-4-amd64 --
timeout 50
totaltimeout 300
</config for syslinux>
I had some trouble using the stock config files, originally obtained for
netbooting the Debian installer, and getting the relative directories
right. I used the stripped down file above instead.
+ setup for NFS exports + installation of the necessary files for tftp
ON THE DISKLESS TARGET SYSTEM
<fstab>
proc /proc proc defaults 0 0
/dev/nfs / nfs defaults 0 0
none /tmp tmpfs defaults 0 0
# Remove /var/run and /var/lock from your fstab; they will probably be
# being mounted on top of /run and /run/lock, which will cause you
# additional problems. See tmpfs(5). They are now tmpfs by default
# on all systems, so you don't need them. Also, the mode isn't being
# set, so it's also insecure.
# Roger Leigh 2012-12-09
#none /var/run tmpfs defaults 0 0
#none /var/lock tmpfs defaults 0 0
none /var/tmp tmpfs defaults 0 0
none /media tmpfs defaults 0 0
</fstab>
<etc/network/interfaces>
auto lo
iface lo inet loopback
# The primary network interface
# do not bring up interface twice--PXE already did it
#allow-hotplug eth0
# if you accidentally try to bring the interface up
# and it is manual, things will be OK. If it is dhcp,
# it will be brought down and you're hosed.
iface eth0 inet manual
</etc/network/interfaces>
# next file presumes resolvconf package is installed
<etc/resolvconf/resolv.conf.d/base>
# PXE brings up networking before the regular system starts
# To preserve NFS root, we do not bring eth0 up later, and so
# the regular machinery to set things up doesn't fire.
# Instead, establish the essentials here.
nameserver 192.168.40.2
domain betterworld.us
</etc/resolvconf/resolv.conf.d/base>
Thanks to Michael Biebl, Paul Gardiner, Neal Murphy, Roger Leigh, Andrei
Popescu, and especially Bob Proulx who conducted a lot of tests to see
how things work ("NFS automount not happening" thread starting
2012-12-08).
My target system is running testing (~ squeeze); my server is on Lenny.
Ross Boylan
On Sun, 2012-11-11 at 18:49 -0800, Ross Boylan wrote:
> I have a computer that is not attached to a monitor or keyboard and has
> no disk/CD.
>
> I thought getting it to boot over the network would be easier than it
> has proven, and would appreciate any pointers. Searching turned up a
> lot of material, but none of it exactly on topic.
>
> My naive view was that I could point the system at an ISO image and have
> something working. Or switch to nfs early on and get everything from
> there.
>
> I've found instructions about netbooting the Debian installer, but the
> instructions seem to be for a system that is diskless but not headless.
> I haven't found anything indicating the installer can be run via ssh.
> There are also some issues with the paths in the config files which
> cause it to start asking for files in the wrong locations part way
> through the PXE process (e.g.,
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=530616). This was with
> the testing installer.
>
> I do have the basic dhcp/tftp infrastructure working; wireshark shows
> the dhcp dialogue, and then tftp serving up 1 or more files before
> failing to find ones with bad paths. As noted, even if I get past that
> I think the installer wants to interact with me on the client machine,
> and so that won't work.
>
> Other instructions on the net call for building an image of the system
> locally. One issue is that the server is 32 bit lenny, while the client
> is 64 bit and really should run 64 bit to use all its memory. I do have
> a testing chroot on the server, and perhaps could use it to get started.
>
> Eventually I probably will want to customize the system running on the
> client; since an ISO image read-only maybe using them isn't a good idea.
>
> Anyway, I'm a bit disoriented and would appreciate some orientation.
>
> I will attach the machine to a monitor eventually, and probably will get
> a wireless keyboard. But since clusters have lots of nodes with just a
> CPU, and I know Debian runs on them, I figure there's some way to get it
> working.
>
> Thanks.
> Ross Boylan
>
Reply to: