Root-on-LVM-on-RAID with debian mkinitrd HOWTO
Thanks to the work of others:
http://lists.debian.org/debian-boot/2003/debian-boot-200301/msg00115.html
http://www.midhgard.it/docs/index_en.html
I have a Debian system running with root on LVM on RAID. Kernel
updates, however, require either a manual update of an initrd file, or a
custom kernel compile to include all required modules.
I really wanted to use apt-get/dpkg to install kernel upgrades straight
from the official Debian binaries, so I spent some time and got Debian's
mkinitrd script working with root on LVM on RAID.
NOTE: This procedure does *NOT* require any modifications to the
existing mkinitrd script, it simply makes use of the hooks provided
already to support LVM (RAID is already supported), so there's little
danger of totally breaking something with a debian upgrade, unless the
initrd structure changes drastically.
To get mkinitrd working with LVM, it first has to be prevented from
probing the root device, as it doesn't understand LVM devices. Just set
root=/dev/md1 (or the appropriate raid device) in
/etc/mkinitrd/mkinitrd.conf. With no probing, you also need to
explicitly list the modules required to boot in /etc/mkinitrd/modules.
To start mount root on an LVM partition, it's also necessary to add a
few programs (/etc/mkinitrd/exe), device files (/etc/mkinitrd/files) and
a script to the initial ramdisk. The acutal LVM modules will likely
already be on your initial ramdisk, if you use the default
"MODULES=most" setting in mkinitrd.conf.
The following settings work for me (x86 arch), and should work well for
other architectures if they use the same initial ramdisk scripts from
Debian. I hope to try this with an Alpha system soon, but have not
tried anything but x86 so far.
-----------------------
$ cat /etc/mkinitrd/exe
-----------------------
/bin/cp
/sbin/lvmiopversion
/sbin/vgscan
/sbin/vgchange
/lib/lvm-10/vgscan
/lib/lvm-10/vgchange
/lib/lvm-default/vgscan
/lib/lvm-default/vgchange
-------------------------
$ cat /etc/mkinitrd/files
-------------------------
/dev/lvm
---------------------------------
$ cat /etc/mkinitrd/mkinitrd.conf
---------------------------------
# /etc/mkinitrd/mkinitrd.conf:
# Configuration file for mkinitrd(8). See mkinitrd.conf(5).
#
# This file is meant to be parsed as a shell script.
# What modules to install.
MODULES=most
# The length (in seconds) of the startup delay during which linuxrc may be
# interrupted.
DELAY=0
# If this is set to probe mkinitrd will try to figure out what's needed to
# mount the root file system. This is equivalent to the old PROBE=on
setting.
ROOT=/dev/md1
# This controls the permission of the resulting initrd image.
UMASK=022
# Command to generate the initrd image.
MKIMAGE='mkcramfs %s %s > /dev/null'
-------------------------------------
$ cat /etc/mkinitrd/scripts/00rootlvm
-------------------------------------
#!/bin/sh
exec >> $INITRDDIR/script
cat << "EOF"
mount_lvm_root() {
unset flags fstype rootdev
set -f
set +f $(cat /proc/cmdline)
for i; do
case "$i" in
rootflags=*)
flags=${i#rootflags=}
;;
rootfstype=*)
fstype=${i#rootfstype=}
;;
root=*)
rootdev=${i#root=}
;;
esac
done
if [ -n "$fstype" ]; then
mount -nrt "$fstype" ${flags:+-o "$flags"} $rootdev /mnt
return
fi
IFS=,
set -f
set +f -- $FSTYPES
unset IFS
for i; do
mount -nrt "$i" ${flags:+-o "$flags"} $rootdev /mnt &&
break
done
}
cd /
mount -nt proc proc proc
mount -nt tmpfs tmpfs etc
mount -nt tmpfs tmpfs dev2
cp -a dev/* dev2/
mount --bind dev2 dev
vgscan
vgchange -ay
mount_lvm_root
# Uncomment sh command below to allow manual debugging at boot-time.
# If you can mount root on /mnt, exit the shell and the system will boot
# normally, allowing you to fix whatever was wrong with this script.
# If you can't mount root, boot a working kernel/initrd and try again.
# sh
cd /
echo 256 > proc/sys/kernel/real-root-dev
umount dev
umount dev2
umount etc
umount proc
cd mnt
[ $DEVFS ] && mount -nt devfs devfs dev
pivot_root . initrd
EOF
-------------------------
Notes on 00rootlvm script
-------------------------
The 000rootlvm script appends some commans to the /script file in the
initial ramdisk created by mkinitrd. By the time the code above is run,
the default debian mkinitrd scripts have mounted the required kernel
modules and activiated the raid partition /root resides on. All that is
left is to start LVM and mount the LVM root partiton.
In order for the LVM scritps to work happily, /dev and /etc need to be
read-write, but the default debian mkinitrd is cramfs (read-only). To
get around this, writable partitions are mounted on top of the existing
cramfs directories. The /etc directory can be empty (it will be
populated with lvmtab and lvmtab.d by vgscan), but the /dev directory
needs at least the lvm and null devices (possibly others). I initially
mount a tmpfs filesystem on dev2 and copy the existing devices to avoid
hard-coding any device knowledge in the script. This way, you can
simply add any special devices required for your system to
/etc/mkinitrd/files and they'll automatically be avaialble when the
script above runs. Once the dev2 filesystem is popluated, it's
remounted on top of /dev with mount --bind.
Finally, root is mounted using the root= portion of the kernel command
line for the device (should be /dev/<volumegroup>/<logicalvolume>), and
any supplied kernel command line parameters for mount flags or filesystem.
If you want to give yourself a chance to look around (and maybe fix
anything broken) before just blindly trying to boot, uncomment the 'sh'
command in the script above. When booting, the script will drop to a
shell after (hopefully) mounting root. As long as you exit the shell
with root properly mounted in /mnt, the system should boot normally.
--------------------------
Additional important notes
--------------------------
- Obviously, you need to setup your boot loader to support initial
ramdisks. If you're lucky, there will be an update-* command for your
bootloader that will automatically create/remove boot entries when you
add/remove a kernel. My /etc/kernel-img.conf contains:
do_symlinks = No
do_initrd = Yes
do_bootloader = No
postinst_hook = /sbin/update-grub
postrm_hook = /sbin/update-grub
- You'll need an appropriate setting for root in the kernel command line
provided by your boot-loader. My /boot/grub/menu.lst file contains the
following setting (inside the debian AUTOMAGIC KERNELS LIST markers):
# kopt=ro root=/dev/SystemCore/root console=ttyS0,38400n8
- My system has 2 raid devices, md0 (/boot) and md1 (the big RAID with
logical volumes). Tweak the root= device in mkinitrd.conf to reflect
the RAID device your root filesystem is on.
- I had to add the following to prevent LVM from spitting out a bunch of
(harmless) warnings at boot when the LVM code looks for logical volumes
on *ALL* available block devices. This is likely specific to x86 arch,
and of course you should leave this device properly enabled if you
actually *HAVE* a BIOS RAID. I'm mentioning it here to hopefully save
you a heart-attack, or at least a google search :-) :
# Disable IDE BIOS RAID devices to prevent (harmless) LVM errors
alias block-major-114 off
Please CC: me directly on any comments or requests for more info or
clarification, as I'm currently only subsribed to these lists through
archive searches (which have been very helpful!). :)
--
Charles Steinkuehler
cstein@newtek.com
Reply to: