Problems/workarounds for install to root on LVM on RAID
I'm trying to install testing onto an x86 box with root on LVM on RAID1,
and I ran into several issues with the latest daily image:
http://cdimage.debian.org/pub/cdimage-testing/sarge_d-i/i386/20040526/sarge-i386-netinst.iso
First, the problems:
--------------------
1) partman won't let you build an LVM on top of a RAID device
2) Kernel install portion of base install fails due to mkinitrd failure
2a) LVM tools missing
2b) Raidtools (actually mdadm) missing
2c) LVM entries in /dev missing on target
2d) initrd defaults to LVM1 instead of LVM2
3) Grub install fails when /boot is on a RAID1 device
Workarounds:
------------
1) partman won't let you build an LVM on top of a RAID device
This is the easiest to fix...just switch over to the console and
manually run pvcreate, etc. to build the LVM VG(s) & LV(s). Once
created, the logical volums are properly recognized by partman, allowing
them to be formatted, mount points to be set, etc.
------------
2) Kernel install portion of base install fails due to mkinitrd failure
2a) LVM tools missing
2b) Raidtools (actually mdadm) missing
Prior to installing the base system, switch to a console and edit
/usr/lib/debootstrap/scripts/sarge, adding the following modules to the
base="..." line near the top of the file:
LVM Packages: binutils libdevmapper1.00 lvm-common lvm2
RAID Packages: mdadm
------------
2c) LVM entries in /dev missing
Switch to a console and copy the entries from the install system to the
target system:
cp -aR /dev/<vg> /target/dev/
cp -aR /dev/mapper/<vg>* /target/dev/mapper/
------------
2d) initrd defaults to LVM1 instead of LVM2
This is a nasty one, and oddly confusing. If both LVM2 and LVM1 are
present (as determined by the existance of appropriate kernel modules),
LVM1 is selected. This seems odd, since LVM2 can talk to LVM1 formatted
volumes, but not the other way around. Anyway, to fix this I did the
following:
- Add initrd (plus dash and cramfsprogs dependencies) to base= line in
sarge script (see 2a & 2b, above) prior to installing base system
- chroot to target system
- Prior to installing kernel, edit /etc/mkinitrd/mkinitrd.conf and set
root=/dev/hda1 (a dummy entry that allows mkinitrd to complete
successfully, letting the install continue)
- Install kernel 2.4.26-1-386
- Move (or delete) /lib/modules/2.4.16-1-386/kernel/drivers/md/lvm-mod.o
so mkinitrd won't find it and default to LVM1
- mount -t proc proc /proc
- verify /etc/mkinitrd/mkinitrd.conf now lists root=probe (kernel
install apparently causes config mods to be lost)
- run mkinitrd to replace currently broken root=/dev/hda1 image:
mkinitrd -o /boot/initrd.img-2.4.26-1-386 2.4.26-1-386
- umount /proc
------------
3) Grub install fails when /boot is on a RAID1 device
Apparently, grub is trying to get 'smarter' about devices, but in the
process is breaking functionality that used to work. With /boot
installed on a raid1 device (md0), grub-install and update-grub both
fail dismally, reporting that /dev/md0 doesn't exist as a BIOS device
(like that's a news flash!).
The stupid hack to get this to work is:
- Install grub from the debian installer menu (it will fail)
- Switch to a console and edit /boot/grub/device.map, replacing the
(hd0) /dev/??? entry with (hd0) /dev/md0
- Re-run install grub from the debian installer menu (it will still
fail, but get farther this time)
- Run update-grub from a console to create /boot/grub/menu.lst
- run grub and install grub to both HDD's:
root (hd0,0)
setup (hd0)
setup (hd1)
- Continue w/o boot loader from the debian installer menu (you manually
installed grub)
It's worth noting that grub from stable has no problems with /boot on a
raid device (once installed). I can see grub-install perhaps failing
with /boot on a md device, but why in the world does update-grub even
care where boot is?!? All it needs to do is edit menu.lst...
NOTE: Your device numbers might be different. Also, it's typically OK
to install grub with root set to (hd0,0) on both hd0 and hd1, since if
hd0 fails, hd1 will usually become hd0 with most BIOS's.
------------
Reboot and cross your fingers!!!!
I can provide additional information, if needed or desired.
--
Charles Steinkuehler
charles@steinkuehler.net
Reply to: