Re: Problems/workarounds for install to root on LVM on RAID
OK, I tried this again from 'bare metal', and verified the overall
procedure works, but there are a couple of additional issues and
corrections (inline).
Using the 20040526 netinst image, there seem to be no problems with the
initial ramdisk image created (it starts both the md device and lvm,
allowing root to be mounted) as has been reported with earlier versions.
The only initrd problem seems to be it's desire to default to LVM1
instead of LVM2 if both are detected...
Charles Steinkuehler wrote:
I'm trying to install testing onto an x86 box with root on LVM on RAID1,
and I ran into several issues with the latest daily image:
http://cdimage.debian.org/pub/cdimage-testing/sarge_d-i/i386/20040526/sarge-i386-netinst.iso
First, the problems:
--------------------
1) partman won't let you build an LVM on top of a RAID device
2) Kernel install portion of base install fails due to mkinitrd failure
2a) LVM tools missing
2b) Raidtools (actually mdadm) missing
2c) LVM entries in /dev missing on target
2d) initrd defaults to LVM1 instead of LVM2
3) Grub install fails when /boot is on a RAID1 device
4) md devices (other than the one holding root) are not started at boot
Workarounds:
------------
1) partman won't let you build an LVM on top of a RAID device
This is the easiest to fix...just switch over to the console and
manually run pvcreate, etc. to build the LVM VG(s) & LV(s). Once
created, the logical volums are properly recognized by partman, allowing
them to be formatted, mount points to be set, etc.
NOTE: You have to run at least pvcreate and vgcreate manually, ie:
pvcreate /dev/md/1
vgcreate -A n VolGroup /dev/md/1
You can then use partman to create logical volumes.
------------
2) Kernel install portion of base install fails due to mkinitrd failure
2a) LVM tools missing
2b) Raidtools (actually mdadm) missing
Prior to installing the base system, switch to a console and edit
/usr/lib/debootstrap/scripts/sarge, adding the following modules to the
base="..." line near the top of the file:
LVM Packages: binutils libdevmapper1.00 lvm-common lvm2
RAID Packages: mdadm
------------
2c) LVM entries in /dev missing
Switch to a console and copy the entries from the install system to the
target system:
cp -aR /dev/<vg> /target/dev/
cp -aR /dev/mapper/<vg>* /target/dev/mapper/
------------
2d) initrd defaults to LVM1 instead of LVM2
This is a nasty one, and oddly confusing. If both LVM2 and LVM1 are
present (as determined by the existance of appropriate kernel modules),
LVM1 is selected. This seems odd, since LVM2 can talk to LVM1 formatted
volumes, but not the other way around. Anyway, to fix this I did the
following:
- Add initrd (plus dash and cramfsprogs dependencies) to base= line in
sarge script (see 2a & 2b, above) prior to installing base system
Should be initrd-tools instead of initrd!
- chroot to target system
- Prior to installing kernel, edit /etc/mkinitrd/mkinitrd.conf and set
root=/dev/hda1 (a dummy entry that allows mkinitrd to complete
successfully, letting the install continue)
- Install kernel 2.4.26-1-386
- Move (or delete) /lib/modules/2.4.16-1-386/kernel/drivers/md/lvm-mod.o
so mkinitrd won't find it and default to LVM1
- mount -t proc proc /proc
- verify /etc/mkinitrd/mkinitrd.conf now lists root=probe (kernel
install apparently causes config mods to be lost)
- run mkinitrd to replace currently broken root=/dev/hda1 image:
mkinitrd -o /boot/initrd.img-2.4.26-1-386 2.4.26-1-386
- umount /proc
------------
3) Grub install fails when /boot is on a RAID1 device
Apparently, grub is trying to get 'smarter' about devices, but in the
process is breaking functionality that used to work. With /boot
installed on a raid1 device (md0), grub-install and update-grub both
fail dismally, reporting that /dev/md0 doesn't exist as a BIOS device
(like that's a news flash!).
The stupid hack to get this to work is:
- Install grub from the debian installer menu (it will fail)
- Switch to a console and edit /boot/grub/device.map, replacing the
(hd0) /dev/??? entry with (hd0) /dev/md0
- Re-run install grub from the debian installer menu (it will still
fail, but get farther this time)
- Run update-grub from a console to create /boot/grub/menu.lst
- run grub and install grub to both HDD's:
root (hd0,0)
setup (hd0)
setup (hd1)
- Continue w/o boot loader from the debian installer menu (you manually
installed grub)
It's worth noting that grub from stable has no problems with /boot on a
raid device (once installed). I can see grub-install perhaps failing
with /boot on a md device, but why in the world does update-grub even
care where boot is?!? All it needs to do is edit menu.lst...
NOTE: Your device numbers might be different. Also, it's typically OK
to install grub with root set to (hd0,0) on both hd0 and hd1, since if
hd0 fails, hd1 will usually become hd0 with most BIOS's.
NOTE: When I tried this a second time, root was defaulting to (hd0,) in
menu.lst. I'm not sure if this was a goof on my part or a problem with
the above procedure. To fix, edit menu.lst and set the default grub
root device to (hd0,0) in the AUTOMAGIC KERNELS LIST section:
# groot=(hd0,0)
and run update-grub. If you forget and reboot anyway, all is not lost:
hit 'e' at the grub menu to manually edit the boot entry, and you should
be able to successfully boot, allowing you to fix menu.lst.
------------
4) Prior to rebooting, edit /etc/mdadm/debian.conf and set:
AUTOSTART=true
I think you might get the option to set this durring install if the
debconf priority is set low enough (I've been running at 'medium', and
wasn't prompted about starting all raid arrays at boot).
NOTE: This can be fixed after rebooting, if required. If not fixed
before reboot, the fsck of /boot will fail (md device is not started),
so just hit enter to drop into maintainence mode, edit
/etc/mdadm/debian.conf, then reboot.
------------
Reboot and cross your fingers!!!!
I can provide additional information, if needed or desired.
--
Charles Steinkuehler
charles@steinkuehler.net
Reply to: