Bug#265868: root-on-LVM-on-RAID workarounds
Package: installation-reports
INSTALL REPORT
Debian-installer-version: Version RC1 on USB stick. Files:
http://people.debian.org/~joeyh/d-i/images/2004-08-09/hd-media/boot.img.gz
http://cdimage.debian.org/pub/cdimage-testing/sarge_d-i/i386/rc1/sarge-i386-businesscard.iso
uname -a: Linux OlafTowr 2.6.7-1-k7 #1 Thu Jul 8 06:45:35 EDT 2004 i686 GNU/Linux
Date: Sa Aug 14 23:31:44 CEST 2004
Method: Booted from USB-stick, install w/ kernel26 image in "normal" mode (but needed console).
Used ftp2.de.debian.org debian mirror, un-proxied.
Machine: Noname PC w/ 3 SATA drives
Processor: AMD Athlon(tm) XP 2200+
Memory: 256MB
Root Device: LVM-on-RAID from 3 SATA hard-drives
(/dev/mapper/vg0-root -> /dev/md/1 -> /dev/discs/disc[012]/part3)
Root Size/partition table:
# sfdisk -d /dev/sda
# partition table of /dev/sda
unit: sectors
/dev/sda1 : start= 1, size= 144584, Id=fd, bootable
/dev/sda2 : start= 144585, size= 530145, Id=82
/dev/sda3 : start= 674730, size=311901975, Id=fd
/dev/sda4 : start= 0, size= 0, Id= 0
(Note: sdb and sdc the same)
# cat /proc/mdstat
Personalities : [raid1] [raid5]
md0 : active raid1 sda1[0] sdc1[2] sdb1[1]
72192 blocks [3/3] [UUU]
md1 : active raid5 sda3[0] sdc3[2] sdb3[1]
311901824 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
unused devices: <none>
# lvscan
Ignoring /dev/md: No such file or directory
ACTIVE '/dev/vg0/root' [256.00 MB] inherit
ACTIVE '/dev/vg0/home' [2.00 GB] inherit
ACTIVE '/dev/vg0/tmp' [512.00 MB] inherit
ACTIVE '/dev/vg0/usr' [2.00 GB] inherit
ACTIVE '/dev/vg0/var' [1.00 GB] inherit
ACTIVE '/dev/vg0/data' [200.00 GB] inherit
Output of lspci and lspci -n:
#lspci
0000:00:00.0 Host bridge: VIA Technologies, Inc. VT8366/A/7 [Apollo KT266/A/333]0000:00:01.0 PCI bridge: VIA Technologies, Inc. VT8366/A/7 [Apollo KT266/A/333 AGP]
0000:00:09.0 RAID bus controller: Silicon Image, Inc. (formerly CMD Technology Inc) SiI 3114 [SATALink/SATARaid] Serial ATA Controller (rev 02)
0000:00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 80)
0000:00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 80)
0000:00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 80)
0000:00:10.3 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 82)
0000:00:11.0 ISA bridge: VIA Technologies, Inc. VT8235 ISA Bridge
0000:00:11.5 Multimedia audio controller: VIA Technologies, Inc. VT8233/A/8235/8237 AC97 Audio Controller (rev 50)
0000:00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev
74)
0000:01:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200 AGP (rev
01)
#lspci -n
0000:00:00.0 0600: 1106:3099
0000:00:01.0 0604: 1106:b099
0000:00:09.0 0104: 1095:3114 (rev 02)
0000:00:10.0 0c03: 1106:3038 (rev 80)
0000:00:10.1 0c03: 1106:3038 (rev 80)
0000:00:10.2 0c03: 1106:3038 (rev 80)
0000:00:10.3 0c03: 1106:3104 (rev 82)
0000:00:11.0 0601: 1106:3177
0000:00:11.5 0401: 1106:3059 (rev 50)
0000:00:12.0 0200: 1106:3065 (rev 74)
0000:01:00.0 0300: 102b:0521 (rev 01)
Base System Installation Checklist:
[O] = OK, [E] = Error (please elaborate below), [ ] = didn't try it
Initial boot worked: [O]
Configure network HW: [O]
Config network: [O]
Detect CD: [ ]
Load installer modules: [O]
Detect hard drives: [O]
Partition hard drives: [E]
Create file systems: [O]
Mount partitions: [O]
Install base system: [E]
Install boot loader: [O]
Reboot: [E]
Comments/Problems:
Hi, many thanks for this wonderful installer. It helped a lot. I wanted a
not officially supported root-on-LVM-on-RAID setup, and it did warn about
this. The needed changes ARE small, but ...
Partition hard drives:
----------------------
Wishlist-list:
- make it possible to create a LVM PV on top of a RAID.
- Manage Raid-level 5 with mdcfg
Workaround: Go to the console and do
# sfdisk -uM /dev/discs/disc0/disc <<EOF
...
EOF
# sfdisk -d /dev/discs/disc0/disc | sfdisc /dev/discs/disc1/disc
and for all other drives
# modprobe md
# mdadm -C /dev/md/0 -l1 -n3 /dev/discs/disc[012]/part1
# mdadm -C /dev/md/1 -l5 -n3 /dev/discs/disc[012]/part3
or to your liking
# modprobe dm-mod
# vgscan
# pvcreate -ff /dev/md/1
# vgcreate -A n vg0 /dev/md/1
# lvcreate -A n -L 256 -n root vg0
and for all other partitions
Then continue with the partitioner normally (it needs to reread the
partition table).
Install base system:
--------------------
debootstrap at some point fails, because it cannot umount /target/proc.
I couldn't confirm this for sure, but probably, this is due to a running
mdadm -F process left over from automatically installing mdadm-1.6.0-1.
Workaraund:
Edit /usr/lib/debootstrap/functions, in the function setup_proc make
these changes:
on_exit "umount $TARGET/proc/bus/usb || true"
+ on_exit "sleep 10"
+ on_exit "umount $TARGET/proc || true"
- on_exit "umount $TARGET/proc"
The sleep gives you a little time to kill the mdadm-process (see below)
and the "|| true" at the umount keeps the thing from failing (it does even
WITH a killed mdadm. I don't see, why).
To kill the mdadm, have this command running during the base installation:
# kline=""; while [ -z "$kline" ]; do
kline=`ps ax|grep -v grep|grep "mdadm -F"`;
done; kuid=`echo $kline|sed "s/ *([0123456789]*).*/1/"`;
kill $kuid
I umounted /target/proc by hand afterwards.
This is a dirty hack! For a better solution, find out, what exactly
causes the umount to fail in the script. If I do it by hand, it works
as long as the mdadm is killed before. But the script still fails, if
the "|| true" is missing... Why? For a better solution:
- debootstrap could try and find all processes using /proc and kill
them before the umount.
- All packages, that start demons, should not do so during debootstrap
(maybe set a defined config-entry debootstrap/installing ?)
Wishlist:
- if /boot is a RAID1 array, call grub-install for all of its drives.
Reboot:
-------
As mentioned in bug-report #262092 for initrd-tools, vgchange does not work
properly in the initrd. What happens seems to be:
On scanning /dev (if no devfs is mounted on top of it), vgchange skips the
symbolic links for /dev/md -> ../devfs/md and thus does not find the physical
volume (if 'scan = [ "/dev" ]' in /etc/lvm/lvm.conf)
If a devfs is mounted on /dev, it crashes (or devfs_mk_dir crashes?) on certain
devices.
Quick-Fix: specify exactly in the /etc/lvm/lvm.conf, which device contains the PV.
For me, the entries are:
scan = [ "/dev/md", "/dev" ]
filter = [ "a|/dev/md/1|", "a|/dev/md1|", "r|.*|" ]
with all other "filter =" entries commented out.
Better solutions:
a) find out, what vgchange does that makes the devfs fail
b) do not mount a devfs on top of /dev in the initrd and make
vgchange follow symlinks.
(Can this last part be forwarded to the Bug #262092 ?)
For more infos, contact me. I am looking forward to this thing becoming
the default installer.
Install logs and other status info is available in /var/log/debian-installer/.
Once you have filled out this report, mail it to submit@bugs.debian.org.
Reply to: