SOLVED: Software-RAID1 on sarge (AMD64) (was: Re: install-mbr on amd64?)
In the last few days, I was struggling to convert a remote machine with
two identical SATA disks (sda and sdb) to a Software RAID 1. Especially
the boot-part was tricky as I had no console access to the machine. The
whole procedure was done remotely via SSH. I use the md tools (mdadm)
and lilo as bootloader. I chose LILO because IMHO it's more
straightforward in this setup than GRUB and I have no other Operating
Systems I would want to boot.
The system was installed on the first disk, the second one has not been
used before. Those are the steps I went through:
1. Install a Software-RAID capable kernel and boot the system with it;
Install the md tools: 'apt-get install mdadm';
2. partition the second harddrive (sdb). I created two partitions, a
large one at the beginning of the disk (sdb1) and a small
swap-partition at the end (sdb2). I do not use separate /boot
partitions.
NOTE: I do not use two swap spaces on the two disks; instead, I
create a RAID array consisting of the two smaller partitions on the
two discs and create the swap space on it. In case of a disk
failure, I don't need to reboot the system because the swap space
is also on RAID. Otherwise, a disk failure would toast one swap
space, probably leaving the system in a unusable state until
rebooted.
Important: both partitions need to be of the type 0xFD "Linux raid
autodetect"
3. Create the RAID arrays:
$ mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb1
$ mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb2
4. Create filesystems
$ mkfs -t xfs /dev/md0
$ mkswap /dev/md1
I use XFS as filesystem because it has such nice features as online
resizing etc and is, IMHO, very stable and mature. Of course you can
use whatever you like.
5. Copy the existing Debian system to the new RAID
$ mkdir -p /mnt/newroot
$ mount /dev/md0 /mnt/newroot
$ cd /
$ find . -xdev | cpio -pm /mnt/newroot
6. To see if the the new RAID array comes up properly after a reboot,
add the following line to /etc/fstab of the system still running
from sda:
/dev/md0 /mnt/newroot xfs defaults 0 0
Reboot and check with mount if /dev/md0 is mounted properly.
7. I now modified /etc/lilo.conf of the system still running from sda
so that on the next reboot, /dev/md0 would be mounted as root
filesystem while lilo would still access /boot on sda:
# START /etc/lilo.conf
lba32
delay=50
map=/boot/map
boot=/dev/sda
image=/boot/vmlinuz-2.6.16.18
label=RAID
root=/dev/md0
read-only
alias=1
image=/boot/vmlinuz-OLD
label=LinuxOLD
root=/dev/sda1
read-only
alias=2
# END /etc/lilo.conf
This way, we still have a working boot (LinuxOLD) which uses
/dev/sda1 as root in case anything goes wrong (NOTE: sda1 is the
root partition of the old, non-RAID system).
8. Run LILO
First, run
$ lilo -t -v
to see what lilo would do. If everything is OK do:
$ lilo -v
$ lilo -v -R RAID
This way, lilo chooses the image labeled RAID on the next reboot. On
every subsequent reboot, the next entry will be used (so the image
specified with -R gets used only one time). So if the system
doesn't come up, you can reset it and lilo will boot the other
image.
9. Edit the new fstab
The new fstab, located at /mnt/newroot/etc/fstab, must now be
changed so that /dev/md0 gets mounted as root filesystem:
# START /mnt/newroot/etc/fstab
/dev/md0 / xfs defaults 0 0
/dev/md1 swap swap
proc /proc proc
# END /mnt/newroot/etc/fstab
10. Reboot the system. If it comes up, check with mount if /dev/md0
is mounted as root filesystem. If it doesn't come up properly,
just reset the machine / reboot it and it will boot the other
image.
11. Integrade sda into RAID array
First, repartition sda exactly as sdb. The partitions must either
be of the exact size or greater than those on sdb. Also, the
partition type must be 0xFD.
Then, integrate the partitions into the existing RAID array:
$ mdadm --add /dev/md0 /dev/sda1
$ mdadm --add /dev/md1 /dev/sda2
Now the arrays are being synchronized. Check with
$ watch cat /proc/mdstat
that the sync proccess is running. You must wait for this
process to complete on both arrays.
12. Modify lilo.conf
Now we want to boot completely from /dev/md0:
# START /etc/lilo.conf
lba32
boot=/dev/md0
root=/dev/md0
install=/boot/boot-menu.b
map=/boot/map
prompt
delay=50
timeout=50
vga=normal
raid-extra-boot=/dev/sda,/dev/sdb
default=RAID
image=/boot/vmlinuz-2.6.16.18
label=RAID
read-only
root=/dev/md0
alias=1
image=/boot/vmlinuz-OLD
label=LinuxOLD
read-only
root=/dev/sda1
alias=2
# END /etc/lilo.conf
That way, we still have LinuxOLD with which we can boot non-RAID
if it doesn't work out. If RAID is woking, this is strongly
discouraged, as it will cause problems with synchronization if we
boot directly from a disk out of a RAID array.
13. Run lilo
First, run
$ lilo -t -v
to see what lilo would do. If everything is OK do:
$ lilo -v
$ lilo -v -R RAID
When I ran this, lilo complained:
Fatal: Filesystem would be destroyed by LILO boot sector: /dev/md0
Digging in the lilo sources revealed that lilo would spit out this
error if it discovers XFS, NTFS or SWAP on the boot sector of the
device and that it can be forced to go on anyway with the -F flag.
So I ran:
$ lilo -F -v
$ lilo -F -v -R RAID
and everything was OK.
14. Reboot and see if the system comes up properly. If it does, again
run
$ /lilo -v -F
Otherwise, on the next reboot, the non-RAID image would be booted
because we specified it with the -R flag.
Then reboot. Done:
| $ mount
| /dev/md0 on / type xfs (rw)
| proc on /proc type proc (rw)
| sysfs on /sys type sysfs (rw)
| devpts on /dev/pts type devpts (rw,gid=5,mode=620)
| tmpfs on /dev/shm type tmpfs (rw)
| usbfs on /proc/bus/usb type usbfs (rw)
| $ cat /proc/mdstat
| Personalities : [linear] [raid0] [raid1] [raid5] [raid4] [raid6]
| [multipath]
| md1 : active raid1 sdb2[1] sda2[0]
| 1951808 blocks [2/2] [UU]
|
| md0 : active raid1 sdb1[1] sda1[0]
| 154336320 blocks [2/2] [UU]
|
| unused devices: <none>
Because we use the raid-extra-boot=/dev/sda,/dev/sdb directive in
lilo.conf, lilo writes the MBR on /dev/sda AND /dev/sdb. If the first
disk fails, you can still boot from the second this way.
The MBRs are completely identical:
| $ dd if=/dev/sda bs=512 count=1 | od -v | head -n 12
| 1+0 records in
| 1+0 records out
| 512 bytes transferred in 0.000044 seconds (11608020 bytes/sec)
| 0000000 165772 000441 000665 044514 047514 003026 142057 042165
| 0000020 000000 000000 000000 000000 122212 000016 000203 070200
| 0000040 027616 001121 140270 107007 136320 004000 051373 003123
| 0000060 176126 154216 166461 134140 011000 033263 010315 130141
| 0000100 164015 000547 005260 061350 130001 164114 000535 017140
| 0000120 100007 177372 001165 171210 000273 105002 017166 150211
| 0000140 162200 030200 074340 036012 071420 173006 016106 072500
| 0000160 104056 063362 073213 063030 173011 021564 132122 131010
| 0000200 051600 011715 071133 007525 145266 077672 041000 030546
| 0000220 040300 070350 063000 133473 000670 001564 167742 051532
| 0000240 073212 137037 000040 045350 132000 063231 077601 046374
| 0000260 046111 072517 057047 100150 003410 155461 032350 072400
|
|
| $ dd if=/dev/sdb bs=512 count=1 | od -v | head -n 12
| 1+0 records in
| 1+0 records out
| 512 bytes transferred in 0.000043 seconds (11930465 bytes/sec)
| 0000000 165772 000441 000665 044514 047514 003026 142057 042165
| 0000020 000000 000000 000000 000000 000534 034756 000203 070201
| 0000040 027616 001121 140270 107007 136320 004000 051373 003123
| 0000060 176126 154216 166461 134140 011000 033263 010315 130141
| 0000100 164015 000547 005260 061350 130001 164114 000535 017140
| 0000120 100007 177372 001165 171210 000273 105002 017166 150211
| 0000140 162200 030200 074340 036012 071420 173006 016106 072500
| 0000160 104056 063362 073213 063030 173011 021564 132122 131010
| 0000200 051600 011715 071133 007525 145266 077672 041000 030546
| 0000220 040300 070350 063000 133473 000670 001564 167742 051532
| 0000240 073212 137037 000040 045350 132000 063231 077601 046374
| 0000260 046111 072517 057047 100150 003410 155461 032350 072400
Thanks for all the help that I got, your comments are greatly appreciated!
-- Kilian
Reply to: