Re: RAID 1 on Sun E250 running 3.0 sparc r1
Daren,
I'm replying to the list directly, although I'm not subscribed. If anyone
has replies to this message, they're probably better off going to this list
where everyone can benefit. With that said...
I wrote a somewhat cryptic guide on setting up a root RAID for Linux Sparc64
on my Ultra 30. You can find the most up-to-date version here:
http://www.doorbot.com/guides/linux/sparc64/rootraid/
I used Linux 2.4.18 with the RAID 1 patch for Sparc64 (to work around the
egcs64 miscompile). Below is a copy of revision 0.2 of the guide:
<begin ramblings>
Mirrored Root RAID On Sparc64 Linux
Summary: Experiences configuring a Sun Ultra 30 to boot off of a mirrored
software RAID under Debian Linux with kernel version 2.4.18.
As part of my migration to a Sun Ultra 30 from my previous Slackware-based
server, I wanted to utilize the Linux built-in software raid ("md") to
protect my root (and boot) partition. I made many mistakes along the way,
and this guide will not attempt to recount all of them. It will attempt to
explain how I (finally) got my UltraSparc to boot off of a software RAID1,
and point out a few of the pitfalls so readers can avoid them.
My chosen operating system is Debian "Woody" running the Linux kernel
version 2.4.18. Newer versions of the Linux kernel are available, and you
may chose to use one of these instead. To start, I did a base install of
Debian, being sure to install the latest versions of the packages as
recommended by apt and dselect.
The RAID devices will be built from partitions on two 18 GB SCSI drives,
/dev/sda and /dev/sdb. The layout is as follows:
* /dev/md0 - /dev/sda1, /dev/sdb1 - RAID1 (Mirror) - Mounted as / (root
directory)
* /dev/md1 - /dev/sda4, /dev/sdb4 - RAID0 (Stripe) - Mounted as /tmp
* /dev/md2 - /dev/sda5, /dev/sdb5 - RAID1 (Mirror) - Mounted as /var/log
* /dev/md3 - /dev/sda6, /dev/sdb6 - RAID0 (Stripe) - Mounted as /var
(mounts before /var/log)
* /dev/md4 - /dev/sda7, /dev/sdb7 - RAID0 (Stripe) - Mounted as /usr
* /dev/md5 - /dev/sda8, /dev/sdb8 - RAID0 (Stripe) - Mounted as /home
* swap - /dev/sda2, /dev/sdb2 - Striped swap (each use "priority=0" as a
mount option)
I started out by getting the Woody install image on CD. In OBP, while the
memory is counting, hit L1-A (Stop-A), then "boot cdrom" to start the Debian
install. The ISO I downloaded would not boot normally with SILO; use the
"rescue" feature and the system will boot off the CD as expected. I
partitioned both drives so they both had identical partition maps. Note that
partition 3 usually fills the whole disk. When this was done, I selected
/dev/sda8 as my root partition, and installed Debian to it.
To compile a kernel on Debian-Sparc64, you need to install a few packages
first: egcs64, libncurses-dev, dpkg-dev, and kernel-package as well as the
"regular" compiler tools. In all likelihood GCC and friends are already
installed, but you will need to install egcs64 to compile the 64-bit kernel
for Sparc. You can install the required packages with the following command:
apt-get install egcs64 libncurses-dev dpkg-dev kernel-package
You can also add these packages through dselect, using the '/' key to find
them by name. If you're adventurous, GCC 3.2.1 may work; it is available as
a package from Debian "Sid" ("unstable" as of this writing).
Next I downloaded the kernel sources. I wanted to use 2.4.20, which was the
latest available, but I always got "mcount" errors on all the modules. After
I finished with this project, I stumbled upon a solution
(http://lists.debian.org/debian-sparc/2002/debian-sparc-200212/msg00124.html)
on the Debian-Sparc Mailing List. I ended up with version 2.4.18, but you
may end up with a different version. You can get the kernel sources via apt,
using a command such as "apt-get source kernel-image-2.4.18" but I opted to
download the source directly from the Linux Kernel Archives. Be sure to get
the "full" source and not just one of the patches. I extracted them (tar
xzvf file.tgz) to /usr/src/linux-2.4.18 and then changed to that directory.
If you're running any kernel prior to 2.4.20, you will need to patch the
RAID1 support in the kernel, as egcs64 miscompiles it on Sparc, and will
lead to an "oops" when you try to write to your new software raid. Here is
the patch
(http://marc.theaimsgroup.com/?l=linux-kernel&m=103075370821523&w=2) from
the Linux Kernel Mailing List.
Configure your kernel as necessary and then compile it; you may wish to
refer to the Debian FAQ on building custom kernels. Be sure to answer "yes"
to RAID support, and verify that RAID1 support is compiled into the kernel
(instead of a module). When you have finished with the kernel configuration,
save the configuration, and run the "make-kpkg" command with the appropriate
arguments to build the new kernel. Next, install the kernel using the "dpkg"
command.
If you already have RAID support enabled in the kernel, we can configure the
RAID devices. Otherwise, reboot the system. Save the following RAID
configuration (/etc/raidtab):
# / raiddev /dev/md0
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 4
persistent-superblock 1
device /dev/sda1
raid-disk 0
device /dev/sdb1
raid-disk 1
# /tmp
raiddev /dev/md1
raid-level 0
nr-raid-disks 2
nr-spare-disks 0
chunk-size 16
persistent-superblock 1
device /dev/sda4
raid-disk 0
device /dev/sdb4
raid-disk 1
# /var/log
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 4
persistent-superblock 1
device /dev/sda5
raid-disk 0
device /dev/sdb5
raid-disk 1
# /var
raiddev /dev/md3
raid-level 0
nr-raid-disks 2
nr-spare-disks 0
chunk-size 16
persistent-superblock 1
device /dev/sda6
raid-disk 0
device /dev/sdb6
raid-disk 1
# /usr
raiddev /dev/md4
raid-level 0
nr-raid-disks 2
nr-spare-disks 0
chunk-size 16
persistent-superblock 1
device /dev/sda7
raid-disk 0
device /dev/sdb7
raid-disk 1
# /home
#raiddev /dev/md5
# raid-level 0
# nr-raid-disks 2
# nr-spare-disks 0
# chunk-size 16
# persistent-superblock 1
# device /dev/sda8
# raid-disk 0
# device /dev/sdb8
# raid-disk 1
Note that /home will not be created until we are finished building the rest
of the RAID devices. Later, you will need to enable /home by uncommenting
the lines below "# /home" by removing the pound sign/hash sign ("#") at the
beginning of each line.
Now we want to make the actual RAID drives and mount them. Run the commands:
mkraid /dev/md0 mke2fs -j /dev/md0
Adjust for each RAID device. This will create the RAIDs and filesystems on
the devices. You can check the RAID build status with the following command:
cat /proc/mdstat
To configure SILO in preparation for the root RAID, I added an
"append=md=0,/dev/sda1,/dev/sdb1" line to /etc/silo.conf and verified that
the root partition was /dev/sda1 (it might work with /dev/md0, but I don't
see how SILO would know /dev/md0 pointed to /dev/sda1 and /dev/sdb1). My
silo.conf looks like this:
partition=1
append="md=0,/dev/sda1,/dev/sdb1"
timeout=0
root=/dev/sda1
image=1/vmlinuz
label=linux
read-only
In the mean time, if you need to reboot for some reason, you can enter
"disk:8 linux root=/dev/sda8" in OBP and/or "linux root=/dev/sda8" at the
SILO prompt to get back to our temporary boot partition. Install SILO into
the boot block as follows:
silo -f
Mount the root device under a suitable directory; I used /mnt but any empty
directory will suffice (but keep it simple). Mount the rest of the devices
under /mnt so what will end up being /usr is currently /mnt/usr and so on.
Now, we will want to copy all of our data over to the new RAID devices. This
can be accomplished a number of ways (usually involving the cp -cipR dir1
dir2 dir3 --targetdir=/mnt command). Be careful that you don't copy /mnt
into /mnt/mnt or you will find your free disk space vanish. Alternately, you
can use the "failed disk" method
(http://marc.theaimsgroup.com/?l=linux-sparc&m=101787980731878&w=2) to get
the data onto the RAID devices.
Reboot and in OBP select "disk:1" as the primary boot device and your new
RAID should boot.
Finally, adjust /etc/raidtab to enable the /home RAID device, make the RAID,
and then the filesystem. You should now be able to mount it and copy the
contents of /home to it.
<end ramblings>
Hopefully you find this helpful!
- Philip L. McMahon
Reply to: