raid software troubles (long)
Hi,
many days ago my Debian server stopped working because one hard disk
broke down.
So I attached a new disk and rebuilded the array successfully.
The server came back up online and I decided to make a reboot.
During bootup phase, the shell returned this message and the process
stopped, suggesting me 2 things:
1. press ctrl+D to go on
2. type root password to access recovery consolle.
(it repeats for every md device...)
fsck.ext3 invalid argument while trying to open /dev/mdX The
superblock could not be read or does not describe a correct ext2
filesystem
If the device is valid and it really contains an ext2 fs (and not swap or
ufs or something else),
then the superblock is corrupt, and you might try running e2fsck with an
alternate superblock:
e2fsck -b 8193 <device>
fsck failed.please repair it manually.
I followed first suggestion (ctrl+D to go on) but the system could
just access to /boot mount point (/dev/md0)...
Then I followed second suggestion, following these steps:
. mdrun (start the raid service)
. mdadm /dev/mdX -a /dev/sdbX (rebuild all arrays)
. mount /deviceMD /mount_point
. exit the recovery console
The above steps worked and let my server finally boot :)
But now if I reboot it stops again showing the same message error...
Please, how can i solve this problem ?
Some useful details...
/etc/fstab
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0
/dev/md0 / ext3 defaults,errors=remount-ro 0 1
/dev/md2 /tmp ext3 defaults 0 2
/dev/md1 /usr ext3 defaults 0 2
/dev/md4 /usr/local ext3 defaults 0 2
/dev/md3 /var ext3 defaults 0 2
/dev/sda2 none swap sw 0 0
/dev/sdb2 none swap sw 0 0
/dev/sdc /media/usb0 auto rw,user,noauto 0 0
/etc/mdadm/mdadm.conf
--------------------------
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=cb7c2f86:92c4fce4:65ba022f:e74122$
devices=/dev/sda5,/dev/sdb5
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=4e0ccf9d:24a4558f:291b4a3a:dbfd6f$
devices=/dev/sdb6
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=cd73e87c:0b15fe41:427f5dab:f6013a$
devices=/dev/sda7,/dev/sdb7
ARRAY /dev/md4 level=raid1 num-devices=2 UUID=6e716c4b:deb9c372:5404f35f:f9eb59$
devices=/dev/sda8,/dev/sdb8
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=3a46ba90:59f0c3fc:7bbef7c8:bba4dc$
devices=/dev/sda1,/dev/sdb1
the string ",/dev/sda6" misses in /dev/md2.... why ? o_O
# cat /proc/mdstat
Personalities : [raid0] [raid1] [raid5]
md4 : active raid1 sda8[0] sdb8[1]
3903680 blocks [2/2] [UU]
md3 : active raid1 sdb7[0] sda7[1]
29294400 blocks [2/2] [UU]
md2 : active raid1 sdb6[0] sda6[1]
1951744 blocks [2/2] [UU]
md1 : active raid1 sdb5[0] sda5[1]
3903680 blocks [2/2] [UU]
md0 : active raid1 sdb1[0] sda1[1]
1951744 blocks [2/2] [UU]
unused devices: <none>
# mdadm --misc --query /dev/md0
/dev/md0: 1.86GiB raid1 2 devices, 0 spares. Use mdadm --detail for more detail
/dev/md0: No md super block found, not an md component.
# mdadm --misc --query /dev/md1
/dev/md1: 3.72GiB raid1 2 devices, 0 spares. Use mdadm --detail for more detail
/dev/md1: No md super block found, not an md component.
srv-mx1:/boot/grub# mdadm --misc --query /dev/md2
/dev/md2: 1.86GiB raid1 2 devices, 0 spares. Use mdadm --detail for more detail
/dev/md2: No md super block found, not an md component.
srv-mx1:/boot/grub# mdadm --misc --query /dev/md3
/dev/md3: 27.94GiB raid1 2 devices, 0 spares. Use mdadm --detail for more detail
/dev/md3: No md super block found, not an md component.
srv-mx1:/boot/grub# mdadm --misc --query /dev/md4
/dev/md4: 3.72GiB raid1 2 devices, 0 spares. Use mdadm --detail for more detail
/dev/md4: No md super block found, not an md component.
Superblock ? How can I fix this ?
# mdadm --misc --examine /dev/md0
mdadm: No super block found on /dev/md0 (Expected magic a92b4efc, got 00000000)
mdadm --misc --examine /dev/md1
mdadm: No super block found on /dev/md1 (Expected magic a92b4efc, got 00000000)
# mdadm --misc --examine /dev/md2
mdadm: No super block found on /dev/md2 (Expected magic a92b4efc, got 00000000)
# mdadm --misc --examine /dev/md3
mdadm: No super block found on /dev/md3 (Expected magic a92b4efc, got 00000000)
# mdadm --misc --examine /dev/md4
mdadm: No super block found on /dev/md4 (Expected magic a92b4efc, got 00000000)
Superblock error again..
# mdadm --misc --query --detail /dev/md0
/dev/md0:
Version : 00.90.01
Creation Time : Tue Mar 14 12:28:34 2006
Raid Level : raid1
Array Size : 1951744 (1.86 GiB 2.00 GB)
Device Size : 1951744 (1.86 GiB 2.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Tue Dec 26 22:03:14 2006
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 3a46ba90:59f0c3fc:7bbef7c8:bba4dcdc
Events : 0.2006858
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 1 1 active sync /dev/sda1
# mdadm --misc --query --detail /dev/md1
/dev/md1:
Version : 00.90.01
Creation Time : Tue Mar 14 12:28:43 2006
Raid Level : raid1
Array Size : 3903680 (3.72 GiB 4.00 GB)
Device Size : 3903680 (3.72 GiB 4.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent
Update Time : Tue Dec 26 22:03:48 2006
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : cb7c2f86:92c4fce4:65ba022f:e7412230
Events : 0.1975861
Number Major Minor RaidDevice State
0 8 21 0 active sync /dev/sdb5
1 8 5 1 active sync /dev/sda5
# mdadm --misc --query --detail /dev/md2
/dev/md2:
Version : 00.90.01
Creation Time : Tue Mar 14 12:28:50 2006
Raid Level : raid1
Array Size : 1951744 (1.86 GiB 2.00 GB)
Device Size : 1951744 (1.86 GiB 2.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Persistence : Superblock is persistent
Update Time : Tue Dec 26 21:57:32 2006
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 4e0ccf9d:24a4558f:291b4a3a:dbfd6fe2
Events : 0.1237841
Number Major Minor RaidDevice State
0 8 22 0 active sync /dev/sdb6
1 8 6 1 active sync /dev/sda6
# mdadm --misc --query --detail /dev/md3
/dev/md3:
Version : 00.90.01
Creation Time : Tue Mar 14 12:29:05 2006
Raid Level : raid1
Array Size : 29294400 (27.94 GiB 30.00 GB)
Device Size : 29294400 (27.94 GiB 30.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 3
Persistence : Superblock is persistent
Update Time : Tue Dec 26 21:58:05 2006
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : cd73e87c:0b15fe41:427f5dab:f6013a06
Events : 0.7559914
Number Major Minor RaidDevice State
0 8 23 0 active sync /dev/sdb7
1 8 7 1 active sync /dev/sda7
# mdadm --misc --query --detail /dev/md4
/dev/md4:
Version : 00.90.01
Creation Time : Thu Dec 21 03:07:52 2006
Raid Level : raid1
Array Size : 3903680 (3.72 GiB 4.00 GB)
Device Size : 3903680 (3.72 GiB 4.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 4
Persistence : Superblock is persistent
Update Time : Tue Dec 26 21:58:16 2006
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 56630b43:37792ebf:dab9c0e7:e3a4081e
Events : 0.26536
Number Major Minor RaidDevice State
0 8 8 0 active sync /dev/sda8
1 8 24 1 active sync /dev/sdb8
-----
Bye.
Reply to: