Bug#433905: initramfs-tools: Boot failure from software-RAID1 with debian on nslu2
Package: initramfs-tools
Version: 0.88
Severity: critical
Justification: breaks the whole system
The Debian installation on my nslu2 is not bootable/accessible,
unless I solder a serial port onto the circuit board. Great fun,
but it should be avoided. The reason is that the initramfs scripts
are not able to initialize the root file system, which is in this
case a 'regular' partition on a RAID-1 volume over two usb-disks.
(no lvm or lvm2 involved). I have two equal usb disks attached in
this configuration:
root: /dev/md1, a raid1 with /dev/sda1 and /dev/sdb1 as slaves
swap: /dev/md2, a raid1 with /dev/sda2 and /dev/sdb2 as slaves
home: /dev/md3, a raid1 with /dev/sda3 and /dev/sdb3 as slaves
The problem is that the usb disks are not yet availlable when the
the root delay loop is entered. Unfortunately, the device /dev/md1
is created with mdadm before the loop is entered. During this loop,
the script checks whether the device /dev/md1 contains a sane
file system. When the usb disks are registered, /dev/md1 does not
become valid.
I think, the only way to solve this, is to modify the scripts in the
initramfs. The current scripts (0.88) check every 0.1 seconds to see
whether the root device has become valid. When this is the case, the
root delay is interrupted. Instead of checking only this, the script
should also try to assemble the raid1 root every 0.1 seconds,
otherwise /dev/md1 will never contain a sane file system.
The simple workaround for me, was to add the following line to the
local script in the root_delay loop:
/sbin/mdadm --assemble --auto=yes --no-degraded /dev/md1
(I don't want to boot from a degraded root device, if possible. This
is another issue with the initramfs scripts, but this should be
discussed in an another big report. Let's first fix this one.)
The downsides of this hack are:
(i) obviously, not generic enough. Ideally, one can think of a
directory local-root-delay-loop that contains the scripts that
should be executed during each iteration of the root-delay loop
(ii) The timing of the root delay is completely borked. On my
slow slug the mdadm command takes much longer than the sleep
command.
I'm willing to write a patch that solves this problem, taking into
account the previous two considerations, but some feedback from
the maintainers would be useful before I start coding. If you need
more info, just ask.
This is the output on the serial console during a failing boot:
+Ethernet eth0: MAC address 00:18:39:30:0d:7d
IP: 192.168.0.1/255.255.255.0, Gateway: 192.168.0.1
Default server: 0.0.0.0, DNS server IP: 0.0.0.0
RedBoot(tm) bootstrap and debug environment [ROMRAM]
Red Hat certified release, version 1.92 - built 15:16:07, Feb 3 2004
Platform: IXDP425 Development Platform (XScale)
Copyright (C) 2000, 2001, 2002, Red Hat, Inc.
RAM: 0x00000000-0x02000000, 0x000723a0-0x01ff3000 available
FLASH: 0x50000000 - 0x50800000, 64 blocks of 0x00020000 bytes each.
== Executing boot script in 2.000 seconds - enter ^C to abort
RedBoot> boot;exec 0x01d00000
have eRcOmM
copy kernel code from flash to RAM
copy ramdisk file from flash to RAM
run kernel
Using base address 0x01d00000 and length 0x0001ffe0
APEX Boot Loader 1.4.15 -- Copyright (c) 2004,2005,2006,2007 Marc
Singer
APEX comes with ABSOLUTELY NO WARRANTY. It is free software and you
are welcome to redistribute it under certain circumstances.
For details, refer to the file COPYING in the program source.
apex => mem:0x00200000+0xa78c (42892 bytes)
env => nor:0x7c000+15k (no-write)
Use the command 'help help' to get started.
# copy -s $kernelsrc $bootaddr
# copy -s fis://kernel 0x00008000
1441760 bytes transferred
# copy -s $ramdisksrc $ramdiskaddr
# copy -s fis://ramdisk 0x01000000
6291440 bytes transferred
# wait 10 Type ^C key to cancel autoboot.
Type ^C key to cancel autoboot.
# boot
ATAG_HEADER
ATAG_MEM: start 0x00000000 size 0x02000000
ATAG_CMDLINE: (55 bytes) 'console=ttyS0,115200 rtc-x1205.probe=0,0x6f
noirqdebug'
ATAG_INITRD2: start 0x01000000 size 0x00400000
ATAG_END
Booting kernel at 0x00008000...
Uncompressing
Linux...............................................................................
done, booting the kernel.
Linux version 2.6.18-4-ixp4xx (Debian 2.6.18.dfsg.1-12etch2)
(dannf@debian.org) (gcc version 4.1.2 20061115 (prerelease) (Debian
4.1.1-21)) #1 Sun Apr 22 08:
34:11 UTC 2007
CPU: XScale-IXP42x Family [690541f1] revision 1 (ARMv5TE), cr=0000397f
Machine: Linksys NSLU2
Memory policy: ECC disabled, Data cache writeback
CPU0: D VIVT undefined 5 cache
CPU0: I cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
CPU0: D cache: 32768 bytes, associativity 32, 32 byte lines, 32 sets
Built 1 zonelists. Total pages: 8192
Kernel command line: console=ttyS0,115200 rtc-x1205.probe=0,0x6f
noirqdebug
IRQ lockup detection disabled
PID hash table entries: 256 (order: 8, 1024 bytes)
Console: colour dummy device 80x30
Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
Memory: 32MB = 32MB total
Memory: 25780KB available (1948K code, 496K data, 92K init)
Security Framework v1.0.0 initialized
SELinux: Disabled at boot.
Capability LSM initialized
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
checking if image is initramfs... it is
Freeing initrd memory: 4096K
NET: Registered protocol family 16
IXP4xx: Using 16MiB expansion bus window size
PCI: IXP4xx is host
PCI: IXP4xx Using direct access for memory space
PCI: bus0: Fast back to back transfers disabled
dmabounce: registered device 0000:00:01.0 on pci bus
dmabounce: registered device 0000:00:01.1 on pci bus
dmabounce: registered device 0000:00:01.2 on pci bus
NET: Registered protocol family 2
IP route cache hash table entries: 256 (order: -2, 1024 bytes)
TCP established hash table entries: 1024 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 1024 bind 512)
TCP reno registered
NetWinder Floating Point Emulator V0.97 (double precision)
audit: initializing netlink socket (disabled)
audit(4.380:1): initialized
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
JFFS2 version 2.2. (NAND) (C) 2001-2006 Red Hat, Inc.
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
IXP4xx Watchdog Timer: heartbeat 60 sec
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing
disabled
serial8250.0: ttyS0 at MMIO 0xc8000000 (irq = 15) is a XScale
serial8250.0: ttyS1 at MMIO 0xc8001000 (irq = 13) is a XScale
RAMDISK driver initialized: 16 RAM disks of 8192K size 1024 blocksize
IXP4XX-Flash.0: Found 1 x16 devices at 0x0 in 16-bit bank
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
cfi_cmdset_0001: Erase suspend on write enabled
Searching for RedBoot partition table in IXP4XX-Flash.0 at offset
0x7e0000
6 RedBoot partitions found on MTD device IXP4XX-Flash.0
Creating 6 MTD partitions on "IXP4XX-Flash.0":
0x00000000-0x00040000 : "RedBoot"
NSLU2 MAC: 00:18:39:30:0d:7d
0x00040000-0x00060000 : "SysConf"
0x00060000-0x00080000 : "Loader"
0x00080000-0x001e0000 : "Kernel"
0x001e0000-0x007e0000 : "Ramdisk"
0x007e0000-0x00800000 : "FIS directory"
mice: PS/2 mouse device common for all mice
i2c /dev entries driver
x1205 0-006f: chip found, driver version 1.0.7
x1205 0-006f: rtc core: registered x1205 as rtc0
Registered led device: ready
Registered led device: status
Registered led device: disk-1
Registered led device: disk-2
NET: Registered protocol family 26
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
Time: OSTS clocksource has been installed.
x1205 0-006f: setting the system clock to 2007-06-16 17:57:41
(1182016661)
Freeing init memory: 92K
Loading, please wait...
Begin: Loading essential drivers... ...
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
md: raid1 personality registered for level 1
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
Done.
Begin: Running /scripts/init-premount ...
PCI: enabling device 0000:00:01.0 (0140 -> 0142)
ohci_hcd 0000:00:01.0: OHCI Host Controller
ohci_hcd 0000:00:01.0: new USB bus registered, assigned bus number 1
ohci_hcd 0000:00:01.0: irq 28, io mem 0x48000000
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 3 ports detected
PCI: enabling device 0000:00:01.1 (0140 -> 0142)
ohci_hcd 0000:00:01.1: OHCI Host Controller
ohci_hcd 0000:00:01.1: new USB bus registered, assigned bus number 2
ohci_hcd 0000:00:01.1: irq 27, io mem 0x48001000
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
PCI: enabling device 0000:00:01.2 (0140 -> 0142)
ehci_hcd 0000:00:01.2: EHCI Host Controller
ehci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 3
ehci_hcd 0000:00:01.2: irq 26, io mem 0x48002000
ehci_hcd 0000:00:01.2: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 5 ports detected
usb 3-1: new high speed USB device using ehci_hcd and address 2
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
usb 3-1: configuration #1 chosen from 1 choice
scsi0 : SCSI emulation for USB Mass Storage devices
usb 3-2: new high speed USB device using ehci_hcd and address 3
usb 3-2: configuration #1 chosen from 1 choice
scsi1 : SCSI emulation for USB Mass Storage devices
Loading Intel IXP400 ethernet driver
Failed to load Intel IXP400 ethernet driver
Begin: Loading MD modules ...
Success: loaded module raid1.
Done.
Begin: Assembling MD array /dev/md1 ...
md: md1 stopped.
mdadm: no devices found for /dev/md1
Failure: failed to start /dev/md1
Done.
Done.
Begin: Waiting for root file system... ...
Vendor: ST350084 Model: 1A Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 00
SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
sda: Write Protect is off
sda: assuming drive cache: write through
SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
sda: Write Protect is off
sda: assuming drive cache: write through
sda:<5> Vendor: ST350084 Model: 1A Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 00
SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB)
sdb: Write Protect is off
sdb: assuming drive cache: write through
SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB)
sdb: Write Protect is off
sdb: assuming drive cache: write through
sdb:udevd-event[967]: wait_for_sysfs: waiting for '/sys/devices/
pci0000:00/0000:00:01.2/usb3/3-1/3-1:1.0/host0/target0:0:0/0:0:0:0/
ioerr_cnt' failed
udevd-event[988]: wait_for_sysfs: waiting for '/sys/devices/
pci0000:00/0000:00:01.2/usb3/3-2/3-2:1.0/host1/target1:0:0/1:0:0:0/
ioerr_cnt' failed
sda1 sda2 sda3
sd 0:0:0:0: Attached scsi disk sda
sdb1 sdb2 sdb3
sd 1:0:0:0: Attached scsi disk sdb
Done.
Begin: Running /scripts/local-premount ...
Done.
mount: Mounting /dev/md1 on /root failed: No such device
Begin: Running /scripts/local-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
mount: Mounting /root/dev on /dev/.static/dev failed: No such file or
directory
Done.
mount: Mounting /sys on /root/sys failed: No such file or directory
mount: Mounting /proc on /root/proc failed: No such file or directory
Target filesystem doesn't have /sbin/init
BusyBox v1.1.3 (Debian 1:1.1.3-4) Built-in shell (ash)
Enter 'help' for a list of built-in commands.
/bin/sh: can't access tty; job control turned off
(initramfs)
-- Package-specific info:
-- /proc/cmdline
console=ttyS0,115200 rtc-x1205.probe=0,0x6f noirqdebug
-- /proc/filesystems
cramfs
ext3
-- lsmod
Module Size Used by
ledtrig_heartbeat 3008 0
nfs 267140 0
ipv6 272732 14
nfsd 238572 17
exportfs 5696 1 nfsd
lockd 69912 3 nfs,nfsd
nfs_acl 3936 2 nfs,nfsd
sunrpc 179632 13 nfs,nfsd,lockd,nfs_acl
dm_snapshot 17664 0
dm_mirror 21652 0
dm_mod 60844 2 dm_snapshot,dm_mirror
rtc_dev 7464 0
ixp4xx_mac 19380 0
ixp4xx_qmgr 8460 5 ixp4xx_mac
ixp4xx_npe 14176 2 ixp4xx_mac
evdev 10496 0
firmware_class 10272 1 ixp4xx_npe
ixp4xx_beeper 3520 0
ehci_hcd 33324 0
ohci_hcd 18404 0
usb_storage 79979 6
usbcore 129180 4 ehci_hcd,ohci_hcd,usb_storage
ext3 135624 2
jbd 59112 1 ext3
mbcache 9028 1 ext3
sd_mod 19248 8
scsi_mod 111276 2 usb_storage,sd_mod
raid1 23840 3
md_mod 81492 5 raid1
-- /etc/kernel-img.conf
# Kernel image management overrides
# See kernel-img.conf(5) for details
do_symlinks = yes
relative_links = yes
do_bootloader = no
do_bootfloppy = no
do_initrd = yes
link_in_boot = yes
postinst_hook = flash-kernel
-- /etc/initramfs-tools/initramfs.conf
MODULES=most
BUSYBOX=y
KEYMAP=n
BOOT=local
DEVICE=eth0
NFSROOT=auto
-- System Information:
Debian Release: 4.0
APT prefers stable
APT policy: (700, 'stable'), (650, 'testing'), (600, 'unstable')
Architecture: arm (armv5tel)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.21-1-ixp4xx
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Versions of packages initramfs-tools depends on:
ii busybox 1:1.1.3-4 Tiny utilities for small and embed
ii cpio 2.6-17 GNU cpio -- a program to manage ar
ii klibc-utils 1.4.34-1 small statically-linked utilities
ii module-init-tools 3.3-pre4-2 tools for managing Linux kernel mo
ii udev 0.105-4 /dev/ and hotplug management daemo
initramfs-tools recommends no packages.
-- no debconf information
Reply to: