Re: jessie won't install/boot on a Dell Poweredge R815

[Jeffrey Mark Siskind <qobi@purdue.edu>]

   > and attempted
   > 
   >    mdadm /dev/md0 --add /dev/sda1
   >    mdadm /dev/md0 --add /dev/sdb1
   >    mdadm /dev/md0 --add /dev/sdc1
   >    mdadm /dev/md0 --add /dev/sdd1
   >    mdadm /dev/md0 --add /dev/sde1
   >    mdadm /dev/md0 --add /dev/sdf1
   > 
   > but these all failed.

   This is the wrong command; it should be mdadm --assemble /dev/md0
   /dev/sd[abcdef]1;

   And that should only be done if the md0 device doesn't show up in the
   initrd when you cat /proc/mdstat.

   What's happened is that the raid1 device now has 12 drives instead of 6,
   which basically isn't going to work at all.

You can see from the transcript that md0 is there and has only 6 drives. Just
that 5 of the six are marked as failed. And you can see that it refused to do
the mdadm --add.

   http://upplysingaoflun.ecn.purdue.edu/~qobi/upgrade-jessie2.script

   root@verstand:~# cat /proc/mdstat
   Personalities : [raid1] [raid6] [raid5] [raid4] 
   md1 : active raid5 sda2[0] sdf2[5] sdd2[4] sdc2[3] sde2[2] sdb2[1]
	 1953118720 blocks super 1.2 level 5, 512k chunk, algorithm 2 [6/6] [UUUUUU]

   md0 : active raid1 sda1[6](F) sdd1[8](F) sdb1[7](F) sde1[9](F) sdc1[10] sdf1[11](F)
	 39157688 blocks super 1.2 [6/1] [__U___]

   unused devices: <none>
   root@verstand:~# mdadm --add/dev/md0 --add /defv/sda1
   mdadm: Cannot open /dev/sda1: Device or resource busy
   root@verstand:~# mdadm /dev/md0 --add /dev/sda1b1
   mdadm: Cannot open /dev/sdb1: Device or resource busy
   root@verstand:~# mdadm /dev/md0 --add /dev/sdb1d1
   mdadm: Cannot open /dev/sdd1: Device or resource busy
   root@verstand:~# mdadm /dev/md0 --add /dev/sdd1e1
   mdadm: Cannot open /dev/sde1: Device or resource busy
   root@verstand:~# mdadm /dev/md0 --add /dev/sde1f1
   mdadm: Cannot open /dev/sdf1: Device or resource busy
   root@verstand:~# mdadm /dev/md0 --add /dev/sdf11c1
   mdadm: Cannot open /dev/sdc1: Device or resource busy

   You should be able to just directly reinstall jessie on this machine;

In earlier posts I explained how this fails. If I do a direct install from
USB, I observe two kinds of errors.

 1. Sometimes, but not every time, (it is nondeterministic) after the first 3
    questions, the installer complains that it can't find the ISO.
 2. Whenever it does find the ISO, the install progresses without error all
    the way to the grub install and then complains that it can't install grub.
    I've tried several different things. Sometimes, I just answer sda to the
    grub install question. (Actually sometimes sdb, because if I plug the USB
    into the front port, the USB gets sdg and the drives get sd[a-f] but if I
    plug the USB into the back port, the USB gets sda and the drives get
    sd[b-g].) But this always fails. Sometimes, I go into ctrl-alt-f2 and do
      chroot target
      grub-install /dev/sda
      ...
      grub-install /dev/sdf
      (or b-g as appropriate)
    but this also fails. At that point, I have no way to install grub. (If I
    abort the install, the machine is unbootable.) Whenever I'm in this state
    I do cat /proc/mdstat and it shows that some components of md0 are failed
    or missing. Some are present. This is nondeterministic. Which components
    are present and which are missing changes each time I attempt this. If I
    attempt to do mdadm --add I get errors. If I reinstall fresh wheezy from
    USB and then in wheezy do mdadm --add, it works and rebuilds the
    array. When it is done it has all 6 components. And then I immediately do
    a fresh install of jessie from USB and the same problem happens.

   I'd also zero out the superblocks on the devices in /dev/md0,

What command?

    Jeff (http://engineering.purdue.edu/~qobi)