Package: apt
Version: 1.0.9.8.4
During initial installation of Debian 8, I used mdadm to create 2 RAID arrays as follows:
/dev/md0 - RAID 1 - 4 drives - contains /boot only
/dev/md1 - RAID 5 - 3 drives + 1 spare - contains /
Both are needed to successfully boot the system.
During installation I name the machine HOSTA. A few days later, I decided to change the machine's host name to HOSTB. After a few weeks of successful operation of HOSTB, I ran apt-update followed by apt-upgrade. Among the updates were linux-image updates, which caused initramfs to be updated. After the updates, rebooting the system landed me here:
=====
Gave up waiting for root device. Common problems:
— Boot args (cat /proc/cmdline)
— Check rootdelay= (did the system wait long enough?)
— Check root= (did the system wait for the right device?)
— Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/disk/by-uuid/054a6dcc-d280-4aff-9977 does not exist.
Dropping to a shell!
BusyBox v.1.22.1 (Ubuntu 1:1.22.0-9deb8u1) built-in shell (ash)
Enter 'help' for list of built-in commands.
/bin/sh: can't access tty: job control turned off
(initramfs)
=====
In the initramfs shell, /etc/mdadm/mdamd.conf contained the new hostname, HOSTB instead of HOSTA:
=====
# definitions of existing MD arrays
ARRAY /dev/md/0 metadata=1.2 UUID=413a2cf3:ef0784ac:1e135c73:7a1723c6 name=HOSTB:0
ARRAY /dev/md/1 metadata=1.2 UUID=ae1ab583:e5e30fb9:e2c807a2:b5e95f42 name=HOSTB:1
spares=1
=====
Editing mdadm.conf to use HOSTA in place of HOSTB allowed me to assemble my arrays and continue booting the system:
=====
(initramfs) mdadm --assemble --scan
mdadm: /dev/md/0 has been started with 4 drives.
mdadm: /dev/md/1 has been started with 3 drives and 1 spare.
(initramfs) exit
=====
After a successful boot, mdadm -D confirms that the original host name, HOSTA is part of the array configuraion:
=====
mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Thu Dec 29 09:02:43 2016
Raid Level : raid5
Array Size : 2130618368 (2031.92 GiB 2181.75 GB)
Used Dev Size : 1065309184 (1015.96 GiB 1090.88 GB)
Raid Devices : 3
Total Devices : 4
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Thu Feb 9 17:25:16 2017
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
Name : HOSTA:1 # <--- here is the original hostname given to the machine during installation
UUID : ae1ab583:e5e30fb9:e2c807a2:b5e95f42
Events : 4831
Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3
1 8 19 1 active sync /dev/sdb3
2 8 35 2 active sync /dev/sdc3
3 8 51 - spare /dev/sdd3
=====
In this case, the updates to initramfs should NOT have changed the hostname in the mdadm.conf file. Doing so prevented the arrays from being assembled and the system from booting.
To get around the issue until the next apt-upgrade installs something causing initramfs to be updated again, on the up and running system, edit /etc/mdadm/mdadm.conf to use the original hostname and then run update-initramfs to install the change. Once done, the system reboots normally again.
Once the issue is known, it is also possible to run apt-upgrade to completion, then before rebooting, edit mdamd.conf and manually execute update-initramfs. This will prevent an untimely visit to the BusyBox screen, and in the case of remote upgrades being done, will prevent the need to physically visit the server in order to complete the boot process.
While this may be considered a problem with mdadm - why is it using a potentially volatile parameter like hostname in its permanent configuration? - until that is addressed, apt-upgrade should not modify the hostname used in the mdadm.conf file.