[->UUIDudev] bad mdadm.conf on jaunty upgrade

[Paul Natsuo Kishimoto]

Binary package hint: mdadm

This occurred on an upgrade from 8.10 to 9.04. I have a md raid1 with two devices (sda1, sdb1) and two spares (sdc1, sdd1) named /dev/md0, which the system boots from. On reboot after the upgrade, the system dropped into busybox with "ALERT! /dev/md0 does not exist" etc.

The previous kernel (2.6.27-11) would still boot without issue. I looked around and discovered that my /etc/mdadm/mdadm.conf looked like this:
=====
DEVICE partitions
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=490f3db6:cea49fa3:345b7477:817747f3
    spares=2
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=0726d925:6e4e7a99:bcdab21b:f117ebf9
    spares=2
ARRAY /dev/md2 level=raid5 num-devices=4 UUID=9d07bdb8:ede8880b:54422443:48105f18
DEVICE partitions
ARRAY /dev/md0 level=raid1 num-devices=2 spares=2 UUID=490f3db6:cea49fa3:345b7477:817747f3
ARRAY /dev/md1 level=raid1 num-devices=2 spares=2 UUID=0726d925:6e4e7a99:bcdab21b:f117ebf9
ARRAY /dev/md2 level=raid5 num-devices=4 UUID=9d07bdb8:ede8880b:54422443:48105f18
MAILADDR root
=====

...i.e. two DEVICE lines, and the 'spares' parameter for md0 and md1 is on a new line, only for the first instance. From "man mdadm.conf" I can't see clearly whether or not this is wrong. I changed the file to:

=====
DEVICE partitions
ARRAY /dev/md0 level=raid1 num-devices=2 spares=2 UUID=490f3db6:cea49fa3:345b7477:817747f3
ARRAY /dev/md1 level=raid1 num-devices=2 spares=2 UUID=0726d925:6e4e7a99:bcdab21b:f117ebf9
ARRAY /dev/md2 level=raid5 num-devices=4 UUID=9d07bdb8:ede8880b:54422443:48105f18
MAILADDR root
=====

...and ran:

# update-initramfs -k 2.6.28-11-server -c

...after which the system boots properly with the new kernel.

I don't know much about boot, but it seems the mdadm.conf file is included in the initramfs (I could see it in /etc/mdadm/mdadm.conf in busybox). I am not sure if
A) the mdadm.conf included in the 2.6.27-11.server initramfs has better/proper syntax (how do I check?), or
B) the mdadm.conf is the same, but the 2.6.27-11-server kernel was more tolerant.

[Paul Natsuo Kishimoto (khaeru)]

I found: http://kernel-handbook.alioth.debian.org/ch-initramfs.html#s-initramfs-exam

=====
khaeru@khaeru-storage:~$ mkdir tmp
khaeru@khaeru-storage:~$ cd tmp
khaeru@khaeru-storage:~/tmp$ zcat /boot/initrd.img-2.6.27-11-server | cpio -i
41267 blocks
khaeru@khaeru-storage:~/tmp$ cat etc/mdadm/mdadm.conf
DEVICE partitions
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=490f3db6:cea49fa3:345b7477:817747f3
   spares=2
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=0726d925:6e4e7a99:bcdab21b:f117ebf9
   spares=2
ARRAY /dev/md2 level=raid5 num-devices=4 UUID=9d07bdb8:ede8880b:54422443:48105f18
DEVICE partitions
ARRAY /dev/md0 level=raid1 num-devices=2 spares=2 UUID=490f3db6:cea49fa3:345b7477:817747f3
ARRAY /dev/md1 level=raid1 num-devices=2 spares=2 UUID=0726d925:6e4e7a99:bcdab21b:f117ebf9
ARRAY /dev/md2 level=raid5 num-devices=4 UUID=9d07bdb8:ede8880b:54422443:48105f18
MAILADDR root
=====

...so it seems the 2.6.27-11-server kernel was able to boot with this odd mdadm.conf, but the 2.6.28-11-server kernel was not.

[davee (davee-sungate)]

> I don't know much about boot, but it seems the mdadm.conf file is included in the initramfs

Yes, it seems it is: using 'stat' to look at access times on mdadm.conf shows that it is at least _read_ when running update-initramfs

Just found this bug when I had a similar non-booting issue following upgrade from 8.10 to 9.04. For general info, here is what I did to get a successful startup, after being dumped into 'initramfs' at boot:

initramfs> mknod /dev/md0 b 9 0
initramfs> mdadm --assemble /dev/md0
initramfs> return

at which point the kernel was able to boot. After starting up, modifying mdadm.conf as described above, then running

update-initramfs -u

was sufficient to make it reboot cleanly.

[Urop (urop)]

This bug affected me too and I can confirm the workarounds in this thread: See http://ubuntuforums.org/showthread.php?p=7410040

It's find it very annoying that I had to experience a failed upgrade, when this bug had been reported and confirmed for over a month. It would be good if a warning about this bug for people running raid could be placed immediately on the release notes of Jaunty, with links to the workarounds presented here. At least it could waste lots of other people time and effort searching for the cause and a solution while the bug is being fixed.

[ceg (ceg)]

UUID-based raid assembly that makes mdadm.conf maintenance obsolete fixes this.