[->UUIDudev] MD/LVM boot broken

[Bryan Larsen]

Binary package hint: initramfs-tools

This is probably very closely related to bug 314879, but since that is marked as fixed, I'm opening a new one.

I upgraded from Intrepid to Jaunty Alpha5. After the reboot, I got dropped into busybox.

To fix it, I needed the following *4* commands at the initramfs prompt:

mdadm --assemble --scan
lvm vgscan
lvm vgchange -ay
exit

My package versions are:

lvm2 - 2.02.39-0ubuntu6
initramfs-tools - 0.92bubuntu20
mdadm - 2.6.7.1-1ubuntu6
kernel 2.6.28-7-generic

[Bryan Larsen (bryan-larsen)]

and watershed version 3

[Yannis Tsop (ogiannhs)]

Same problem here. Cannot boot, even the mentioned solution does not work

[Bryan Larsen (bryan-larsen)]

The problem has just gotten severely worse. I did a apt-get dist-upgrade today, and am no longer able to boot my machine.

At the (first) busybox prompt I type the above commands.

The computer then pauses for a few minutes and then returns with many messages of the form:

"/sys/devices/pci0000:00/0000:00:1c.4/0000:03:00.1/host6/target6:0:0/6:0;);)/block/sdd/sd1 (10437)"

The only thing that changes is the last number. It decreases, but not necessarily by one.

Following is:

"Gave up waiting for root device. Common problems:
  - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
  - Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/mapper/root_vg-root_v does not exist. Dropping to a shell!"

But if I type

mkdir /mnt
mount /dev/mapper/root_vg-root-v /mnt

it succeeds.

Booting from CD can also give me access to the volume. fsck tells me that it's clean.

HELP! I'm dead in the water.

[Bryan Larsen (bryan-larsen)]

I managed to boot with a 2.6.24 kernel -- both 2.6.27 and 2.6.28 exhibited the same problem. (Well, the 2.6.27 wasn't exactly the same).

However, in 2.6.24, my udevd eats up 80% of my CPU and makes my hard disk churn!

Looking through my ps -ax, it may be the "/lib/udev/watershed sh -c /sbin/lvm vgscan; /sbin/lvm vgchange -a y" that is causing udevd to go crazy. If I kill the "lvm" process, another one gets immediately restarted.

My version numbers are the same as above except for:

initramfs-tools: 0.92bubuntu21
mdadm: 2.6.7.1-1ubuntu7

kernel that booted: 2.6.24-21-generic
kernel that didn't boot: 2.6.28-8.24

[Bryan Larsen (bryan-larsen)]

Note that I have two md's. The one I don't care about get's started automatically, the mdadm --assemble --scan is only required to start the md that has my root partition.

[Scott James Remnant (Canonical) (canonical-scott)]

On Mon, 2009-02-23 at 21:53 +0000, Bryan Larsen wrote:

> I managed to boot with a 2.6.24 kernel -- both 2.6.27 and 2.6.28
> exhibited the same problem. (Well, the 2.6.27 wasn't exactly the same).
>
> However, in 2.6.24, my udevd eats up 80% of my CPU and makes my hard
> disk churn!
>
This is bug #332270, for which patches have been uploaded this evening.

Scott
--
Scott James Remnant
<email address hidden>

[Bryan Larsen (bryan-larsen)]

Thanks Scott, but udev 138-2 does not fix the second problem, but it does make it different. Since this is quite probably two different bugs, I've opened another report with the details at bug 333614.

[Scott James Remnant (Canonical) (canonical-scott)]

On Tue, 2009-02-24 at 00:48 +0000, Bryan Larsen wrote:

> Thanks Scott, but udev 138-2 does not fix the second problem, but it
> does make it different. Since this is quite probably two different
> bugs, I've opened another report with the details at bug 333614.
>
Did you also upgrade lvm2?

Scott
--
Scott James Remnant
<email address hidden>

[Bryan Larsen (bryan-larsen)]

Thanks for the pointer that there's a new lvm2. Unfortunately, 2.02.39-0ubuntu8 does not affect my problem in any way.

[Bryan Larsen (bryan-larsen)]

This is not a duplicate of bug 332270 -- the fix for 332270 just fixed comment #3 -- the original problem is still extant.

[Bryan Larsen (bryan-larsen)]

This is not a duplicate of bug 332270! Would somebody please reopen this bug!

I finally fixed this bug. My upgrade to Jaunty produced the attached bad mdadm.conf. I ran "dpkg-reconfigure mdadm" and it fixed the bug, producing the attached correct mdadm.conf.

[Bryan Larsen (bryan-larsen)]

This is not a duplicate. The bug appears to be in mdadm.

[Daniel Hahler (blueyed)]

Diff between bad and good mdadm.conf from the comments above:

--- mdadm.conf.backup 2009-03-14 15:39:56.000000000 +0100
+++ mdadm.conf 2009-03-14 15:41:53.000000000 +0100
@@ -1,5 +1,3 @@
 DEVICE partitions
-ARRAY /dev/md0 level=raid5 num-devices=3 UUID=5ab28c4c:2cc90775:9b315729:d7b43017
-DEVICE partitions
-ARRAY /dev/md0 level=raid5 num-devices=3 UUID=5ab28c4c:2cc90775:9b315729:d7b43017
+ARRAY /dev/md0 level=raid5 num-devices=3 metadata=00.90 UUID=5ab28c4c:2cc90775:9b315729:d7b43017
 MAILADDR <email address hidden>

(Please note that you re-open/un-duplicate bugs yourself, doing so now - and assigning to mdadm accordingly)

[ceg (ceg)]

This should be no issue with UUID-based raid assembly that makes mdadm.conf ARRAY definition maintanance obsolete.

[ceg (ceg)]

UUID-based raid assembly: Bug #158918