Comment 16 for bug 802626

It seems pretty clear what is happening here: udev's 85-lvm rule fires off vgchange (through watershed); which activates lv, then sends udev a DM_COOKIE with the address of a semaphore, then waits for that semaphore to drop to zero; udev fires off 55-dm, which calls dmsetup udevcomplete ${DM_COOKIE}, which clears the semaphore; udev sometimes exits before calling dmsetup udevcomplete, so vgchange never exits, but initramfs' udev waits for it to exit (until finally being killed after timeout).

Daemonizing watershed worked because it allowed udev to exit while vgchange continues to wait. I *believe* the rootfs udev then always continued to call dmsetup udevcomplete to let vgchange exit, though it is possible that there were cases of vgchange hanging which I simply never noticed.

The other possible solution would be to have vgchange not wait for the semaphore. Note that theoretically this could cause errors to be hidden (when vgchange should have hung but didn't), except that vgchange sticking around is easy to not notice if nothing else goes wrong. However, I've tried having the vgchange -a y in initramfs use --ignoremonitoring, and several boots hung completely very early on, while others succeeded. It's possible I did something else wrong.

A third solution *could* be for udev to not clear queues and workers so long as there are non-idle workers. The code to do so should be pretty easy (I have a sample patch but haven't tested it)