Comment 9 for bug 1560710

I thought about such a "master/slave" implementation in watershed for a while, and this is a bit tricky. The "slaves" need to communicate their desire to run the command to the master, and both the master and slaves need to use locking to ensure that only one instance is the master and runs the command. However, the acts of "(try to) acquire the lock" and "refresh the desire to run the command" need to happen atomically, otherwise race conditions occur. E. g. in a slave instance some time might pass between the failure to acquire the lock and increasing/refreshing the stamp, and if the master finishes in between that last request is lost.

The standard tool for such a race free and atomic counter which is simultaneously a lock is a semaphore. However, both named and unnamed POSIX semaphores rely on /dev/shm/, and we cannot rely on that in udev rules (no /dev/shm in the initrd). We can use semaphores if we can assert that the udev rule is not crucial during early boot. I *think* it should be okay as it gets re-exercised after pivoting and everything gets mounted.

I wanted to compare the hotplug behaviour under Debian and Ubuntu. These commands can be run in a minimal VM:

   apt-get install -y lvm2
   reboot # lvm daemons don't seem to start right after installation; get a clean slate

   modprobe scsi_debug
   pvcreate /dev/sda
   vgcreate testvg /dev/sda
   lvcreate -L 4MB testvg

Now we have one PV, VG, and LV each, and an usable block device:

   lrwxrwxrwx 1 root root 7 Mar 30 08:01 /dev/testvg/lvol0 -> ../dm-0

Let's hot-remove the device. This does not automatically clean up the mapped device, so do this manually:

   echo 1 > /sys/block/sda/device/delete
   dmsetup remove /dev/testvg/lvol0

Now hotplug back the block device:

   echo '0 0 0' > /sys/class/scsi_host/host2/scan

Under *both* Debian and Ubuntu this correctly brings up the PV, VG, and LV, and /dev/testvg/lvol0 exists again. I can even remove our udev rule 85-lvm2.rules, update the initrd, reboot, and run the above test.

Thus it seems our Ubuntu specific udev rule is entirely obsolete. Indeed these days /lib/udev/rules.d/69-lvm-metad.rules (which calls pvscan --cache --activate) and lvmetad seem to be responsible for that, see /usr/share/doc/lvm2/udev_assembly.txt. So it seems we are now just doing extra work for no benefit.

I also noticed this in our Ubuntu delta description:

        - do not install activation systemd generator for lvm2, since udev starts LVM.

The activation generator is relevant if the admin disabled lvmetad, then the generator builds up the VGs at boot time. It's a no-op if lvmetad is enabled. We should put that back to match the current documentation and reduce our delta.