Race condition in handling of udev generated symlinks

[Joseph Glanville]

In several components in both Cinder and Nova udev provided symlinks are expected to appear immediately.
This is not the case as udev rules run async of device plugging.
Volume drivers in Cinder and the libvirt driver in Nova seem to be the primary culprits.

To solve this we should wait for udevd to finish processing events, this can be done with udevadm by calling settle. This is done by simply calling "udevadm settle". Additionally it can be called with a timeout (probably a good idea).
Excerpt from udevadm settle --help:

Usage: udevadm settle OPTIONS
  --timeout=<seconds> maximum time to wait for events
  --seq-start=<seqnum> first seqnum to wait for
  --seq-end=<seqnum> last seqnum to wait for
  --exit-if-exists=<file> stop waiting if file exists
  --quiet do not print list after timeout
  --help

For more intelligent use we could wrap this in a function that can use the --exit-if-exists behavior.

This will ensure the symlink actually exists before we try use it.

[Vish Ishaya (vishvananda)]

good suggestion. This might allow us to remove some of the retrys that we use.

[John Griffith (john-griffith)]

We should investigate this a bit more, I did a little looking (because I'm worried that this isn't making any progress) and it seems that there were some very good reasons why udev settle was disabled for LVM a while back. There appears to be some significant issues around race-conditions caused by udev-settle, though to be quite honest I don't fully understand all of these things yet.

I should also add sheepishly, I'm not even sure what versions of Ubuntu for example no longer pull this in by default, it seems that RedHat still does (again, I could be wrong, but first glance that appears to be the case).

_med: any info on this based on the work you've been looking in to so far?

[David Medberry (med)]

john-griffith: No, no update.

[Sean Dague (sdague)]

This is unclear that it's a real bug, as it was disabled in the past. Definitely not in a triaged state.