On any lvm2 version without lvmetad running, I can get cinder-volume to hang on issuing lvm related commands after deleting snapshots (that are non-thin provisioned LVM snapshots with clear_volume set to zero).
the issue is that lvm locks up due to trying to access suspended device mapper entries, and at some point cinder-volume does a lvm related command and hangs on that. a setting of ignore_suspended_devices = 1 in lvm.conf helps with that, as lvremove hangs on scanning the device state (which it
needs to do because it doesn't have current information available via lvmetad).
I can use this script to trigger the issue:
=== cut hang.sh ===
enable_fix=1
#enable_fix=0
vg=cinder-volumes
v=testvol.$$
lvcreate --name $v $vg -L 1g
sleep 2
lvcreate --name snap-$v --snapshot $vg/$v -L 1g
vgp=/dev/mapper/${vg/-/--}-snap--${v/-/--}
sleep 2
( sleep 10 < $vgp-cow ) &
test "$enable_fix" -eq "1" && lvchange -y -an $vg/snap-$v
lvremove -f $vg/snap-$v
sleep 1
lvremove -f $vg/$v
=== cut hang.sh ===
vg needs to be set to a lvm VG that exists and can take a few gig of space. whenever enable_fix is set to 0, lvremove -f ends with :
Unable to deactivate open cinder--volumes-snap--testvol.27700-cow (252:5)
Failed to resume snap-testvol.27700.
libdevmapper exiting with 1 device(s) still suspended.
this is because the sleep command before keeps a fd open on the -cow. The script then also never finishes and any other lvm command hangs as well.
apparently in real-life this is either udev or the dd command still having the fd open for some reason I have not yet understood.
The deactivation before removing seems to help.
Fix proposed to branch: master
Review: https:/