Comment 5 for bug 1432387

The lock files are left around because the fastener library used for locking closes files after releasing them, as opposed to deleting them (https://github.com/harlowja/fasteners/blob/master/fasteners/process_lock.py#L169). This makes sense for the current implementation, as other processes waiting on the lock will already have the same file open, so deleting the file would cause further complications.

Some options that jump to mind for fixing are:
    1. Modify fasteners/process_lock.trylock to check that the file exists each time, and if it doesn't will re-open it (like Michal suggested). Downside would be performance hit from calling os.path.isfile every time a process tries regrabbing a lock
    2. Have some periodic cleanup that deletes locks which haven't been used in a certain amount of time. We'd need to be guessing at some max amount of time that a locked operation can take. Could be useful for deleting locks related to volumes which have already been deleted.

FWIW this issue is not only in Cinder - it's in other projects as well (look in /opt/stack/data/nova/instances/locks). The only place I've found where we actually clean up these locks currently is in nova/virt/libvirt/imagecache where we call lockutils.remove_external_lock_file