Comment 6 for bug 1432387

Revision history for this message
Joshua Harlow (harlowja) wrote :

Perhaps use something like:

    import os
    import fasteners

    class LockBucket(object):
        def __init__(self, lock_path, amount,
                     lock_prefix='', lock_postfix='.lock'):
            if amount <= 0:
                raise ValueError("At least one lock must be created")
            self.lock_path = lock_path
            self.locks = []
            for i in range(0, amount):
                i_lock_path = os.path.join(lock_path,
                                           "%s%s%s" % (lock_prefix, i + 1,
                                                       lock_postfix))
                self.locks.append(fasteners.InterProcessLock(i_lock_path))

        def __getitem__(self, item):
            m = hash(item)
            r = m % len(self.locks)
            return self.locks[r]

    lock_dir = "/tmp/cinder-locks"
    try:
        os.mkdir(lock_dir)
    except OSError:
        pass
    locks = LockBucket(lock_dir, 10)
    print locks['b']

Creating random arbitrary locks per instance, or per resource is not going to end well for anyone, since as u discovered deleting locks (actually deleting the file handle associated with that lock) across process is not easily made safe (because other processes trying to wait on the lock still refer to the old file handle); The above though creates a stable bucket of known amount of locks that will forever exist and can be well defined to only be X amount (what X is needs to be decided, but 100 seems reasonable); then there will never be more than 100 locks, but there will also never been more than 100 current requests to those locks, so 100 should probably == the max number of greenthreads that can be active per process (so that a greenthread is never unable to get a lock/starved).

Do note that http://semanchuk.com/philip/sysv_ipc/ and others also have this same problem...

'''
remove()

    Removes (deletes) the semaphore from the system.

    As far as I can tell, the effect of deleting a semaphore that other processes are still using is OS-dependent. Check your system's man pages for semctl(IPC_RMID).
'''

http://semanchuk.com/philip/posix_ipc/ is I think closer to what u want where that library has the following:

'''
unlink()

    Destroys the semaphore, with a caveat. If any processes have the semaphore open when unlink is called, the call to unlink returns immediately but destruction of the semaphore is postponed until all processes have closed the semaphore.

    Note, however, that once a semaphore has been unlinked, calls to open() with the same name should refer to a new semaphore. Sound confusing? It is, and you'd probably be wise structure your code so as to avoid this situation.
'''

I forget which one (sysv_ipc or posix_ipc) supports releasing automatically on unintentional process death (kill -9) but only one of the above does (file based locks do this automatically, because a process killed releases its held files)...