| 2014-06-09 06:35:30 |
Joshua Harlow |
bug |
|
|
added bug |
| 2014-06-09 06:35:59 |
Joshua Harlow |
description |
If a program is terminated via ctrl-c or other signal and it has acquired locks using the posix_ipc sempahore those sempahores are never released (unlike the filelock which does release automatically on program termination). This can be easily seen by trying the following in a shell (see below). You can come back after minutes and hours and try to acquire the abandoned semaphore and still be unable to (its still busy), perhaps there is a automatic timeout that needs to be provided? I can imagine that if people do upgrades of a service using this lock and the program is currently locked that they would have to have some kind of sempahore cleaning script to even restart that application (which seems bad).
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16123
>>> lockutils.InterProcessLock("testing")
<openstack.common.lockutils._PosixLock object at 0x7f8bb4a8e210>
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire()
<openstack.common.lockutils._PosixLock object at 0x7f8bb30e7ad0>
>>> exit()
(.dev)josh@lappy:~/Dev/oslo-incubator$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16128
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire(timeout=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "openstack/common/lockutils.py", line 168, in acquire
self.semaphore.acquire(timeout)
posix_ipc.BusyError: Semaphore is busy
.... |
If a program is terminated via ctrl-c or other signal and it has acquired locks using the posix_ipc sempahore those sempahores are never released (unlike the filelock which does release automatically on program termination). This can be easily seen by trying the following in a shell (see below). You can come back after minutes and hours and try to acquire the abandoned semaphore and still be unable to (it's still apparently busy), perhaps there is a automatic timeout that needs to be provided? I can imagine that if people do upgrades of a service using this lock and the program is currently locked that they would have to have some kind of sempahore cleaning script to even restart that application (which seems bad).
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16123
>>> lockutils.InterProcessLock("testing")
<openstack.common.lockutils._PosixLock object at 0x7f8bb4a8e210>
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire()
<openstack.common.lockutils._PosixLock object at 0x7f8bb30e7ad0>
>>> exit()
(.dev)josh@lappy:~/Dev/oslo-incubator$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16128
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire(timeout=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "openstack/common/lockutils.py", line 168, in acquire
self.semaphore.acquire(timeout)
posix_ipc.BusyError: Semaphore is busy
.... |
|
| 2014-06-09 06:58:26 |
Joshua Harlow |
description |
If a program is terminated via ctrl-c or other signal and it has acquired locks using the posix_ipc sempahore those sempahores are never released (unlike the filelock which does release automatically on program termination). This can be easily seen by trying the following in a shell (see below). You can come back after minutes and hours and try to acquire the abandoned semaphore and still be unable to (it's still apparently busy), perhaps there is a automatic timeout that needs to be provided? I can imagine that if people do upgrades of a service using this lock and the program is currently locked that they would have to have some kind of sempahore cleaning script to even restart that application (which seems bad).
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16123
>>> lockutils.InterProcessLock("testing")
<openstack.common.lockutils._PosixLock object at 0x7f8bb4a8e210>
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire()
<openstack.common.lockutils._PosixLock object at 0x7f8bb30e7ad0>
>>> exit()
(.dev)josh@lappy:~/Dev/oslo-incubator$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16128
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire(timeout=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "openstack/common/lockutils.py", line 168, in acquire
self.semaphore.acquire(timeout)
posix_ipc.BusyError: Semaphore is busy
.... |
If a program is terminated via ctrl-c or other signal and it has acquired locks using the posix_ipc sempahore those sempahores are never released (unlike the filelock which does release automatically on program termination). This can be easily seen by trying the following in a shell (see below). You can come back after minutes and hours and try to acquire the abandoned semaphore and still be unable to (it's still apparently busy), perhaps there is a automatic timeout that needs to be provided? I can imagine that if people do upgrades of a service (using for example `service nova-api stop`) using this lock and the program is currently locked that they would have to have some kind of sempahore cleaning script to even restart that application (which seems bad).
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16123
>>> lockutils.InterProcessLock("testing")
<openstack.common.lockutils._PosixLock object at 0x7f8bb4a8e210>
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire()
<openstack.common.lockutils._PosixLock object at 0x7f8bb30e7ad0>
>>> exit()
(.dev)josh@lappy:~/Dev/oslo-incubator$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from openstack.common import lockutils
>>> import os
>>> os.getpid()
16128
>>> a = lockutils.InterProcessLock("testing")
>>> a.acquire(timeout=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "openstack/common/lockutils.py", line 168, in acquire
self.semaphore.acquire(timeout)
posix_ipc.BusyError: Semaphore is busy
.... |
|
| 2014-06-09 15:01:18 |
Ben Nemec |
oslo: status |
New |
Triaged |
|
| 2014-06-09 15:01:21 |
Ben Nemec |
oslo: importance |
Undecided |
High |
|
| 2014-07-23 10:58:11 |
OpenStack Infra |
oslo: status |
Triaged |
In Progress |
|
| 2014-07-23 10:58:11 |
OpenStack Infra |
oslo: assignee |
|
Yuriy Taraday (yorik-sar) |
|
| 2014-08-13 18:02:38 |
Ben Nemec |
oslo: importance |
High |
Critical |
|
| 2014-08-21 23:07:39 |
OpenStack Infra |
oslo: assignee |
Yuriy Taraday (yorik-sar) |
Ben Nemec (bnemec) |
|
| 2014-09-01 21:44:12 |
Doug Hellmann |
bug task added |
|
oslo.concurrency |
|
| 2014-09-01 21:44:40 |
Doug Hellmann |
oslo.concurrency: importance |
Undecided |
Critical |
|
| 2014-09-01 21:44:45 |
Doug Hellmann |
oslo.concurrency: status |
New |
In Progress |
|
| 2014-09-01 21:44:53 |
Doug Hellmann |
oslo.concurrency: assignee |
|
Ben Nemec (bnemec) |
|
| 2014-09-02 16:16:57 |
Ben Nemec |
oslo.concurrency: status |
In Progress |
Fix Committed |
|
| 2014-09-02 16:47:06 |
Doug Hellmann |
oslo.concurrency: milestone |
|
juno-rc1 |
|
| 2014-09-04 16:59:30 |
Ben Nemec |
oslo-incubator: milestone |
|
juno-rc1 |
|
| 2014-09-05 13:03:39 |
Doug Hellmann |
oslo-incubator: status |
In Progress |
Fix Released |
|
| 2014-09-05 13:09:31 |
Thierry Carrez |
oslo.concurrency: milestone |
|
next-juno |
|
| 2014-09-09 13:25:11 |
Thierry Carrez |
oslo-incubator: milestone |
juno-rc1 |
juno-3 |
|
| 2014-09-13 13:41:34 |
Doug Hellmann |
oslo.concurrency: milestone |
next-juno |
juno-rc1 |
|
| 2014-09-23 18:05:41 |
Doug Hellmann |
oslo.concurrency: milestone |
next-juno |
next-kilo |
|
| 2014-10-24 17:08:36 |
Doug Hellmann |
oslo.concurrency: status |
Fix Committed |
Fix Released |
|
