| 2016-07-27 09:06:53 |
Roman Podoliaka |
bug |
|
|
added bug |
| 2016-07-27 09:07:26 |
Roman Podoliaka |
tags |
|
mos-nova |
|
| 2016-07-27 09:07:30 |
Roman Podoliaka |
tags |
mos-nova |
area-nova |
|
| 2016-07-27 09:07:35 |
Roman Podoliaka |
mos: assignee |
|
Roman Podoliaka (rpodolyaka) |
|
| 2016-07-27 09:07:39 |
Roman Podoliaka |
mos: milestone |
|
10.0 |
|
| 2016-07-27 09:07:50 |
Roman Podoliaka |
mos: status |
New |
Confirmed |
|
| 2016-07-27 09:07:59 |
Roman Podoliaka |
mos: importance |
Undecided |
High |
|
| 2016-07-27 09:08:05 |
Roman Podoliaka |
nominated for series |
|
mos/9.x |
|
| 2016-07-27 09:08:05 |
Roman Podoliaka |
bug task added |
|
mos/9.x |
|
| 2016-07-27 09:08:12 |
Roman Podoliaka |
nominated for series |
|
mos/10.0.x |
|
| 2016-07-27 09:08:12 |
Roman Podoliaka |
bug task added |
|
mos/10.0.x |
|
| 2016-07-27 09:08:17 |
Roman Podoliaka |
mos/10.0.x: status |
New |
Confirmed |
|
| 2016-07-27 09:08:19 |
Roman Podoliaka |
mos/10.0.x: importance |
Undecided |
High |
|
| 2016-07-27 09:08:20 |
Roman Podoliaka |
mos/10.0.x: assignee |
|
Roman Podoliaka (rpodolyaka) |
|
| 2016-07-27 09:08:23 |
Roman Podoliaka |
mos/10.0.x: milestone |
|
10.0 |
|
| 2016-07-27 09:08:25 |
Roman Podoliaka |
mos/9.x: milestone |
10.0 |
9.1 |
|
| 2016-07-27 10:21:17 |
Roman Podoliaka |
attachment added |
|
a process core dump, python2.7 ubuntu 16.04 https://bugs.launchpad.net/mos/+bug/1606825/+attachment/4708049/+files/core.dump.gz |
|
| 2016-07-27 10:28:19 |
Roman Podoliaka |
description |
While executing a call to librbd nova-compute can hang for a while and eventually go down in nova service-list output.
strace'ing shows that a process is stuck on acquiring a mutex:
root@node-153:~# strace -p 16675
Process 16675 attached
futex(0x7fff084ce36c, FUTEX_WAIT_PRIVATE, 1, NULL
gdb allows to see the traceback:
http://paste.openstack.org/show/542101/
^ which basically means calls to librbd (C library) are not monkey-patched and do not allow switch context to another green thread in a eventlet-based process.
To fix this we need something similar to:
https://review.openstack.org/#/c/223077/ |
While executing a call to librbd nova-compute can hang for a while and eventually go down in nova service-list output.
strace'ing shows that a process is stuck on acquiring a mutex:
root@node-153:~# strace -p 16675
Process 16675 attached
futex(0x7fff084ce36c, FUTEX_WAIT_PRIVATE, 1, NULL
gdb allows to see the traceback:
http://paste.openstack.org/show/542534/
^ which basically means calls to librbd (C library) are not monkey-patched and do not allow switch context to another green thread in a eventlet-based process.
To fix this we need something similar to:
https://review.openstack.org/#/c/223077/ |
|
| 2016-07-28 17:06:29 |
Roman Podoliaka |
description |
While executing a call to librbd nova-compute can hang for a while and eventually go down in nova service-list output.
strace'ing shows that a process is stuck on acquiring a mutex:
root@node-153:~# strace -p 16675
Process 16675 attached
futex(0x7fff084ce36c, FUTEX_WAIT_PRIVATE, 1, NULL
gdb allows to see the traceback:
http://paste.openstack.org/show/542534/
^ which basically means calls to librbd (C library) are not monkey-patched and do not allow switch context to another green thread in a eventlet-based process.
To fix this we need something similar to:
https://review.openstack.org/#/c/223077/ |
While executing a call to librbd nova-compute may hang for a while and eventually go down in nova service-list output.
strace'ing shows that a process is stuck on acquiring a mutex:
root@node-153:~# strace -p 16675
Process 16675 attached
futex(0x7fff084ce36c, FUTEX_WAIT_PRIVATE, 1, NULL
gdb allows to see the traceback:
http://paste.openstack.org/show/542534/
^ which basically means calls to librbd (C library) are not monkey-patched and do not allow to switch the execution context to another green thread in a eventlet-based process.
To avoid blocking of the whole nova-compute process on calls to librbd we should wrap them with tpool.execute() (http://eventlet.net/doc/threading.html#eventlet.tpool.execute) |
|
| 2016-07-28 17:06:55 |
Roman Podoliaka |
description |
While executing a call to librbd nova-compute may hang for a while and eventually go down in nova service-list output.
strace'ing shows that a process is stuck on acquiring a mutex:
root@node-153:~# strace -p 16675
Process 16675 attached
futex(0x7fff084ce36c, FUTEX_WAIT_PRIVATE, 1, NULL
gdb allows to see the traceback:
http://paste.openstack.org/show/542534/
^ which basically means calls to librbd (C library) are not monkey-patched and do not allow to switch the execution context to another green thread in a eventlet-based process.
To avoid blocking of the whole nova-compute process on calls to librbd we should wrap them with tpool.execute() (http://eventlet.net/doc/threading.html#eventlet.tpool.execute) |
Upstream bug: https://bugs.launchpad.net/nova/+bug/1607461
While executing a call to librbd nova-compute may hang for a while and eventually go down in nova service-list output.
strace'ing shows that a process is stuck on acquiring a mutex:
root@node-153:~# strace -p 16675
Process 16675 attached
futex(0x7fff084ce36c, FUTEX_WAIT_PRIVATE, 1, NULL
gdb allows to see the traceback:
http://paste.openstack.org/show/542534/
^ which basically means calls to librbd (C library) are not monkey-patched and do not allow to switch the execution context to another green thread in a eventlet-based process.
To avoid blocking of the whole nova-compute process on calls to librbd we should wrap them with tpool.execute() (http://eventlet.net/doc/threading.html#eventlet.tpool.execute) |
|
| 2016-09-09 10:33:16 |
Roman Podoliaka |
mos/9.x: status |
Confirmed |
Fix Committed |
|
| 2016-09-09 10:33:52 |
Roman Podoliaka |
mos/10.0.x: status |
Confirmed |
Fix Committed |
|
| 2016-09-13 11:16:01 |
Timur Nurlygayanov |
mos/10.0.x: status |
Fix Committed |
Fix Released |
|
| 2016-09-13 11:16:04 |
Timur Nurlygayanov |
mos/9.x: status |
Fix Committed |
Fix Released |
|
