| 2019-04-19 22:12:45 |
iain MacDonnell |
bug |
|
|
added bug |
| 2019-04-19 22:21:18 |
iain MacDonnell |
bug |
|
|
added subscriber Chris Dent |
| 2019-04-19 22:21:44 |
iain MacDonnell |
bug |
|
|
added subscriber Matthew Booth |
| 2019-04-19 22:22:30 |
iain MacDonnell |
description |
Stein nova-api running under uWSGI presents an AMQP issue. The first API call that requires RPC creates an AMQP connection and successfully completes. Normally regular heartbeats would be sent from this point on, to maintain the connection. This is not happening. After a few minutes, the AMQP server (rabbitmq, in my case) notices that there have been no heartbeats, and drops the connection. A later nova API call that requires RPC tries to use the old connection, and throws a "connection reset by peer" exception and the API call fails. A mailing-list reponse suggests that this is affecting mod_wsgi also:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005310.html
I've discovered that this problem seems to be caused by eventlet monkey-patching, which was introduced in:
https://github.com/openstack/nova/commit/23ba1c690652832c655d57476630f02c268c87ae
It was later rearranged in:
https://github.com/openstack/nova/commit/3c5e2b0e9fac985294a949852bb8c83d4ed77e04
but this problem remains.
If I comment out the import of nova.monkey_patch in nova/api/openstack/__init__.py the problem goes away.
Seems that eventlet monkey-patching and uWSGI are not getting along for some reason... |
Stein nova-api running under uWSGI presents an AMQP issue. The first API call that requires RPC creates an AMQP connection and successfully completes. Normally regular heartbeats would be sent from this point on, to maintain the connection. This is not happening. After a few minutes, the AMQP server (rabbitmq, in my case) notices that there have been no heartbeats, and drops the connection. A later nova API call that requires RPC tries to use the old connection, and throws a "connection reset by peer" exception and the API call fails. A mailing-list response suggests that this is affecting mod_wsgi also:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005310.html
I've discovered that this problem seems to be caused by eventlet monkey-patching, which was introduced in:
https://github.com/openstack/nova/commit/23ba1c690652832c655d57476630f02c268c87ae
It was later rearranged in:
https://github.com/openstack/nova/commit/3c5e2b0e9fac985294a949852bb8c83d4ed77e04
but this problem remains.
If I comment out the import of nova.monkey_patch in nova/api/openstack/__init__.py the problem goes away.
Seems that eventlet monkey-patching and uWSGI are not getting along for some reason... |
|
| 2019-04-20 10:15:30 |
Damien Ciabrini |
bug |
|
|
added subscriber Damien Ciabrini |
| 2019-04-25 13:39:34 |
Emilien Macchi |
bug |
|
|
added subscriber Emilien Macchi |
| 2019-04-28 23:27:43 |
Bogdan Katynski |
bug |
|
|
added subscriber Bogdan Katynski |
| 2019-05-27 10:02:38 |
Bogdan Dobrelya |
nova: status |
New |
Confirmed |
|
| 2019-08-07 16:24:58 |
HT |
bug |
|
|
added subscriber HT |
| 2019-10-02 17:54:50 |
OpenStack Infra |
tags |
|
in-stable-stein |
|
| 2020-04-28 22:28:23 |
OpenStack Infra |
nova: status |
Confirmed |
In Progress |
|
| 2020-04-28 22:28:23 |
OpenStack Infra |
nova: assignee |
|
sean mooney (sean-k-mooney) |
|
| 2020-05-07 18:47:45 |
OpenStack Infra |
nova: assignee |
sean mooney (sean-k-mooney) |
Dan Smith (danms) |
|
| 2020-05-08 02:32:36 |
OpenStack Infra |
nova: status |
In Progress |
Fix Released |
|
| 2020-05-22 15:06:39 |
OpenStack Infra |
tags |
in-stable-stein |
in-stable-stein in-stable-ussuri |
|
| 2020-07-08 16:08:54 |
sean mooney |
nominated for series |
|
nova/train |
|
| 2020-07-08 16:08:54 |
sean mooney |
bug task added |
|
nova/train |
|
| 2020-07-08 16:08:54 |
sean mooney |
nominated for series |
|
nova/stein |
|
| 2020-07-08 16:08:54 |
sean mooney |
bug task added |
|
nova/stein |
|
| 2020-07-08 16:08:54 |
sean mooney |
nominated for series |
|
nova/ussuri |
|
| 2020-07-08 16:08:54 |
sean mooney |
bug task added |
|
nova/ussuri |
|
| 2020-07-08 16:09:02 |
sean mooney |
nova/ussuri: status |
New |
Fix Released |
|
| 2020-07-08 16:09:13 |
sean mooney |
nova/train: status |
New |
In Progress |
|
| 2020-07-08 16:09:19 |
sean mooney |
nova/train: assignee |
|
sean mooney (sean-k-mooney) |
|
| 2020-07-08 16:09:22 |
sean mooney |
nova/ussuri: assignee |
|
sean mooney (sean-k-mooney) |
|
| 2020-07-08 16:09:27 |
sean mooney |
nova/stein: assignee |
|
sean mooney (sean-k-mooney) |
|
| 2020-07-08 16:09:29 |
sean mooney |
nova/stein: status |
New |
In Progress |
|
| 2020-07-08 16:09:34 |
sean mooney |
nova: importance |
Undecided |
Low |
|
| 2020-07-08 16:09:37 |
sean mooney |
nova/stein: importance |
Undecided |
Low |
|
| 2020-07-08 16:09:42 |
sean mooney |
nova/train: importance |
Undecided |
Low |
|
| 2020-07-08 16:09:46 |
sean mooney |
nova/ussuri: importance |
Undecided |
Low |
|
| 2020-07-14 12:55:16 |
OpenStack Infra |
nova/train: assignee |
sean mooney (sean-k-mooney) |
Lee Yarwood (lyarwood) |
|
| 2020-07-17 17:36:05 |
OpenStack Infra |
nova/train: assignee |
Lee Yarwood (lyarwood) |
sean mooney (sean-k-mooney) |
|
| 2020-08-05 21:30:05 |
OpenStack Infra |
nova/train: status |
In Progress |
Fix Committed |
|
| 2020-09-02 07:37:46 |
Ivan Hitos |
bug |
|
|
added subscriber Ivan Hitos |
| 2020-11-24 13:34:52 |
Elod Illes |
nova/train: status |
Fix Committed |
Fix Released |
|
| 2021-08-17 06:12:01 |
Brett Milford |
bug |
|
|
added subscriber Brett Milford |
