| 2016-07-27 09:09:04 |
John Schwarz |
bug |
|
|
added bug |
| 2016-07-27 09:11:21 |
John Schwarz |
description |
The scenario which initially revealed this issue involved multiple controllers and an extra compute node (total of 4) but it should also reproduce on deployments smaller than described.
The issue is that if an agent tries to report_state to the neutron-server and it fails because of a timeout (raising oslo_messaging.MessagingTimeout), then there is an exponential back-off effect which was put in place by [1]. The feature was intended for heavy RPC calls (like get_routers()) and not for light calls such as report_state, so this can be considered a regression.
A solution would be to ensure PluginReportStateAPI doesn't use the exponential backoff, instead seeking to always time out after rpc_response_timeout.
[1]: https://review.openstack.org/#/c/280595/14/neutron/common/rpc.py |
The scenario which initially revealed this issue involved multiple controllers and an extra compute node (total of 4) but it should also reproduce on deployments smaller than described.
The issue is that if an agent tries to report_state to the neutron-server and it fails because of a timeout (raising oslo_messaging.MessagingTimeout), then there is an exponential back-off effect which was put in place by [1]. The feature was intended for heavy RPC calls (like get_routers()) and not for light calls such as report_state, so this can be considered a regression. This can be reproduced by restarting the controllers on a triple-O deployment and specified before.
A solution would be to ensure PluginReportStateAPI doesn't use the exponential backoff, instead seeking to always time out after rpc_response_timeout.
[1]: https://review.openstack.org/#/c/280595/14/neutron/common/rpc.py |
|
| 2016-07-27 09:11:34 |
John Schwarz |
tags |
|
mitaka-backport-potential |
|
| 2016-07-27 09:12:18 |
John Schwarz |
tags |
mitaka-backport-potential |
liberty-backport-potential mitaka-backport-potential |
|
| 2016-07-27 09:12:44 |
OpenStack Infra |
neutron: status |
New |
In Progress |
|
| 2016-07-27 09:12:44 |
OpenStack Infra |
neutron: assignee |
|
John Schwarz (jschwarz) |
|
| 2016-07-27 10:56:02 |
Jakub Libosvar |
neutron: importance |
Undecided |
Medium |
|
| 2016-07-29 16:21:38 |
OpenStack Infra |
neutron: status |
In Progress |
Fix Released |
|
| 2016-08-04 18:23:25 |
OpenStack Infra |
tags |
liberty-backport-potential mitaka-backport-potential |
in-stable-mitaka liberty-backport-potential mitaka-backport-potential |
|
| 2016-08-06 02:54:26 |
OpenStack Infra |
tags |
in-stable-mitaka liberty-backport-potential mitaka-backport-potential |
in-stable-liberty in-stable-mitaka liberty-backport-potential mitaka-backport-potential |
|
| 2016-10-07 16:13:09 |
Ihar Hrachyshka |
tags |
in-stable-liberty in-stable-mitaka liberty-backport-potential mitaka-backport-potential |
in-stable-liberty in-stable-mitaka |
|
