| 2016-10-12 02:54:46 |
zhichao zhu |
bug |
|
|
added bug |
| 2016-10-12 02:54:57 |
zhichao zhu |
neutron: assignee |
|
zhichao zhu (rtmdk) |
|
| 2016-10-12 02:56:26 |
zhichao zhu |
tags |
l3-dvr |
l3-agent |
|
| 2016-10-12 03:14:03 |
zhichao zhu |
description |
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent [req-5d499217-05b6-4a56-a3b7-5681adb53d6c - d2b95803757641b6bc55f6309c12c6e9 - - -] Failed to process compatible router 'da82aeb4-07a4-45ca-ae7a-570aec69df29'
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 501, in _process_router_update
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 438, in _process_router_if_compatible
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_added_router(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 446, in _process_added_router
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent ri.process(self)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_local_router.py", line 488, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrLocalRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_router_base.py", line 30, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrRouterBase, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 386, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(HaRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 385, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.logger(e)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.force_reraise()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 382, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent return func(*args, **kwargs)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 964, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.process_address_scope()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_edge_router.py", line 239, in process_address_scope
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.snat_iptables_manager, ports_scopemark)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.gen.next()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 461, in defer_apply
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent raise n_exc.IpTablesApplyException(msg)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent IpTablesApplyException: Failure applying iptables rules
this ERROR information will fill l3-agent log file continuously until solving the problem ,it will fill the file space. |
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent [req-5d499217-05b6-4a56-a3b7-5681adb53d6c - d2b95803757641b6bc55f6309c12c6e9 - - -] Failed to process compatible router 'da82aeb4-07a4-45ca-ae7a-570aec69df29'
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 501, in _process_router_update
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 438, in _process_router_if_compatible
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_added_router(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 446, in _process_added_router
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent ri.process(self)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_local_router.py", line 488, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrLocalRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_router_base.py", line 30, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrRouterBase, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 386, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(HaRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 385, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.logger(e)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.force_reraise()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 382, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent return func(*args, **kwargs)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 964, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.process_address_scope()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_edge_router.py", line 239, in process_address_scope
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.snat_iptables_manager, ports_scopemark)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.gen.next()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 461, in defer_apply
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent raise n_exc.IpTablesApplyException(msg)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent IpTablesApplyException: Failure applying iptables rules
for example,this ERROR information will fill l3-agent log file continuously until solving the problem ,it will fill the log file space.
because we resyc the ERROR update into the queue when the update is not been handle successfully.then the greenthread in l3-agent will deal with the update periodicly,so print the log periodicly, but the l3 agent has been deal with this update,wo should delete this update. |
|
| 2016-10-12 03:17:13 |
zhichao zhu |
tags |
l3-agent |
l3-dvr-backlog |
|
| 2016-10-12 03:26:59 |
zhichao zhu |
description |
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent [req-5d499217-05b6-4a56-a3b7-5681adb53d6c - d2b95803757641b6bc55f6309c12c6e9 - - -] Failed to process compatible router 'da82aeb4-07a4-45ca-ae7a-570aec69df29'
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 501, in _process_router_update
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 438, in _process_router_if_compatible
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_added_router(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 446, in _process_added_router
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent ri.process(self)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_local_router.py", line 488, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrLocalRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_router_base.py", line 30, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrRouterBase, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 386, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(HaRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 385, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.logger(e)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.force_reraise()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 382, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent return func(*args, **kwargs)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 964, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.process_address_scope()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_edge_router.py", line 239, in process_address_scope
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.snat_iptables_manager, ports_scopemark)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.gen.next()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 461, in defer_apply
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent raise n_exc.IpTablesApplyException(msg)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent IpTablesApplyException: Failure applying iptables rules
for example,this ERROR information will fill l3-agent log file continuously until solving the problem ,it will fill the log file space.
because we resyc the ERROR update into the queue when the update is not been handle successfully.then the greenthread in l3-agent will deal with the update periodicly,so print the log periodicly, but the l3 agent has been deal with this update,wo should delete this update. |
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent [req-5d499217-05b6-4a56-a3b7-5681adb53d6c - d2b95803757641b6bc55f6309c12c6e9 - - -] Failed to process compatible router 'da82aeb4-07a4-45ca-ae7a-570aec69df29'
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 501, in _process_router_update
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 438, in _process_router_if_compatible
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self._process_added_router(router)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 446, in _process_added_router
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent ri.process(self)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_local_router.py", line 488, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrLocalRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_router_base.py", line 30, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(DvrRouterBase, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/ha_router.py", line 386, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent super(HaRouter, self).process(agent)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 385, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.logger(e)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.force_reraise()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/common/utils.py", line 382, in call
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent return func(*args, **kwargs)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 964, in process
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.process_address_scope()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/l3/dvr_edge_router.py", line 239, in process_address_scope
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.snat_iptables_manager, ports_scopemark)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent self.gen.next()
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 461, in defer_apply
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent raise n_exc.IpTablesApplyException(msg)
2016-10-12 10:04:38.587 25667 ERROR neutron.agent.l3.agent IpTablesApplyException: Failure applying iptables rules
for example,this ERROR information will fill l3-agent log file continuously until solving the problem ,it will fill the log file space.
because we resyc the ERROR update into the queue when the update is not been handle successfully.then the greenthread in l3-agent will deal with the update periodicly,so print the log periodicly, but the l3 agent has been deal with this update,wo should delete this update.
we could disable l3-agent in a networknode in ha model, then create router,then restart the l3-agent ，we will see this problem。 |
|
| 2016-10-12 03:37:34 |
Brian Haley |
bug |
|
|
added subscriber Brian Haley |
| 2016-10-13 23:06:39 |
OpenStack Infra |
neutron: status |
New |
In Progress |
|
| 2016-10-18 19:58:39 |
Dariusz Smigiel |
neutron: status |
In Progress |
Incomplete |
|
| 2017-02-16 12:35:26 |
OpenStack Infra |
neutron: status |
Incomplete |
In Progress |
|
| 2017-02-16 12:35:26 |
OpenStack Infra |
neutron: assignee |
zhichao zhu (rtmdk) |
Quan Tian (tianquan23) |
|
| 2017-02-16 16:27:28 |
Brian Haley |
tags |
l3-dvr-backlog |
l3-dvr-backlog newton-backport-potential |
|
| 2017-03-21 08:22:48 |
Lv Jiawei |
bug |
|
|
added subscriber Lv Jiawei |
| 2017-04-06 21:53:23 |
OpenStack Infra |
neutron: assignee |
Quan Tian (tianquan23) |
Ihar Hrachyshka (ihar-hrachyshka) |
|
| 2017-04-11 16:34:06 |
OpenStack Infra |
neutron: assignee |
Ihar Hrachyshka (ihar-hrachyshka) |
Brian Haley (brian-haley) |
|
| 2017-04-17 18:16:29 |
OpenStack Infra |
tags |
l3-dvr-backlog newton-backport-potential |
in-stable-ocata l3-dvr-backlog newton-backport-potential |
|
| 2017-05-05 23:49:58 |
OpenStack Infra |
tags |
in-stable-ocata l3-dvr-backlog newton-backport-potential |
in-stable-newton in-stable-ocata l3-dvr-backlog newton-backport-potential |
|
| 2017-06-13 22:01:31 |
Ihar Hrachyshka |
tags |
in-stable-newton in-stable-ocata l3-dvr-backlog newton-backport-potential |
in-stable-newton in-stable-ocata l3-dvr-backlog neutron-proactive-backport-potential newton-backport-potential |
|
| 2017-06-14 16:38:47 |
Ihar Hrachyshka |
neutron: status |
In Progress |
Fix Released |
|
| 2017-06-14 16:38:52 |
Ihar Hrachyshka |
tags |
in-stable-newton in-stable-ocata l3-dvr-backlog neutron-proactive-backport-potential newton-backport-potential |
in-stable-newton in-stable-ocata l3-dvr-backlog newton-backport-potential |
|
| 2017-06-14 16:39:00 |
Ihar Hrachyshka |
tags |
in-stable-newton in-stable-ocata l3-dvr-backlog newton-backport-potential |
in-stable-newton in-stable-ocata l3-dvr-backlog |
|
