| 2024-09-04 08:35:37 |
Darin Chakalov |
description |
When flow logging is enabled, try
router set --disable <router>
And in neutron-l3-agent.log you’ll see:
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent [-] Error while deleting router ac2068d0-2dba-4411-b74e-e5fd5ecd1179: TypeError: unhashable type: 'dict'
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron/agent/l3/agent.py", line 514, in _safe_router_removed
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent self.l3_ext_manager.delete_router(self.context, ri.router)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron/agent/l3/l3_agent_extensions_manager.py", line 63, in delete_router
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent extension.obj.delete_router(context, data)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron/services/logapi/agent/l3/base.py", line 105, in delete_router
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent self.log_driver.stop_logging(context, router_info=data)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron_fwaas/services/logapi/agents/drivers/iptables/log.py", line 250, in stop_logging
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent self._cleanup_nflog_process(router_info)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron_fwaas/services/logapi/agents/drivers/iptables/log.py", line 180, in _cleanup_nflog_process
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent if router_info in self.nflog_proc_map:
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent TypeError: unhashable type: 'dict'
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent
2024-08-21 06:59:47.589 16 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for ac2068d0-2dba-4411-b74e-e5fd5ecd1179, action 3
The issue is caused by an incorrect key lookup which was preventing the killing of the process on router deletion which was leading to orphaned processes |
When flow logging is enabled, try
router set --disable <router>
And in neutron-l3-agent.log you’ll see:
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent [-] Error while deleting router ac2068d0-2dba-4411-b74e-e5fd5ecd1179: TypeError: unhashable type: 'dict'
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron/agent/l3/agent.py", line 514, in _safe_router_removed
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent self.l3_ext_manager.delete_router(self.context, ri.router)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron/agent/l3/l3_agent_extensions_manager.py", line 63, in delete_router
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent extension.obj.delete_router(context, data)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron/services/logapi/agent/l3/base.py", line 105, in delete_router
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent self.log_driver.stop_logging(context, router_info=data)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron_fwaas/services/logapi/agents/drivers/iptables/log.py", line 250, in stop_logging
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent self._cleanup_nflog_process(router_info)
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib64/python3.11/site-packages/neutron_fwaas/services/logapi/agents/drivers/iptables/log.py", line 180, in _cleanup_nflog_process
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent if router_info in self.nflog_proc_map:
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent TypeError: unhashable type: 'dict'
2024-08-21 06:59:47.588 16 ERROR neutron.agent.l3.agent
2024-08-21 06:59:47.589 16 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for ac2068d0-2dba-4411-b74e-e5fd5ecd1179, action 3
The issue is caused by an incorrect key lookup which was preventing the killing of the process on router deletion which was leading to orphaned processes
Suggesded fix: https://review.opendev.org/c/openstack/neutron-fwaas/+/927905 |
|
