new info: run twice confirm the issues. Watch the netns change, and find: The netns qrouter* missed at: qrouter-d7faa15d-6f88-44de-8fef-7de54e7fee88 (id: 1) qdhcp-5de5301b-a3f4-482d-8a1b-23badaff80c4 (id: 0) Thu Feb 20 07:37:21 GMT 2020 2 qrouter-d7faa15d-6f88-44de-8fef-7de54e7fee88 (id: 1) qdhcp-5de5301b-a3f4-482d-8a1b-23badaff80c4 (id: 0) Thu Feb 20 07:37:22 GMT 2020 1 07:37:21 And that time the l3-agent had already tried for iptables restore many times. 2020-02-20 07:37:21.735 12 ERROR neutron.agent.linux.iptables_manager line 2: CHAIN_ADD failed (Device or resource busy): chain OUTPUT 2020-02-20 07:37:21.735 12 ERROR neutron.agent.linux.iptables_manager line 12: RULE_INSERT failed (No such file or directory): rule in chain OUTPUT 2020-02-20 07:37:21.735 12 ERROR neutron.agent.linux.iptables_manager line 13: RULE_INSERT failed (No such file or directory): rule in chain POSTROUTING 2020-02-20 07:37:21.735 12 ERROR neutron.agent.linux.iptables_manager line 14: RULE_APPEND failed (No such file or directory): rule in chain POSTROUTING 2020-02-20 07:37:21.735 12 ERROR neutron.agent.linux.iptables_manager line 15: RULE_INSERT failed (No such file or directory): rule in chain PREROUTING 2020-02-20 07:37:21.735 12 ERROR neutron.agent.linux.iptables_manager 2020-02-20 07:37:21.735 12 ERROR neutron.agent.linux.iptables_manager 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info [-] Failure applying iptables rules: IpTablesApplyException: Failure applying iptables rules 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info Traceback (most recent call last): 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/common/utils.py", line 158, in call 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info return func(*args, **kwargs) 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 1189, in process 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info self.process_address_scope() 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 1152, in process_address_scope 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info self.process_floating_ip_address_scope_rules() 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info self.gen.next() 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 438, in defer_apply 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info raise l3_exc.IpTablesApplyException(msg) 2020-02-20 07:37:21.736 12 ERROR neutron.agent.l3.router_info IpTablesApplyException: Failure applying iptables rules 2020-02-20 07:37:21.738 12 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for d7faa15d-6f88-44de-8fef-7de54e7fee88, action 3 The last error message here for l3-agent is also 07:37:21 ================================================================== After that the router namespace disappear. So the root cause is with l3-agent service. After reboot, it recover and netns qrouter* come back.