This is reproducible on centos7/train (Dincer case): need to reconfigure and (extra) restart l3 agent container. The issue is that on 2nd restart container has invalid references to netns inside itself - instead of proc mounts, it has a bunch of regular files which are unusable for l3 agent. Logs: 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent [-] Error while initializing router 87c3c36f-1038-45d4-afc0-c9c109239b66: ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: setting the network namespace "qrouter-87c3c36f-1038-45d4-afc0-c9c109239b66" failed: Inv alid argument 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 485, in _router_added 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor) 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/router_info.py", line 167, in initialize 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent self.router_namespace.create() 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/namespaces.py", line 97, in create 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent ip_wrapper.netns.execute(cmd) 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 713, in execute 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent run_as_root=run_as_root) 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 147, in execute 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent returncode=returncode) 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: setting the network namespace "qrouter-87c3c36f-1038-45d4-afc0-c9c109239b66" failed: Invalid argument 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent 2020-02-27 12:20:58.626 17 ERROR neutron.agent.l3.agent 2020-02-27 12:20:58.644 17 WARNING neutron.agent.l3.agent [-] Hit retry limit with router update for 87c3c36f-1038-45d4-afc0-c9c109239b66, action 3 2020-02-27 12:20:58.644 17 WARNING neutron.agent.l3.agent [-] Info for router 87c3c36f-1038-45d4-afc0-c9c109239b66 was not found. Performing router cleanup 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent [-] Error while deleting router 87c3c36f-1038-45d4-afc0-c9c109239b66: OSError: [Errno 22] failed to open netns 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 506, in _safe_router_removed 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent self._router_removed(ri, router_id) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 529, in _router_removed 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent self.namespaces_manager.ensure_router_cleanup(router_id) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/namespace_manager.py", line 129, in ensure_router_cleanup 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent self._cleanup(ns_prefix, ns_id) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/namespace_manager.py", line 143, in _cleanup 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent ns.delete() 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/namespaces.py", line 72, in wrapped 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent return f(self, *args, **kwargs) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/l3/namespaces.py", line 136, in delete 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent for d in ns_ip.get_devices(): 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 167, in get_devices 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent devices = privileged.get_device_names(self.namespace) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/privileged/agent/linux/ip_lib.py", line 582, in get_device_names 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent in get_link_devices(namespace, **kwargs)] 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron/privileged/agent/linux/ip_lib.py", line 53, in sync_inner 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent return input_func(*args, **kwargs) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 245, in _wrap 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent return self.channel.remote_call(name, args, kwargs) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 204, in remote_call 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent raise exc_type(*result[2]) 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent OSError: [Errno 22] failed to open netns 2020-02-27 12:20:58.657 17 ERROR neutron.agent.l3.agent