It's seen that these tests interfered(removed router namespace) with test_metadata_proxy_rate_limiting_ipv6 but can interfere with others too, let's run these serially to avoid random failures.
Seen in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_08a/885449/22/check/neutron-functional-with-uwsgi/08af95d/testr_results.html
Fails like:-
ft1.5: neutron.tests.functional.agent.l3.test_metadata_proxy.MetadataL3AgentTestCase.test_metadata_proxy_rate_limiting_ipv6testtools.testresult.real._StringException: stderr: {{{
Traceback (most recent call last):
File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
timer()
File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
cb(*args, **kw)
File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
result = function(*args, **kwargs)
File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/oslo_concurrency/lockutils.py", line 412, in inner
return f(*args, **kwargs)
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/external_process.py", line 274, in _check_child_processes
self._execute_action(service_id)
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/external_process.py", line 285, in _execute_action
action_function(service_id)
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/external_process.py", line 291, in _respawn_action
self._monitored_processes[service_id].enable()
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/external_process.py", line 105, in enable
ip_wrapper.netns.execute(cmd, addl_env=self.cmd_addl_env,
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 775, in execute
return utils.execute(cmd, check_exit_code=check_exit_code,
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py", line 156, in execute
raise exceptions.ProcessExecutionError(msg,
neutron_lib.exceptions.ProcessExecutionError: Exit code: 255; Cmd: ['ip', 'netns', 'exec', 'qrouter-0156eaec-06cc-43a7-83cb-1a72504f7624', 'env', 'PROCESS_TAG=haproxy-0156eaec-06cc-43a7-83cb-1a72504f7624', 'haproxy', '-f', '/tmp/tmpgh6ctwf3/tmpuategvfx/ns-metadata-proxy/0156eaec-06cc-43a7-83cb-1a72504f7624.conf']; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-0156eaec-06cc-43a7-83cb-1a72504f7624": No such file or directory
}}}
Traceback (most recent call last):
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/test_metadata_proxy.py", line 109, in _query_metadata_proxy
raw_headers = machine.execute(cmd)
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/common/machine_fixtures.py", line 56, in execute
return ns_ip_wrapper.netns.execute(*args, **kwargs)
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/ip_lib.py", line 775, in execute
return utils.execute(cmd, check_exit_code=check_exit_code,
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py", line 156, in execute
raise exceptions.ProcessExecutionError(msg,
neutron_lib.exceptions.ProcessExecutionError: Exit code: 28; Cmd: ['ip', 'netns', 'exec', 'test-458fcfac-f975-44ef-8a9d-86076a7af8c9', 'curl', '--max-time', 60, '-D-', 'http://[fe80::a9fe:a9fe%port0e40b9]:80']; Stdin: ; Stdout: ; Stderr: % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
curl: (28) Connection timed out after 60000 milliseconds
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
return f(self, *args, **kwargs)
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/test_metadata_proxy.py", line 194, in test_metadata_proxy_rate_limiting_ipv6
self._test_rate_limiting(2, machine, ipv6=True, interface=interface)
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/test_metadata_proxy.py", line 169, in _test_rate_limiting
firstline = self._query_metadata_proxy(machine, ipv6=ipv6,
File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/agent/l3/test_metadata_proxy.py", line 116, in _query_metadata_proxy
self.fail('metadata proxy unreachable '
File "/usr/lib/python3.10/unittest/case.py", line 675, in fail
raise self.failureException(msg)
AssertionError: metadata proxy unreachable on http://[fe80::a9fe:a9fe%port0e40b9]:80 before timeout
From test_periodic_sync_routers_task_routers_deleted_while_agent_sync.txt
2024-06-11 01:28:12.647 38519 DEBUG neutron.privileged.agent.linux.ip_lib [-] Namespace qrouter-0156eaec-06cc-43a7-83cb-1a72504f7624 deleted. remove_netns /home/zuul/src/opendev.org/openstack/neutron/neutron/privileged/agent/linux/ip_lib.py:607
From failing test:-
2024-06-11 01:29:20.707 38222 ERROR neutron.agent.linux.utils [-] Exit code: 255; Cmd: ['ip', 'netns', 'exec', 'qrouter-0156eaec-06cc-43a7-83cb-1a72504f7624', 'env', 'PROCESS_TAG=haproxy-0156eaec-06cc-43a7-83cb-1a72504f7624', 'haproxy', '-f', '/tmp/tmpgh6ctwf3/tmpuategvfx/ns-metadata-proxy/0156eaec-06cc-43a7-83cb-1a72504f7624.conf']; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "qrouter-0156eaec-06cc-43a7-83cb-1a72504f7624": No such file or directory
Fix proposed to branch: master
Review: https:/