Failure examples:
https://b30211aa4f809fc4a91b-baf4f807d40559415da582760ebf9456.ssl.cf2.rackcdn.com/817525/7/check/neutron-tempest-plugin-scenario-ovn/c356679/testr_results.html
https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b75/815962/5/check/neutron-tempest-plugin-scenario-ovn/b75474f/testr_results.html
Stacktrace:
Traceback (most recent call last):
File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 107, in _get_ssh_connection
ssh.connect(self.host, port=self.port, username=self.username,
File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/paramiko/client.py", line 368, in connect
raise NoValidConnectionsError(errors)
paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.24.5.220
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/scenario/test_mac_learning.py", line 166, in test_mac_learning_vms_on_same_network
self._prepare_listener(non_receiver, 2)
File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/scenario/test_mac_learning.py", line 138, in _prepare_listener
self._check_cmd_installed_on_server(server['ssh_client'], server['id'],
File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/scenario/test_mac_learning.py", line 121, in _check_cmd_installed_on_server
ssh_client.execute_script('which %s' % cmd)
File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/common/ssh.py", line 224, in execute_script
channel = self.open_session()
File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/common/ssh.py", line 149, in open_session
client = self.connect()
File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/common/ssh.py", line 137, in connect
return super(Client, self)._get_ssh_connection(*args, **kwargs)
File "/opt/stack/tempest/tempest/lib/common/ssh.py", line 126, in _get_ssh_connection
raise exceptions.SSHTimeout(host=self.host,
tempest.lib.exceptions.SSHTimeout: Connection to the 172.24.5.220 via SSH timed out.
User: ubuntu, Password: None
We need to check why this specific test is failing in the ovn job more often than other tests.
In both the job failures and an other[1] i see "No correct output in console of server <id> found. Guest operating system status can't be checked.". Also in these tests console output is not getting collected so it's hard to say what's going wrong.
In this tests 3 servers are getting created, and in the three failures linked here different server(sender, receiver and non_receiver) preparation failed, so it looks more generic.
Also seen a failure in different test(test_ multiple_ ports_portrange _remote) in same job[2] which also creating 3 servers, but as console log got collected there it got known that it failed as there was kernel panic. May be test_mac_ learning_ vms_on_ same_network test also faced similar issue and collecting console log in it may give better idea.
[1] https:/ /storage. gra.cloud. ovh.net/ v1/AUTH_ dcaab5e32b234d5 6b626f72581e364 4c/zuul_ opendev_ logs_296/ 818443/ 7/check/ neutron- tempest- plugin- scenario- ovn/2969549/ testr_results. html /9470f97960e4f6 89dfc7- d2f445d35e133f7 0dec3633372bebb ba.ssl. cf2.rackcdn. com/818911/ 2/check/ neutron- tempest- plugin- scenario- ovn/c2ecf34/ testr_results. html
[2] https:/