Periodic fs020 train job got failed while running basic ops tests.
http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-train/7e12b11/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz
tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic[compute,id-e79f879e-debb-440c-a7e4-efeda05b6848,network]
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/tempest/common/utils/__init__.py", line 89, in wrapper
return f(*func_args, **func_kwargs)
File "/usr/lib/python2.7/site-packages/tempest/scenario/test_security_groups_basic_ops.py", line 508, in test_cross_tenant_traffic
self._test_cross_tenant_allow(source_tenant, dest_tenant, ruleset)
File "/usr/lib/python2.7/site-packages/tempest/scenario/test_security_groups_basic_ops.py", line 424, in _test_cross_tenant_allow
self.check_remote_connectivity(access_point_ssh, ip, protocol=protocol)
File "/usr/lib/python2.7/site-packages/tempest/scenario/manager.py", line 1078, in check_remote_connectivity
self.fail(msg)
File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 690, in fail
raise self.failureException(msg)
AssertionError: Timed out waiting for 10.0.0.102 to become reachable from 10.0.0.117
It traied to ping 10.0.0.102 that ip but always there is a packet loss.
1 packets transmitted, 0 packets received, 100% packet loss
2019-12-11 02:37:39,592 307032 WARNING [tempest.scenario.manager] Failed to check icmp connectivity for IP 10.0.0.102 via a ssh connection from: 10.0.0.117.
2019-12-11 02:37:40,593 307032 DEBUG [tempest.lib.common.utils.linux.remote_client] Remote command: set -eu -o pipefail; PATH=$PATH:/sbin:/usr/sbin; ping -c1 -w1 -s56 10.0.0.102
2019-12-11 02:37:40,594 307032 INFO [tempest.lib.common.ssh] Creating ssh connection to '10.0.0.117:22' as 'cirros' with public key authentication
2019-12-11 02:37:40,653 307032 INFO [paramiko.transport] Connected (version 2.0, client dropbear_2015.67)
2019-12-11 02:37:40,972 307032 INFO [paramiko.transport] Authentication (publickey) successful!
2019-12-11 02:37:41,001 307032 INFO [tempest.lib.common.ssh] ssh connection to cirros@10.0.0.117 successfully created
2019-12-11 02:37:42,204 307032 ERROR [tempest.lib.common.utils.linux.remote_client] (TestSecurityGroupsBasicOps:test_cross_tenant_traffic) Executing command on 10.0.0.117 failed. Error: Command 'set -eu -o pipefail; PATH=$PATH:/sbin:/usr/sbin; ping -c1 -w1 -s56 10.0.0.102' failed, exit status: 1, stderr:
I checked the neutron and ovn logs nothing interesting found, adding more info as got more.
Not sure it is linked https:/