Scenario tests neutron_tempest_plugin.bgpvpn.scenario.test_bgpvpn_basic.TestBGPVPNBasic are failing very often

[Slawek Kaplonski]

It seems like tests from module neutron_tempest_plugin.bgpvpn.scenario.test_bgpvpn_basic.TestBGPVPNBasic are failing very often recently on jobs neutron-tempest-plugin-bgpvpn-bagpipe-ussuri and neutron-tempest-plugin-bgpvpn-bagpipe

Example of errors:

https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_2b3/748367/16/check/neutron-tempest-plugin-bgpvpn-bagpipe-ussuri/2b36cb5/testr_results.html

It can be sometimes that only one test fails, or 2 or in some cases even 3.

Stacktrace is always like:

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/common/utils/__init__.py", line 89, in wrapper
    return f(*func_args, **func_kwargs)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/bgpvpn/scenario/test_bgpvpn_basic.py", line 601, in test_bgpvpn_tenant_separation_and_local_connectivity
    self._check_l3_bgpvpn(self.servers[0], self.servers[1],
  File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/bgpvpn/scenario/test_bgpvpn_basic.py", line 1291, in _check_l3_bgpvpn
    self._check_l3_bgpvpn_by_specific_ip(from_server=from_server,
  File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/neutron_tempest_plugin/bgpvpn/scenario/test_bgpvpn_basic.py", line 1330, in _check_l3_bgpvpn_by_specific_ip
    self.assertTrue(result, msg)
  File "/opt/stack/tempest/.tox/tempest/lib/python3.8/site-packages/unittest2/case.py", line 702, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true : Timed out waiting for 10.100.1.30 to become reachable

[Lajos Katona (lajos-katona)]

Not sure if it is related or not, but in passing jobs the kernel is: kernel_version=4.15.0-117-generic
while in case of failing: 4.15.0-118-generic

The strange is that the new 4.15.0-118-generic kernel is both in bionic and in focal.

[Lajos Katona (lajos-katona)]

Patch uploaded: https://review.opendev.org/c/openstack/networking-bagpipe/+/778330

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/networking-bagpipe/+/778330
Reason: If needed, feel free to restore the patch and fix the merge conflict