OpenStack Octavia Charm

Tempest tests fail w/ error: Failed to establish a new connection: [Errno 113] No route to host

Bug #1976391 reported by Konstantinos Kaskavelis on 2022-05-31

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Octavia Charm	New	Undecided	Unassigned

Bug Description

We notice 8 tempest tests failing with the following error:

Traceback (most recent call last):
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connection.py", line 175, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
OSError: [Errno 113] No route to host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn
    conn.connect()
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connection.py", line 187, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7efefbeb39e8>: Failed to establish a new connection: [Errno 113] No route to host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/snap/fcbtest/29/.rally/verification/verifier-f43e44b2-5754-459e-9976-a00f532fd057/repo/tempest/test.py", line 181, in setUpClass
    raise value.with_traceback(trace)
  File "/home/ubuntu/snap/fcbtest/29/.rally/verification/verifier-f43e44b2-5754-459e-9976-a00f532fd057/repo/tempest/test.py", line 174, in setUpClass
    cls.resource_setup()
  File "/snap/fcbtest/29/lib/python3.6/site-packages/octavia_tempest_plugin/tests/test_base.py", line 149, in resource_setup
    cls.api_version = cls.mem_lb_client.get_max_api_version()
  File "/snap/fcbtest/29/lib/python3.6/site-packages/octavia_tempest_plugin/services/load_balancer/v2/base_client.py", line 452, in get_max_api_version
    response, body = self.get('/')
  File "/home/ubuntu/snap/fcbtest/29/.rally/verification/verifier-f43e44b2-5754-459e-9976-a00f532fd057/repo/tempest/lib/common/rest_client.py", line 314, in get
    return self.request('GET', url, extra_headers, headers)
  File "/home/ubuntu/snap/fcbtest/29/.rally/verification/verifier-f43e44b2-5754-459e-9976-a00f532fd057/repo/tempest/lib/common/rest_client.py", line 688, in request
    body=body, chunked=chunked)
  File "/home/ubuntu/snap/fcbtest/29/.rally/verification/verifier-f43e44b2-5754-459e-9976-a00f532fd057/repo/tempest/lib/common/rest_client.py", line 568, in _request
    chunked=chunked
  File "/home/ubuntu/snap/fcbtest/29/.rally/verification/verifier-f43e44b2-5754-459e-9976-a00f532fd057/repo/tempest/lib/common/rest_client.py", line 608, in raw_request
    body=body, chunked=chunked)
  File "/home/ubuntu/snap/fcbtest/29/.rally/verification/verifier-f43e44b2-5754-459e-9976-a00f532fd057/repo/tempest/lib/common/http.py", line 111, in request
    *args, **new_kwargs)
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/request.py", line 75, in request
    method, url, fields=fields, headers=headers, **urlopen_kw
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/request.py", line 96, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/poolmanager.py", line 376, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connectionpool.py", line 826, in urlopen
    **response_kw
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connectionpool.py", line 826, in urlopen
    **response_kw
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connectionpool.py", line 826, in urlopen
    **response_kw
  [Previous line repeated 7 more times]
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/snap/fcbtest/29/lib/python3.6/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='octavia.silo1.solutionsqa', port=9876): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7efefbeb39e8>: Failed to establish a new connection: [Errno 113] No route to host',))

The tests that are failing are the following:

test_load_balancer_delete
test_load_balancer_show
test_load_balancer_show_stats
test_load_balancer_update
test_HTTPS_RR_pool_delete
test_HTTPS_RR_pool_show
test_HTTPS_RR_pool_update
test_HTTP_RR_pool_list