ExternalGatewayForFloatingIPNotFound: External network is not reachable from subnet Therefore, cannot associate Port with a Floating IP

Bug #1978553 reported by chandan kumar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Unassigned

Bug Description

periodic-tripleo-ci-centos-9-standalone-tempest-wallaby job is failing at following tempest test,
```
tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_attach_detach_volume_to_instance[compute,id-fff42874-7db5-4487-a8e1-ddda5fb5288d,smoke]
-------------------------------------------------------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/tempest/common/utils/__init__.py", line 70, in wrapper
        return f(*func_args, **func_kwargs)
      File "/usr/lib/python3.9/site-packages/tempest/api/volume/test_volumes_actions.py", line 45, in test_attach_detach_volume_to_instance
        server = self.create_server()
      File "/usr/lib/python3.9/site-packages/tempest/api/volume/base.py", line 230, in create_server
        body, _ = compute.create_test_server(
      File "/usr/lib/python3.9/site-packages/tempest/common/compute.py", line 353, in create_test_server
        LOG.exception('Server %s failed to delete in time',
      File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in __exit__
        self.force_reraise()
      File "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
        raise self.value
      File "/usr/lib/python3.9/site-packages/tempest/common/compute.py", line 324, in create_test_server
        _setup_validation_fip(
      File "/usr/lib/python3.9/site-packages/tempest/common/compute.py", line 101, in _setup_validation_fip
        clients.floating_ips_client.update_floatingip(
      File "/usr/lib/python3.9/site-packages/tempest/lib/services/network/floating_ips_client.py", line 41, in update_floatingip
        return self.update_resource(uri, post_data)
      File "/usr/lib/python3.9/site-packages/tempest/lib/services/network/base.py", line 77, in update_resource
        resp, body = self.put(req_uri, req_post_data)
      File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 363, in put
        return self.request('PUT', url, extra_headers, headers, body, chunked)
      File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 703, in request
        self._error_checker(resp, resp_body)
      File "/usr/lib/python3.9/site-packages/tempest/lib/common/rest_client.py", line 809, in _error_checker
        raise exceptions.NotFound(resp_body, resp=resp)
    tempest.lib.exceptions.NotFound: Object not found
    Details: {'type': 'ExternalGatewayForFloatingIPNotFound', 'message': 'External network 84bf1927-4067-46da-a289-1636bd1f2f64 is not reachable from subnet 99c55e30-7435-47fb-9be9-a26cd46b2fcf. Therefore, cannot associate Port 8faba232-4e36-4cb7-8353-4af1edccb327 with a Floating IP.', 'detail': ''}

```
Same tempest rpm openstack-tempest-31.0.0-0.20220608165917.abddb57.el9.noarch(exists in both component ci testing and current-tripleo) is used in passing and failing job.

The job started failing from https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-tempest-wallaby&skip=0 2022-06-13 18:08:38

It is a tracker bug, will add more information as we investigate.

Links:
[failing job]. https://logserver.rdoproject.org/54/36254/116/check/periodic-tripleo-ci-centos-9-standalone-tempest-wallaby/d073cbf/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz

[Passing job] https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-tempest-wallaby/31fe87c/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz

Revision history for this message
chandan kumar (chkumar246) wrote :

By looking at neutron logs and compairing with passing job,

https://logserver.rdoproject.org/54/36254/116/check/periodic-tripleo-ci-centos-9-standalone-tempest-wallaby/d073cbf/logs/undercloud/var/log/containers/neutron/server.log.txt.gz

```
if_exists=True) do_commit /usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
2022-06-14 02:24:28.930 15 ERROR ovsdbapp.backend.ovs_idl.transaction [req-3f5cec56-4039-4520-b6ec-d993b56fe469 9a0073051e7c4a6aae3ed9eb107c44bf e803f82a18c742eba606e656a2fc1e54 - default default] Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
    txn.results.put(txn.do_commit())
  File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
    command.run_idl(txn)
  File "/usr/lib/python3.9/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py", line 763, in run_idl
    raise ovn_exc.RevisionConflict(
neutron.common.ovn.exceptions.RevisionConflict: OVN revision number for b30bc58e-35c1-497c-9482-52ad32b0dc19 (type: ports) is equal or higher than the given resource. Skipping update
```

Not sure above error is linked with the failure.

tags: removed: promotion-blocker
tags: added: promotion-blocker
Revision history for this message
chandan kumar (chkumar246) wrote :
Revision history for this message
chandan kumar (chkumar246) wrote :
Ronelle Landy (rlandy)
Changed in tripleo:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.