The neutron update call in api_crud.toggle_hm_port is not enclosed in try/except

Bug #1888611 reported by Frode Nordahl
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Octavia Charm
Triaged
Medium
Unassigned

Bug Description

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-octavia-1/charm/reactive/octavia_handlers.py", line 142, in setup_hm_port
    if api_crud.setup_hm_port(
  File "lib/charm/openstack/api_crud.py", line 464, in setup_hm_port
    toggle_hm_port(identity_service,
  File "lib/charm/openstack/api_crud.py", line 386, in toggle_hm_port
    nc.update_port(port['id'], {'port': {'admin_state_up': enabled}})
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/neutronclient/v2_0/client.py", line 810, in update_port
    return self._update_resource(self.port_path % (port), body=body,
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/neutronclient/v2_0/client.py", line 2424, in _update_resource
    return self.put(path, **kwargs)
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/neutronclient/v2_0/client.py", line 362, in put
    return self.retry_request("PUT", action, body=body,
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/neutronclient/v2_0/client.py", line 330, in retry_request
    return self.do_request(method, action, body=body,
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/neutronclient/v2_0/client.py", line 281, in do_request
    resp, replybody = self.httpclient.do_request(action, method, body=body,
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/neutronclient/client.py", line 340, in do_request
    return self.request(url, method, **kwargs)
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/neutronclient/client.py", line 328, in request
    resp = super(SessionClient, self).request(*args, **kwargs)
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/keystoneauth1/adapter.py", line 257, in request
    return self.session.request(url, method, **kwargs)
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/keystoneauth1/session.py", line 921, in request
    resp = send(**kwargs)
  File "/var/lib/juju/agents/unit-octavia-1/.venv/lib/python3.8/site-packages/keystoneauth1/session.py", line 1028, in _send_request
    raise exceptions.ConnectFailure(msg)
keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to https://172.17.105.16:9696/v2.0/ports/687aa3f0-4970-4913-bde6-c0e52f21ffd7: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Frode Nordahl (fnordahl)
Changed in charm-octavia:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
John George (jog) wrote :

SolutionsQA has run into this bug a couple times. In the latest case Octavia tries to connect to the neutron-api unit while a relation changed hook is being handled, which results in 'Remote end closed connection without response'. The neutron hook completes within a couple seconds. However, by this time the Octavia charm had already raised a juju error.

SolutionsQA does not run with hook retries enabled, as we want to catch instances such as this, where the charm should retry.

Here is a link to the test run. The test artifacts are available by follow the link at the bottom of the page.

https://solutions.qa.canonical.com/openstack/testRun/34d3a781-986d-4772-a2f3-10ff49e3f5bd

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.