In some situations the Octavia charm will end up in error state during deploy.
Log excerpt:
2018-12-20 12:00:01 INFO juju-log identity-service:57: Reactive main running for hook identity-service-relation-changed
2018-12-20 12:00:04 INFO juju-log identity-service:57: Initializing Leadership Layer (is leader)
2018-12-20 12:00:04 INFO juju-log identity-service:57: Invoking reactive handler: hooks/relations/keystone/requires.py:62:changed
2018-12-20 12:00:09 INFO juju-log identity-service:57: Invoking reactive handler: reactive/octavia_handlers.py:48:setup_neutron_lbaas_proxy
2018-12-20 12:00:09 INFO juju-log identity-service:57: Invoking reactive handler: reactive/octavia_handlers.py:69:setup_hm_port
2018-12-20 12:00:11 DEBUG identity-service-relation-changed Failed to discover available identity versions when contacting http://172.16.122.20:35357/. Attempting to parse version from URL.
2018-12-20 12:00:11 ERROR juju-log identity-service:57: Hook error:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 138, in _do_create_plugin
authenticated=False)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 610, in get_discovery
authenticated=authenticated)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/discover.py", line 1442, in get_discovery
disc = Discover(session, url, authenticated=authenticated)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/discover.py", line 526, in __init__
authenticated=authenticated)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/discover.py", line 101, in get_version_data
resp = session.get(url, headers=headers, authenticated=authenticated)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/session.py", line 1011, in get
return self.request(url, 'GET', **kwargs)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/session.py", line 869, in request
raise exceptions.from_response(resp, method, url)
keystoneauth1.exceptions.http.ServiceUnavailable: Service Unavailable (HTTP 503)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/__init__.py", line 73, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-octavia-0/charm/reactive/octavia_handlers.py", line 87, in setup_hm_port
octavia_charm):
File "lib/charm/openstack/api_crud.py", line 276, in setup_hm_port
octavia_charm.local_address)
File "lib/charm/openstack/api_crud.py", line 154, in get_hm_port
resp = nc.list_networks(tags='charm-octavia')
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 809, in list_networks
**_params)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 369, in list
for r in self._pagination(collection, path, **params):
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination
res = self.get(path, params=params)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 354, in get
headers=headers, params=params)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 331, in retry_request
headers=headers, params=params)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 282, in do_request
headers=headers)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/client.py", line 342, in do_request
self._check_uri_length(url)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/client.py", line 335, in _check_uri_length
uri_len = len(self.endpoint_url) + len(url)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/client.py", line 349, in endpoint_url
return self.get_endpoint()
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 247, in get_endpoint
return self.session.get_endpoint(auth or self.auth, **kwargs)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/session.py", line 1113, in get_endpoint
return auth.get_endpoint(self, **kwargs)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 380, in get_endpoint
allow_version_hack=allow_version_hack, **kwargs)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data
service_catalog = self.get_access(session).service_catalog
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
self.auth_ref = self.get_auth_ref(session)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 206, in get_auth_ref
self._plugin = self._do_create_plugin(session)
File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 161, in _do_create_plugin
'auth_url is correct. %s' % e)
keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. Service Unavailable (HTTP 503)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed Traceback (most recent call last):
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 138, in _do_create_plugin
2018-12-20 12:00:11 DEBUG identity-service-relation-changed authenticated=False)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 610, in get_discovery
2018-12-20 12:00:11 DEBUG identity-service-relation-changed authenticated=authenticated)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/discover.py", line 1442, in get_discovery
2018-12-20 12:00:11 DEBUG identity-service-relation-changed disc = Discover(session, url, authenticated=authenticated)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/discover.py", line 526, in __init__
2018-12-20 12:00:11 DEBUG identity-service-relation-changed authenticated=authenticated)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/discover.py", line 101, in get_version_data
2018-12-20 12:00:11 DEBUG identity-service-relation-changed resp = session.get(url, headers=headers, authenticated=authenticated)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/session.py", line 1011, in get
2018-12-20 12:00:11 DEBUG identity-service-relation-changed return self.request(url, 'GET', **kwargs)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/session.py", line 869, in request
2018-12-20 12:00:11 DEBUG identity-service-relation-changed raise exceptions.from_response(resp, method, url)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed keystoneauth1.exceptions.http.ServiceUnavailable: Service Unavailable (HTTP 503)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed
2018-12-20 12:00:11 DEBUG identity-service-relation-changed During handling of the above exception, another exception occurred:
2018-12-20 12:00:11 DEBUG identity-service-relation-changed
2018-12-20 12:00:11 DEBUG identity-service-relation-changed Traceback (most recent call last):
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/charm/hooks/identity-service-relation-changed", line 22, in <module>
2018-12-20 12:00:11 DEBUG identity-service-relation-changed main()
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/__init__.py", line 73, in main
2018-12-20 12:00:11 DEBUG identity-service-relation-changed bus.dispatch(restricted=restricted_mode)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 390, in dispatch
2018-12-20 12:00:11 DEBUG identity-service-relation-changed _invoke(other_handlers)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 359, in _invoke
2018-12-20 12:00:11 DEBUG identity-service-relation-changed handler.invoke()
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/charms/reactive/bus.py", line 181, in invoke
2018-12-20 12:00:11 DEBUG identity-service-relation-changed self._action(*args)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/charm/reactive/octavia_handlers.py", line 87, in setup_hm_port
2018-12-20 12:00:11 DEBUG identity-service-relation-changed octavia_charm):
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "lib/charm/openstack/api_crud.py", line 276, in setup_hm_port
2018-12-20 12:00:11 DEBUG identity-service-relation-changed octavia_charm.local_address)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "lib/charm/openstack/api_crud.py", line 154, in get_hm_port
2018-12-20 12:00:11 DEBUG identity-service-relation-changed resp = nc.list_networks(tags='charm-octavia')
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 809, in list_networks
2018-12-20 12:00:11 DEBUG identity-service-relation-changed **_params)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 369, in list
2018-12-20 12:00:11 DEBUG identity-service-relation-changed for r in self._pagination(collection, path, **params):
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination
2018-12-20 12:00:11 DEBUG identity-service-relation-changed res = self.get(path, params=params)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 354, in get
2018-12-20 12:00:11 DEBUG identity-service-relation-changed headers=headers, params=params)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 331, in retry_request
2018-12-20 12:00:11 DEBUG identity-service-relation-changed headers=headers, params=params)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/v2_0/client.py", line 282, in do_request
2018-12-20 12:00:11 DEBUG identity-service-relation-changed headers=headers)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/client.py", line 342, in do_request
2018-12-20 12:00:11 DEBUG identity-service-relation-changed self._check_uri_length(url)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/client.py", line 335, in _check_uri_length
2018-12-20 12:00:11 DEBUG identity-service-relation-changed uri_len = len(self.endpoint_url) + len(url)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/neutronclient/client.py", line 349, in endpoint_url
2018-12-20 12:00:11 DEBUG identity-service-relation-changed return self.get_endpoint()
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 247, in get_endpoint
2018-12-20 12:00:11 DEBUG identity-service-relation-changed return self.session.get_endpoint(auth or self.auth, **kwargs)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/session.py", line 1113, in get_endpoint
2018-12-20 12:00:11 DEBUG identity-service-relation-changed return auth.get_endpoint(self, **kwargs)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 380, in get_endpoint
2018-12-20 12:00:11 DEBUG identity-service-relation-changed allow_version_hack=allow_version_hack, **kwargs)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data
2018-12-20 12:00:11 DEBUG identity-service-relation-changed service_catalog = self.get_access(session).service_catalog
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
2018-12-20 12:00:11 DEBUG identity-service-relation-changed self.auth_ref = self.get_auth_ref(session)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 206, in get_auth_ref
2018-12-20 12:00:11 DEBUG identity-service-relation-changed self._plugin = self._do_create_plugin(session)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed File "/var/lib/juju/agents/unit-octavia-0/.venv/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 161, in _do_create_plugin
2018-12-20 12:00:11 DEBUG identity-service-relation-changed 'auth_url is correct. %s' % e)
2018-12-20 12:00:11 DEBUG identity-service-relation-changed keystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. Service Unavailable (HTTP 503)
2018-12-20 12:00:11 ERROR juju.worker.uniter.operation runhook.go:132 hook "identity-service-relation-changed" failed: exit status 1
Fix proposed to branch: master /review. openstack. org/626575
Review: https:/