On a deployment testing juju 2.9.37 with Jammy Yoga there was a single barbican (primary) and barbican-vault (subordinate) unit that failed to get the network address for the internal space:
unit-barbican-2: 07:11:30 ERROR unit.barbican/2.juju-log shared-db:59: Hook error:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-barbican-2/.venv/lib/python3.10/site-packages/charmhelpers/core/hookenv.py", line 1374, in network_get_primary_address
response = subprocess.check_output(
File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['network-get', '--primary-address', 'shared-db']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/juju/agents/unit-barbican-2/.venv/lib/python3.10/site-packages/charms/reactive/__init__.py", line 74, in main
bus.dispatch(restricted=restricted_mode)
File "/var/lib/juju/agents/unit-barbican-2/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 390, in dispatch
_invoke(other_handlers)
File "/var/lib/juju/agents/unit-barbican-2/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 359, in _invoke
handler.invoke()
File "/var/lib/juju/agents/unit-barbican-2/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 181, in invoke
self._action(*args)
File "/var/lib/juju/agents/unit-barbican-2/charm/reactive/layer_openstack_api.py", line 33, in default_setup_database
database.configure(**db)
File "/var/lib/juju/agents/unit-barbican-2/charm/hooks/relations/mysql-shared/requires.py", line 56, in configure
hostname = hookenv.network_get_primary_address(
File "/var/lib/juju/agents/unit-barbican-2/.venv/lib/python3.10/site-packages/charmhelpers/core/hookenv.py", line 1180, in inner_translate_exc2
return f(*args, **kwargs)
File "/var/lib/juju/agents/unit-barbican-2/.venv/lib/python3.10/site-packages/charmhelpers/core/hookenv.py", line 1379, in network_get_primary_address
raise NoNetworkBinding("No network binding for {}"
charmhelpers.core.hookenv.NoNetworkBinding: No network binding for shared-db
but looking at the juju show-unit for that application the endpoint has a set address, and the binding is set:
- relation-id: 59
endpoint: shared-db
related-endpoint: shared-db
application-data: {}
related-units:
barbican-mysql-router/0:
in-scope: true
data:
egress-subnets: 192.168.33.188/32
ingress-address: 192.168.33.188
private-address: 192.168.33.188
barbican-mysql-router/1:
in-scope: true
data:
egress-subnets: 192.168.33.171/32
ingress-address: 192.168.33.171
private-address: 192.168.33.171
barbican-mysql-router/2:
in-scope: true
data:
ingress-address: 10.246.65.100
private-address: 10.246.65.100
The testrun can be found at:
https://solutions.qa.canonical.com/v2/testruns/48b95cac-f227-4474-8935-6d44d6bdc9e7/
and the crashdump can be found at:
https://oil-jenkins.canonical.com/artifacts/48b95cac-f227-4474-8935-6d44d6bdc9e7/generated/generated/openstack/juju-crashdump-openstack-2022-11-10-07.13.14.tar.gz
for each unit inside the crashdump there are logs for juju show-status-log, juju show-machine, and juju show-unit. where the bindings and address can be seen.
When network-get is run, it is not guaranteed that Juju has collated all the address info - link layer devices, instance ips etc - for the host machine. Therefore the api call may return 0 network info records. This results in the charmhelper "NoNetworkBinding" error.
The Juju agent on the host machine gathers the link layer device info, and the controller polls for cloud allocated host instance addresses, so the network info does eventually become available.
The charm needs to be resilient to the fact that the address info may not immediately be known. It needs to take account of that error and try again a short time later. It needs to set its status to "Waiting" with a suitable message. And if after several attempts the address info is not available, it should set its status to "Blocked".
If this error happens, can you retry the hook - it should run ok the next time, assuming the address info has been populated. This will confirm the above theory and indicate that the charm needs to be fixed.