openstack-loadbalancer hook failed: "loadbalancer-relation-changed"

[Bas de Bruijne]

In testrun https://solutions.qa.canonical.com/testruns/testRun/72b29a95-7cdd-47f6-a028-073989abd4e7 which is openstack yoga on jammy, openstack-loadbalancer fails with the following logs:

```
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 1595, in _run
    result = run(args, **kwargs)
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('/var/lib/juju/tools/unit-openstack-loadbalancer-1/network-get', 'public', '--format=json')' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 153, in <module>
    main(OpenstackLoadbalancerCharm)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/main.py", line 431, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/main.py", line 142, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/interface_openstack_loadbalancer/loadbalancer.py", line 213, in _on_relation_changed
    self.on.lb_requested.emit()
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 150, in _process_lb_requests
    self._send_loadbalancer_response()
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 115, in _send_loadbalancer_response
    for binding, vips in self._get_space_vip_mapping().items():
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 104, in _get_space_vip_mapping
    for binding_name, subnets in self._get_binding_subnet_map().items():
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/./src/charm.py", line 94, in _get_binding_subnet_map
    network = self.model.get_binding(binding_name).network
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 556, in network
    self._network = Network(self._backend.network_get(self.name, self._relation_id))
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 1851, in network_get
    return self._run(*cmd, return_output=True, use_json=True)
  File "/var/lib/juju/agents/unit-openstack-loadbalancer-1/charm/venv/ops/model.py", line 1597, in _run
    raise ModelError(e.stderr)
ops.model.ModelError: b'ERROR no network config found for binding "public"\n'
```

This could be related to LP: #1991552 which we are seeing a lot on OpenStack on jammy.

Crashdumps and configs can be found here:
https://oil-jenkins.canonical.com/artifacts/72b29a95-7cdd-47f6-a028-073989abd4e7/index.html

[Bas de Bruijne (basdbruijne)]

I'm also seeing the same symptom but with a different traceback in https://solutions.qa.canonical.com/v2/testruns/161701b7-1956-4bdd-9a38-8930d98a7c76

```
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 153, in <module>
    main(OpenstackLoadbalancerCharm)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/main.py", line 431, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/main.py", line 142, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/interface_openstack_loadbalancer/loadbalancer.py", line 213, in _on_relation_changed
    self.on.lb_requested.emit()
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 283, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 743, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/ops/framework.py", line 790, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 149, in _process_lb_requests
    self._configure_haproxy()
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 146, in _configure_haproxy
    _render_configs()
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/charmhelpers/core/host.py", line 778, in wrapped_f
    return restart_on_change_helper(
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/charmhelpers/core/host.py", line 864, in restart_on_change_helper
    _post_restart_on_change_helper(checksums,
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/venv/charmhelpers/core/host.py", line 934, in _post_restart_on_change_helper
    restart_functions[service_name](service_name)
  File "/var/lib/juju/agents/unit-ceph-dashboard-loadbalancer-0/charm/./src/charm.py", line 33, in reload_service
    subprocess.check_call(['systemctl', 'reload', service_name])
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['systemctl', 'reload', 'haproxy']' returned non-zero exit status 1.
```

[Bas de Bruijne (basdbruijne)]

This is still a relevant bug, subscribing Jeffrey for follow-ups