ceph-mon in error state due to hook failed: "leader-elected"

Bug #2046566 reported by Konstantinos Kaskavelis
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceph Monitor Charm
New
Undecided
Unassigned

Bug Description

Solutions QA has a failed jammy baremetal run where ceph-mon charm stayed in an error state with the message: hook failed: "leader-elected"

From the logs:

2023-12-14 16:53:31 DEBUG unit.ceph-mon/0.juju-log server.go:325 Emitting Juju event leader_elected.
2023-12-14 16:53:31 ERROR unit.ceph-mon/0.juju-log server.go:325 Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/model.py", line 2955, in _run
    result = subprocess.run(args, **kwargs) # type: ignore
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('/var/lib/juju/tools/unit-ceph-mon-0/network-get', 'metrics-endpoint', '-r', '16', '--format=json')' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/./src/charm.py", line 317, in <module>
    main(CephMonCharm)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/main.py", line 436, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/main.py", line 144, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/framework.py", line 351, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/framework.py", line 853, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/framework.py", line 942, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/lib/charms/prometheus_k8s/v0/prometheus_scrape.py", line 1555, in _set_scrape_job_spec
    self._set_unit_ip(event)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/lib/charms/prometheus_k8s/v0/prometheus_scrape.py", line 1586, in _set_unit_ip
    unit_ip = str(self._charm.model.get_binding(relation).network.bind_address)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/model.py", line 929, in network
    self._network = self._network_get(self.name, self._relation_id)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/model.py", line 922, in _network_get
    return Network(self._backend.network_get(name, relation_id))
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/model.py", line 3240, in network_get
    network = self._run(*cmd, return_output=True, use_json=True)
  File "/var/lib/juju/agents/unit-ceph-mon-0/charm/venv/ops/model.py", line 2957, in _run
    raise ModelError(e.stderr) from e
ops.model.ModelError: ERROR no network config found for binding "metrics-endpoint"

2023-12-14 16:53:32 ERROR juju.worker.uniter.operation runhook.go:180 hook "leader-elected" (via hook dispatching script: dispatch) failed: exit status 1

Failed run:
https://solutions.qa.canonical.com/testruns/aab54177-8719-41be-b0ce-0ddd77e3b3be

Logs https://oil-jenkins.canonical.com/artifacts/aab54177-8719-41be-b0ce-0ddd77e3b3be/index.html (available for two weeks)

Tags: cdo-qa
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.