Comment 1 for bug 2067247

Some more context from https://oil-jenkins.canonical.com/artifacts/66675a8b-8d56-477f-9e4a-5731c88135c2/generated/generated/sunbeam/juju_debug_log.txt

It looks like we might be calling `ceph config set` prematurely before the ceph.conf is fully populated

unit-microceph-1: 12:15:42 ERROR unit.microceph/1.juju-log peers:1: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-microceph-1/charm/./src/charm.py", line 343, in <module>
    main(MicroCephCharm)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/main.py", line 544, in main
    manager.run()
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/main.py", line 520, in run
    self._emit()
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/main.py", line 509, in _emit
    _emit_charm_event(self.charm, self.dispatcher.event_name)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/main.py", line 143, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/framework.py", line 350, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/framework.py", line 849, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/framework.py", line 939, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-microceph-1/charm/src/relation_handlers.py", line 261, in on_changed
    self._rel_changed_nonldr(event)
  File "/var/lib/juju/agents/unit-microceph-1/charm/src/relation_handlers.py", line 254, in _rel_changed_nonldr
    self.on.node_added.emit(**event_args)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/framework.py", line 350, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/framework.py", line 849, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-microceph-1/charm/venv/ops/framework.py", line 939, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-microceph-1/charm/src/relation_handlers.py", line 325, in _on_node_added
    self.callback_f(event)
  File "/var/lib/juju/agents/unit-microceph-1/charm/./src/charm.py", line 113, in configure_charm
    self.configure_ceph(event)
  File "/var/lib/juju/agents/unit-microceph-1/charm/./src/charm.py", line 339, in configure_ceph
    raise e
  File "/var/lib/juju/agents/unit-microceph-1/charm/./src/charm.py", line 327, in configure_ceph
    microceph.set_pool_size("", str(default_rf))
  File "/var/lib/juju/agents/unit-microceph-1/charm/src/microceph.py", line 251, in set_pool_size
    _run_cmd(cmd)
  File "/var/lib/juju/agents/unit-microceph-1/charm/src/microceph.py", line 44, in _run_cmd
    raise e
  File "/var/lib/juju/agents/unit-microceph-1/charm/src/microceph.py", line 39, in _run_cmd
    process = subprocess.run(cmd, capture_output=True, text=True, check=True, timeout=180)
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sudo', 'microceph', 'pool', 'set-rf', '--size', '1', '']' returned non-zero exit status 1.
unit-microceph-1: 12:15:43 ERROR juju.worker.uniter.operation hook "peers-relation-changed" (via hook dispatching script: dispatch) failed: exit status 1
unit-microceph-1: 12:15:43 INFO juju.worker.uniter awaiting error resolution for "relation-changed" hook
unit-microceph-1: 12:15:48 INFO juju.worker.uniter awaiting error resolution for "relation-changed" hook
unit-microceph-1: 12:15:48 INFO unit.microceph/1.juju-log peers:1: Entering guarded section: 'Bootstrapping'
unit-microceph-1: 12:15:48 INFO unit.microceph/1.juju-log peers:1: Setting active status
unit-microceph-1: 12:15:48 INFO unit.microceph/1.juju-log peers:1: Completed guarded section fully: 'Bootstrapping'
unit-microceph-1: 12:15:48 ERROR unit.microceph/1.juju-log peers:1: Failed executing cmd: ['sudo', 'microceph', 'pool', 'set-rf', '--size', '1', ''], error: Error: failed setting replication factor: failed to set pool size default: Failed to run: ceph config set global osd_pool_default_size 1: exit status 1 (Error initializing cluster client: ObjectNotFound('RADOS object not found (error calling conf_read_file)'))