Comment 38 for bug 1719436

We just hit this bug today on xenial-ocata.

charm: cs:ceph-mon-22

ceph-mon/0:
        workload-status:
          current: maintenance
          message: Bootstrapping MON cluster
          since: 12 Mar 2018 17:39:56Z
        juju-status:
          current: executing
          message: running mon-relation-changed hook
          since: 12 Mar 2018 17:40:51Z
          version: 2.4-beta1
        leader: true
        machine: 0/lxd/0

juju log:
2018-03-12 17:40:42 DEBUG mon-relation-changed added entity mon. auth auth(auid = 18446744073709551615 key=AQAfuqZacc4AIBAA7U0lTDc6JQci7g1GHOMBWQ== with 0 caps)
2018-03-12 17:40:42 DEBUG mon-relation-changed '/var/lib/ceph/mon/ceph-juju-5fc73a-0-lxd-0' already exists and is not empty: monitor may already exist
2018-03-12 17:40:46 DEBUG mon-relation-changed Traceback (most recent call last):
2018-03-12 17:40:46 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/mon-relation-changed", line 725, in <module>
2018-03-12 17:40:46 DEBUG mon-relation-changed hooks.execute(sys.argv)
2018-03-12 17:40:46 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/charmhelpers/core/hookenv.py", line 800, in execute
2018-03-12 17:40:46 DEBUG mon-relation-changed self._hooks[hook_name]()
2018-03-12 17:40:46 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/mon-relation-changed", line 356, in mon_relation
2018-03-12 17:40:46 DEBUG mon-relation-changed ceph.bootstrap_monitor_cluster(leader_get('monitor-secret'))
2018-03-12 17:40:46 DEBUG mon-relation-changed File "lib/ceph/utils.py", line 1323, in bootstrap_monitor_cluster
2018-03-12 17:40:46 DEBUG mon-relation-changed init_marker)
2018-03-12 17:40:46 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-mon-0/charm/hooks/charmhelpers/core/decorators.py", line 40, in _retry_on_exception_inner_2
2018-03-12 17:40:46 DEBUG mon-relation-changed return f(*args, **kwargs)
2018-03-12 17:40:46 DEBUG mon-relation-changed File "lib/ceph/utils.py", line 1360, in add_keyring_to_ceph
2018-03-12 17:40:46 DEBUG mon-relation-changed osstat = os.stat("/etc/ceph/ceph.client.admin.keyring")
2018-03-12 17:40:46 DEBUG mon-relation-changed FileNotFoundError: [Errno 2] No such file or directory: '/etc/ceph/ceph.client.admin.keyring'
2018-03-12 17:40:46 ERROR juju.worker.uniter.operation runhook.go:114 hook "mon-relation-changed" failed: exit status 1