FYI, failing on the latest run @ https://review.opendev.org/#/c/717068/ with artifacts @ https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_func_full/openstack/charm-ceph-rbd-mirror/717068/5/5855/index.html

ceph-rbd-mirror-b/0*        waiting   executing  20       172.17.110.19             Waiting for pools to be created
ceph-rbd-mirror/0*          blocked   idle       13       172.17.110.10             Unit is ready (Pools ERROR (1) OK (1) )


...



2020-05-20 23:11:45 INFO juju-log Pools: "{'cinder-ceph': {'applications': {'rbd': {}}, 'parameters': {'pg_num': '32', 'size': '3'}, 'quota': {}}, 'device_health_metrics': {'applications': {'mgr_devicehealth': {}}, 'parameters': {'pg_num': '1', 'size': '3'}, 'quota': {}}, 'glance': {'applications': {'rbd': {}}, 'parameters': {'pg_num': '128', 'size': '3'}, 'quota': {}}}"
2020-05-20 23:11:45 DEBUG update-status creating /var/lib/ceph/rbd-mirror/ceph.client.rbd-mirror.juju-ba0618-zaza-89869a5cef84-13.keyring
2020-05-20 23:11:45 DEBUG update-status added entity client.rbd-mirror.juju-ba0618-zaza-89869a5cef84-13 auth(key=AQDbscVeezgELxAAl5eTogKSgfJ3dykmZ1McTA==)
2020-05-20 23:11:45 DEBUG juju-log Changing permissions on existing content: 33184 -> 416
2020-05-20 23:11:46 DEBUG juju-log Changing permissions on existing content: 33184 -> 416
2020-05-20 23:11:46 INFO juju-log Invoking reactive handler: reactive/ceph_rbd_mirror_handlers.py:93:configure_pools
2020-05-20 23:11:46 INFO juju-log Invoking reactive handler: hooks/relations/ceph-rbd-mirror/requires.py:51:joined:ceph-local
2020-05-20 23:11:46 INFO juju-log Invoking reactive handler: hooks/relations/ceph-rbd-mirror/requires.py:51:joined:ceph-remote
2020-05-20 23:11:46 INFO juju-log Invoking reactive handler: hooks/relations/ceph-rbd-mirror/requires.py:55:changed:ceph-local
2020-05-20 23:11:46 INFO juju-log Invoking reactive handler: hooks/relations/ceph-rbd-mirror/requires.py:55:changed:ceph-remote
2020-05-20 23:11:46 INFO juju-log Invoking reactive handler: hooks/relations/tls-certificates/requires.py:109:broken:certificates
2020-05-20 23:11:46 DEBUG juju-log Running _assess_status()
2020-05-20 23:11:47 DEBUG juju-log mirror_pools_summary = "{'pool_health': defaultdict(<class 'int'>, {'ERROR': 1, 'OK': 1}), 'image_states': defaultdict(<class 'int'>, {})}"


...


2020-05-20T22:40:53.600+0000 7fabe6ae8500  0 ceph version 15.2.1 (9fd2f65f91d9246fae2c841a6222d34d121680ee) octopus (stable), process rbd-mirror, pid 18606
2020-05-20T22:40:53.624+0000 7fabe6ae8500  1 mgrc service_daemon_register rbd-mirror.4917 metadata {arch=x86_64,ceph_release=octopus,ceph_version=ceph version 15.2.1 (9fd2f65f91d9246fae2c841a6222d34d121680ee) octopus (stable),ceph_version_short=15.2.1,cpu=Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz,distro=ubuntu,distro_description=Ubuntu 20.04 LTS,distro_version=20.04,hostname=juju-ba0618-zaza-89869a5cef84-13,id=juju-ba0618-zaza-89869a5cef84-13,instance_id=4917,kernel_description=#33-Ubuntu SMP Wed Apr 29 14:32:27 UTC 2020,kernel_version=5.4.0-29-generic,mem_swap_kb=0,mem_total_kb=2035420,os=Linux}
2020-05-20T22:41:53.724+0000 7fabe6ae8500 -1 rbd::mirror::PoolReplayer: 0x55f357b49200 init: error accessing remote pool glance: (2) No such file or directory
2020-05-20T22:42:23.736+0000 7fabe6ae8500 -1 rbd::mirror::Mirror: 0x55f356f2b560 update_pool_replayers: restarting failed pool replayer for uuid: b4ddab6a-b215-437c-ab89-67e5bccdd818 cluster: remote client: client.rbd-mirror.juju-ba0618-zaza-89869a5cef84-13
2020-05-20T22:42:23.848+0000 7fabe6ae8500 -1 rbd::mirror::PoolReplayer: 0x55f357b49200 init: failed to initialize remote pool poller: (2) No such file or directory
2020-05-20T22:42:53.868+0000 7fabe6ae8500 -1 rbd::mirror::Mirror: 0x55f356f2b560 update_pool_replayers: restarting failed pool replayer for uuid: b4ddab6a-b215-437c-ab89-67e5bccdd818 cluster: remote client: client.rbd-mirror.juju-ba0618-zaza-89869a5cef84-13
2020-05-20T22:48:24.227+0000 7fabe6ae8500 -1 rbd::mirror::PoolReplayer: 0x55f3596e0600 init: failed to initialize remote pool poller: (2) No such file or directory
2020-05-20T22:48:54.246+0000 7fabe6ae8500 -1 rbd::mirror::Mirror: 0x55f356f2b560 update_pool_replayers: restarting failed pool replayer for uuid: a10634e8-c8a3-47b6-8b6d-5d7f19dccf26 cluster: remote client: client.rbd-mirror.juju-ba0618-zaza-89869a5cef84-13