Comment 7 for bug 1851869

I have a deployment up on MAAS (icarus virt nodes, using /dev/vdb for OSDs) and while it's a lot faster:

2019-11-13 15:08:40 [INFO] Validate that a volume created through Cinder is mirrored.
2019-11-13 15:08:40 [INFO] ...
2019-11-13 15:08:44 [INFO] Using keystone API V3 (or later) for overcloud auth
2019-11-13 15:08:45 [WARNING] Version 2 is deprecated, use alternative version 3 instead.
2019-11-13 15:08:57 [INFO] creating
2019-11-13 15:09:01 [INFO] creating
2019-11-13 15:09:03 [INFO] downloading
2019-11-13 15:09:08 [INFO] downloading
2019-11-13 15:09:16 [INFO] downloading
2019-11-13 15:09:32 [INFO] downloading
2019-11-13 15:10:04 [INFO] available

it still hangs after this point with ceph-rbd-mirror-b/0 Pools WARNING.

$ sudo rbd --id rbd-mirror.juju-c2ffd9-6-lxd-1 mirror pool status --verbose cinder-ceph
health: WARNING
images: 1 total
    1 unknown

volume-61605fa3-59a4-42ae-81f9-863e0fb05598:
  global_id: f397d9e3-2a20-487c-b70f-54cd1e207f8f
  state: down+replaying
  description: replaying, master_position=[object_number=65, tag_tid=2, entry_tid=1], mirror_position=[object_number=65, tag_tid=2, entry_tid=1], entries_behind_master=0
  last_update: 2019-11-13 15:18:23

/var/log/ceph/ceph*.log from ceph-rbd-mirror-b/0 has:

2019-11-13 15:01:12.918 7f069f9fb300 1 mgrc service_daemon_register rbd-mirror.4813 metadata {arch=x86_64,ceph_release=nautilus,ceph_version=ceph version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be) nautilus (stable),ceph_version_short=14.2.2,cpu=Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz,distro=ubuntu,distro_description=Ubuntu 18.04.3 LTS,distro_version=18.04,hostname=juju-c2ffd9-6-lxd-1,id=juju-c2ffd9-6-lxd-1,instance_id=4813,kernel_description=#78-Ubuntu SMP Wed Nov 6 11:30:24 UTC 2019,kernel_version=4.15.0-69-generic,mem_swap_kb=4194300,mem_total_kb=16424724,os=Linux}
2019-11-13 15:09:12.651 7f05ea7fc700 0 rbd::mirror::ImageReplayer: 0x7f067401fff0 [2/2d19d024-6352-4634-8699-5b83e82b1f8d] handle_shut_down: remote image no longer exists: scheduling deletion
2019-11-13 15:09:13.147 7f05ea7fc700 0 rbd::mirror::ImageReplayer: 0x7f067401fff0 [2/2d19d024-6352-4634-8699-5b83e82b1f8d] handle_shut_down: mirror image no longer exists

/var/log/ceph/ceph*.log from ceph-rbd-mirror/0 has:
2019-11-13 15:01:12.971 7f35e43db300 1 mgrc service_daemon_register rbd-mirror.5028 metadata {arch=x86_64,ceph_release=nautilus,ceph_version=ceph version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be) nautilus (stable),ceph_version_short=14.2.2,cpu=Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz,distro=ubuntu,distro_description=Ubuntu 18.04.3 LTS,distro_version=18.04,hostname=juju-c2ffd9-3-lxd-1,id=juju-c2ffd9-3-lxd-1,instance_id=5028,kernel_description=#78-Ubuntu SMP Wed Nov 6 11:30:24 UTC 2019,kernel_version=4.15.0-69-generic,mem_swap_kb=4194300,mem_total_kb=16424724,os=Linux}
2019-11-13 15:01:43.059 7f35e43db300 -1 rbd::mirror::PoolReplayer: 0x55c43a3dc500 init: error accessing remote pool glance: (2) No such file or directory
2019-11-13 15:02:13.064 7f35e43db300 -1 rbd::mirror::Mirror: 0x55c43a270b70 update_pool_replayers: restarting failed pool replayer for uuid: 88c9d11b-e713-414a-abcd-b5d1a2c042db cluster: remote client: client.rbd-mirror.juju-c2ffd9-3-lxd-1
2019-11-13 15:02:13.172 7f35c6267700 0 rbd::mirror::PoolReplayer: 0x55c43a3dc500 handle_init_remote_pool_watcher: remote peer does not have mirroring configured
2019-11-13 15:04:13.253 7f35e43db300 -1 rbd::mirror::PoolReplayer: 0x7f3594027d60 init: error accessing remote pool cinder-ceph: (2) No such file or directory
2019-11-13 15:04:43.261 7f35e43db300 -1 rbd::mirror::Mirror: 0x55c43a270b70 update_pool_replayers: restarting failed pool replayer for uuid: 55da9ea0-a636-4caa-850e-a15070d69626 cluster: remote client: client.rbd-mirror.juju-c2ffd9-3-lxd-1
2019-11-13 15:04:43.381 7f35c6267700 0 rbd::mirror::PoolReplayer: 0x7f3594027d60 handle_init_remote_pool_watcher: remote peer does not have mirroring configured
2019-11-13 15:09:13.348 7f352f7fe700 0 rbd::mirror::ImageReplayer: 0x7f35b8021230 [2/2d19d024-6352-4634-8699-5b83e82b1f8d] handle_shut_down: mirror image no longer exists