ceph-radosgw and ceph-mon relation failed to complete, radosgw not started

[Francis Ginther]

This was encountered with an openstack autopilot deploy with landscape:
 - landscape: 16.08~bzr10497+jenkins2987-2
 - maas: 2.0.0+bzr5189-0ubuntu1
 - juju: 2.0-beta16-xenial-amd64
 - Note: Landscape had juju-2.0 and maas-2.0 feature flags enabled.

The ceph radosgw service never started. Log file was empty. Autopilot deployment was stuck trying to sync glance images (which depends on a functioning radosgw service).

At 18:33 (log time), I manually removed the ceph-mon/ceph-radosgw relation. At 18:35, I manually added the relation back (with juju-2.0 commands directly on the autopilot juju model). Shortly after the add, ceph radosgw was running, the image sync finished and the deploy was a 'success'.

I could find nothing in ceph logs that looked like an obvious (at least to my untrained eye) failure. It did look like it was waiting on information from the mon relation, which is what led to the manual recovery steps.

I've attached ceph* and landscape-server logs which includes the manual recover steps (starting at 18:33).