Deploying cluster with default space and public network binding generates wrong cluster addr config

[Sandor Zeestraten]

# Problem
Deploying a cluster with default space (either via default "" binding or model-config default-space) and a public network binding defined, the charm generates a wrong ceph.conf where `cluster addr` uses the default space. This leads to OSD daemons to bind to the default space as cluster network. This problem exists for both ceph-mon and ceph-osd charms.

Deploying without a default space is not possible with juju as it fails to deploy machines in MAAS with the error message `matching subnets to zones: cannot use space "alpha" as deployment target: no subnets`.

# Bundle
relations:
- - ceph-osd:mon
  - ceph-mon:osd
series: focal
applications:
  ceph-mon:
    charm: cs:ceph-mon-73
    num_units: 3
    constraints: tags=ceph-mon
    bindings:
      "": site2-oam
      public: site2-ceph-public
    options:
      monitor-count: 3
      source: cloud:focal-xena
  ceph-osd:
    charm: cs:ceph-osd-513
    num_units: 3
    constraints: tags=ceph-osd
    options:
      osd-devices: "/dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm"
      source: cloud:focal-xena
    bindings:
      "": site2-oam
      public: site2-ceph-public

# /etc/ceph/ceph.conf
```
public network =
cluster network =
public addr = 172.17.104.20
cluster addr = 10.42.200.22
```

# juju show-application ceph-mon
ceph-mon:
  charm: ceph-mon
  series: focal
  channel: stable
  constraints:
    arch: amd64
    tags:
    - ceph-mon
  principal: true
  exposed: false
  remote: false
  endpoint-bindings:
    "": site2-oam
    admin: site2-oam
    bootstrap-source: site2-oam
    client: site2-oam
    cluster: site2-oam
    dashboard: site2-oam
    mds: site2-oam
    mon: site2-oam
    nrpe-external-master: site2-oam
    osd: site2-oam
    prometheus: site2-oam
    public: site2-ceph-public
    radosgw: site2-oam
    rbd-mirror: site2-oam

# juju show-application ceph-osd
ceph-osd:
  charm: ceph-osd
  series: focal
  channel: stable
  constraints:
    arch: amd64
    tags:
    - ceph-osd
  principal: true
  exposed: false
  remote: false
  endpoint-bindings:
    "": site2-oam
    cluster: site2-oam
    mon: site2-oam
    nrpe-external-master: site2-oam
    public: site2-ceph-public
    secrets-storage: site2-oam

# juju spaces
Name Space ID Subnets
alpha 0
site2-os-public 1 10.42.208.0/23
site2-neutron-external 2 10.42.210.0/23
site2-os-data 3 172.17.100.0/23
site2-os-internal 4 172.17.102.0/23
site2-ceph-public 5 172.17.104.0/23
site2-ceph-cluster 6 172.17.106.0/23
site2-oam 7 10.42.200.0/23
site2-provision 8 10.42.196.0/23
                                  10.42.198.0/23

# versions
juju 2.9.27
ceph-mon rev 73
ceph-osd rev 513