The charm should support converting non-multisite to multisite

[Andrey Grebennikov]

Currently the charm only supports deploying ceph-radosgw in a multisite scenario from scratch.
When trying to convert single-site to multisite the charm will fail at the step of creating respective user via radosgw-admin.

I was able to make it working if Before enabling the multisite relation I perform the following in the master site:

radosgw-admin --id=rgw.juju-81d572-0-lxd-0 realm create --rgw-realm replicated1 --default
radosgw-admin --id=rgw.juju-81d572-0-lxd-0 zonegroup create --rgw-zonegroup us1 --default --master --endpoints http://192.168.122.70:80
radosgw-admin --id=rgw.juju-81d572-0-lxd-0 zone modify --rgw-zonegroup us1 --default --master --endpoints http://192.168.122.70:80 --rgw-zone zone1

After this I added the relation and the charm did all the work.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-ceph-radosgw/+/848739
Committed: https://opendev.org/openstack/charm-ceph-radosgw/commit/44fee84d4d61e8ccda78dfb9afd68e0048686d4a
Submitter: "Zuul (22348)"
Branch: master

commit 44fee84d4d61e8ccda78dfb9afd68e0048686d4a
Author: utkarshbhatthere <email address hidden>
Date: Tue Jul 5 17:27:36 2022 +0530

    Adds support for migration to multi-site system.

    1.) Currently multi-site can only be configured when system is being
    deployed from scratch, migration works by renaming the existing
    Zone/Zonegroups (Z/ZG) to Juju config values on primary site before
    secondary site pulls the realm data and then rename and configure
    secondary Zone accordingly.

    During migration:
    2.) If multiple Z/ZG not matching the config values are present at
    primary site, the leader unit will block and prompt use of
    'force-enable-multisite' which renames and configures selected Z/ZG
    according to multisite config values.

    3.) If the site being added as a secondary already contain Buckets,
    the unit will block and prompt the operator to purge all such Buckets
    before proceeding.

    Closes-Bug: #1959837
    Change-Id: I01a4c1c4551c797f0a32951dfbde8a1a4126c2d6
    func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/840