charm only accepts dynamic ipv6 addresses

[Federico Bosi]

The charm currently accepts only dynamic addresses.
My environment has only static ones meaning it would refuse use them.
I tried setting up a small cluster (3 osds, 3 mons) with a slightly modified version of the charm where I replaced all the get_ipv6_addr() calls with get_ipv6_addr(dynamic_only=False).
The machines have a single global static ipv6 address.
Everything works correctly.

I faced a couple problems in a mixed (static / dynamic addresses) environment but only with the MONs, so that's for another issue.

I'll upload a change review with these changes.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-ceph-mon/+/917764

[Federico Bosi (rhxto)]

Both the ceph-osd and ceph-mon changes have a binding check; here's why:
mon host = [ipv6-prefix:142:0:1:23] [ipv6-prefix:142:0:1:4] [ipv6-prefix:142:0:1:5]
fsid = bf39d814-0709-11ef-8fd5-05b2bca2c749

log to syslog = false
err to syslog = false
clog to syslog = false
debug osd = 1/5

ms_bind_ipv6 = true
ms_bind_ipv4 = false

public network =
cluster network =
public addr = ipv6-prefix:142:1:c3dc:fd7
cluster addr = ipv6-prefix:142:1:c3dc:fd7

The mons are stuck in "Bootstrapping MON cluster" state because they connect to one of the addresses in the array, but none of them is actually listening for one.
The mon host are addresses provided by juju which we get with get_{public,cluster}_addr, while the public and cluster addr lines are filled with get_ipv6_addr[0].
If the machine has multiple addresses (ipv6-prefix:142:1:c3dc:fd7 and ipv6-prefix:142:0:1:23) it will bind ceph to the first address it finds ignoring what juju assigned it.
With the binding check, we first get what juju assigned us and use it if we find it in the local addresses, otherwise default to the previous behavior.

After the change the config looks correctly like this:
mon host = [ipv6-prefix:142:0:1:23] [ipv6-prefix:142:0:1:4] [ipv6-prefix:142:0:1:5]
fsid = bf39d814-0709-11ef-8fd5-05b2bca2c749

log to syslog = false
err to syslog = false
clog to syslog = false
debug osd = 1/5

ms_bind_ipv6 = true
ms_bind_ipv4 = false

public network =
cluster network =
public addr = ipv6-prefix:142:0:1:23
cluster addr = ipv6-prefix:142:0:1:23

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-ceph-mon/+/917764
Committed: https://opendev.org/openstack/charm-ceph-mon/commit/5b2f48000bdc24864a2056d9496f5a02c93cf9da
Submitter: "Zuul (22348)"
Branch: master

commit 5b2f48000bdc24864a2056d9496f5a02c93cf9da
Author: Federico Bosi <email address hidden>
Date: Tue Apr 30 18:45:45 2024 +0200

    Allow static ipv6 addresses & binding check

    The charm used to only allow dynamic ipv6 addresses.
    Following some testing (see LP issue) I ensured that it works.

    The charm wouldn't check if the first local address returned
    by get_ipv6_addr was the same one that juju provided for the
    public endpoint. This resulted in the charm binding ceph to
    some random address while correctly using juju's one to connect
    to other mons, which none were listening to.
    We log a warning in case this happens, but default to the
    previous behavior.
    Closes-Bug: #2061836

    Change-Id: I63eefba2714ccc3189f5260012b376c531052b86