charm only accepts dynamic ipv6 addresses

[Federico Bosi]

The charm currently accepts only dynamic addresses.
My environment has only static ones meaning it would refuse use them.
I tried setting up a small cluster (3 osds, 3 mons) with a slightly modified version of the charm where I replaced all the get_ipv6_addr() calls with get_ipv6_addr(dynamic_only=False).
The machines have a single global static ipv6 address.
Everything works correctly.

I faced a couple problems in a mixed (static / dynamic addresses) environment but only with the MONs, so that's for another issue.

I'll upload a change review with these changes.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-ceph-mon/+/917764

[Federico Bosi (rhxto)]

Both the ceph-osd and ceph-mon changes have a binding check; here's why:
mon host = [ipv6-prefix:142:0:1:23] [ipv6-prefix:142:0:1:4] [ipv6-prefix:142:0:1:5]
fsid = bf39d814-0709-11ef-8fd5-05b2bca2c749

log to syslog = false
err to syslog = false
clog to syslog = false
debug osd = 1/5

ms_bind_ipv6 = true
ms_bind_ipv4 = false

public network =
cluster network =
public addr = ipv6-prefix:142:1:c3dc:fd7
cluster addr = ipv6-prefix:142:1:c3dc:fd7

The mons are stuck in "Bootstrapping MON cluster" state because they connect to one of the addresses in the array, but none of them is actually listening for one.
The mon host are addresses provided by juju which we get with get_{public,cluster}_addr, while the public and cluster addr lines are filled with get_ipv6_addr[0].
If the machine has multiple addresses (ipv6-prefix:142:1:c3dc:fd7 and ipv6-prefix:142:0:1:23) it will bind ceph to the first address it finds ignoring what juju assigned it.
With the binding check, we first get what juju assigned us and use it if we find it in the local addresses, otherwise default to the previous behavior.

After the change the config looks correctly like this:
mon host = [ipv6-prefix:142:0:1:23] [ipv6-prefix:142:0:1:4] [ipv6-prefix:142:0:1:5]
fsid = bf39d814-0709-11ef-8fd5-05b2bca2c749

log to syslog = false
err to syslog = false
clog to syslog = false
debug osd = 1/5

ms_bind_ipv6 = true
ms_bind_ipv4 = false

public network =
cluster network =
public addr = ipv6-prefix:142:0:1:23
cluster addr = ipv6-prefix:142:0:1:23