Comment 6 for bug 2061836

Revision history for this message
Federico Bosi (rhxto) wrote (last edit ):

Both the ceph-osd and ceph-mon changes have a binding check; here's why:
mon host = [ipv6-prefix:142:0:1:23] [ipv6-prefix:142:0:1:4] [ipv6-prefix:142:0:1:5]
fsid = bf39d814-0709-11ef-8fd5-05b2bca2c749

log to syslog = false
err to syslog = false
clog to syslog = false
debug osd = 1/5

ms_bind_ipv6 = true
ms_bind_ipv4 = false

public network =
cluster network =
public addr = ipv6-prefix:142:1:c3dc:fd7
cluster addr = ipv6-prefix:142:1:c3dc:fd7

The mons are stuck in "Bootstrapping MON cluster" state because they connect to one of the addresses in the array, but none of them is actually listening for one.
The mon host are addresses provided by juju which we get with get_{public,cluster}_addr, while the public and cluster addr lines are filled with get_ipv6_addr[0].
If the machine has multiple addresses (ipv6-prefix:142:1:c3dc:fd7 and ipv6-prefix:142:0:1:23) it will bind ceph to the first address it finds ignoring what juju assigned it.
With the binding check, we first get what juju assigned us and use it if we find it in the local addresses, otherwise default to the previous behavior.

After the change the config looks correctly like this:
mon host = [ipv6-prefix:142:0:1:23] [ipv6-prefix:142:0:1:4] [ipv6-prefix:142:0:1:5]
fsid = bf39d814-0709-11ef-8fd5-05b2bca2c749

log to syslog = false
err to syslog = false
clog to syslog = false
debug osd = 1/5

ms_bind_ipv6 = true
ms_bind_ipv4 = false

public network =
cluster network =
public addr = ipv6-prefix:142:0:1:23
cluster addr = ipv6-prefix:142:0:1:23