Ceph-mon fails deployment looking for IP address

[Andrew]

unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.juju-log Invalid upgrade path from xenial. Only trusty is currently supported
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed Traceback (most recent call last):
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/config-changed", line 709, in <module>
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed hooks.execute(sys.argv)
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/charmhelpers/core/hookenv.py", line 715, in execute
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed self._hooks[hook_name]()
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/charmhelpers/contrib/hardening/harden.py", line 79, in _harden_inner2
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed return f(*args, **kwargs)
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/config-changed", line 346, in config_changed
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed log('Monitor hosts are ' + repr(get_mon_hosts()))
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/config-changed", line 388, in get_mon_hosts
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed addr = get_public_addr()
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/charmhelpers/core/hookenv.py", line 72, in wrapper
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed res = func(*args, **kwargs)
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/utils.py", line 85, in get_public_addr
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed return get_network_addrs('ceph-public-network')[0]
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed File "/var/lib/juju/agents/unit-ceph-mon-2/charm/hooks/utils.py", line 140, in get_network_addrs
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed raise Exception(msg)
unit-ceph-mon-2: 17:21:44 INFO unit.ceph-mon/2.config-changed Exception: Could not find an address on any of '[u'10.115.0.0/16']' - resolve this error to retry
unit-ceph-mon-2: 17:21:44 ERROR juju.worker.uniter.operation hook "config-changed" failed: exit status 1

In the advanced config of the charm I put in

CEPH-MON
ceph-public-network = 10.115.0.0/16
ceph-cluster-network = 10.130.0.0/16
Monitor-secret = AQDwFf5XLiNALhAA4peWleWjyPFVJs7Hon2Lqw==

so not sure why it has the ip addresses it has??

Trying to find out why

If I ssh into the container if has two ip addresses assigned

eth0 Link encap:Ethernet HWaddr 00:16:3e:97:a9:51
          inet addr:10.135.0.3 Bcast:10.135.255.255 Mask:255.255.0.0
          inet6 addr: fe80::216:3eff:fe97:a951/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:20529 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18084 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:121810552 (121.8 MB) TX bytes:1276532 (1.2 MB)

eth1 Link encap:Ethernet HWaddr 00:16:3e:20:c5:b9
          inet addr:10.100.0.8 Bcast:10.100.255.255 Mask:255.255.0.0
          inet6 addr: fe80::216:3eff:fe20:c5b9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:4556 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5255 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:23144109 (23.1 MB) TX bytes:940176 (940.1 KB)

neither on the network defined

[Andrew (amoss-6)]

It looks like the containers are inheriting the configuration from the MAAS node that is chosen at random.

If conjure-up can use the spaces and can support targeting this would work

otherwise I will need to configure every node for every OS network - otherwise the random selection will cause issues

[Adam Stokes (adam-stokes)]

Thanks Andrew,

We need to look into what it will take to have conjure-up honor those spaces from MAAS

[Adam Stokes (adam-stokes)]

This will be addressed in our upcoming placement work.

[Adam Stokes (adam-stokes)]

Moving this bug to https://github.com/conjure-up/conjure-up/issues/591