juju selects wrong address for API

[Felipe Reyes]

[Environment]

Distributor ID: Ubuntu
Description: Ubuntu 14.10
Release: 14.10
Codename: utopic

juju-core:
  Installed: 1.24.0-0ubuntu1~14.10.1~juju1
  Candidate: 1.24.0-0ubuntu1~14.10.1~juju1
  Version table:
 *** 1.24.0-0ubuntu1~14.10.1~juju1 0
        500 http://ppa.launchpad.net/juju/stable/ubuntu/ utopic/main amd64 Packages
        100 /var/lib/dpkg/status
     1.20.10-0ubuntu1 0
        500 http://cl.archive.ubuntu.com/ubuntu/ utopic/universe amd64 Packages

environments.yaml:

environments:
    local:
        type: local
        default-series: trusty
        lxc-clone: true
        container: kvm
        network-bridge: br0
        apt-http-proxy: http://192.168.0.103:3142

These are IPs my system has:
$ ip a | egrep -e '^ inet '
    inet 127.0.0.1/8 scope host lo
    inet 192.168.0.103/24 brd 192.168.0.255 scope global br0
    inet 10.0.3.1/24 brd 10.0.3.255 scope global lxcbr0
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
    inet 192.168.123.1/24 brd 192.168.123.255 scope global virbr1
    inet 192.168.10.1/24 brd 192.168.10.255 scope global virbr2
    inet 10.172.192.118/18 brd 10.172.255.255 scope global tun0

Steps to reproduce:

$ juju bootstrap
$ juju add-machine

The /var/lib/juju/agents/machine-1/agents.conf contains:
# format 1.18
tag: machine-1
datadir: /var/lib/juju
[...]
stateaddresses:
- 192.168.0.103:37017
[...]
apiaddresses:
- 10.172.192.118:17070
[...]
values:
  AGENT_SERVICE_NAME: jujud-machine-1
  CONTAINER_TYPE: kvm
  NAMESPACE: freyes-local
  PROVIDER_TYPE: local
  SECURE_STATESERVER_CONNECTION: "true"

So the launched VM cannot connect to the API, the VM's IP address is in the subnet 192.168.0.0/24 so it should be connecting to 192.168.0.103 just like the state address.

[Darryl Weaver (dweaver)]

This problem also surfaces in a MAAS environment with multiple subnets on the bootstrap node.

[Amir Sabbaghi (asaba90)]

I have the same problem. LXC is installed on juju bootstrap node and other machines are trying to connect to 10.0.3.1:17070
"juju status" shows that all machines and agents are down and I am unable to upgrade my charms.
Is there any workaround for this bug?
I have purged LXC from juju bootstrap node and changed agent.conf from both bootstrap node and the unit with no luck. It seems that the IP has been kept in juju-db.

[Cheryl Jennings (cherylj)]

This problem stems from juju selecting all IPs on the host as possible connection points for API connections.

A workaround for the local provicer, which may not be feasible for everyone, is to ensure that the subnet used by the bridge is the "lowest" IP on the host system, since juju will sort the available IPs and take the lowest one when connecting.

[Felipe Reyes (freyes)]

In comment #2, Darryl indicates that MAAS provider also suffers from this, so it would be good if juju had an option to force an specific subnet/ip to be used for the API, so if you're in an environment where you can't change IPs and juju is not selecting the correct one, you are not left dead in the water.

Would this be possible?

[Dimiter Naydenov (dimitern)]

Can you provide the output of "ip routes show", "brctl show", and "brctl showmacs br0" ?

Also, can you see if specifying "bootstrap-ip: 192.168.0.103" in env.yaml for that "local" environment and retrying will solve your case? If it does not work, please retry again with --debug to bootstrap and "logging-config: '<root>=TRACE'" in envs.yaml, then attach the machine-0.log (scrubbed of secrets).

[Launchpad Janitor (janitor)]

[Expired for juju-core because there has been no activity for 60 days.]