juju private-address returns wrong IP

[Haw Loeung]

Hi,

Nagios checks etc. uses results returned from private-address. Seems on hosts where there's multiple, this can sometimes be wrong:

| ubuntu@comet:~$ sudo juju-run infra/0 "unit-get private-address"
| 10.220.254.1

Is it returning the first?

| ubuntu@comet:~$ sudo juju-run infra/0 "ip -4 addr"
| 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
| inet 127.0.0.1/8 scope host lo
| valid_lft forever preferred_lft forever
| 8: br-eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
| inet 10.220.254.1/24 brd 10.220.254.255 scope global br-eno2
| valid_lft forever preferred_lft forever
| 9: br-eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
| inet 10.222.128.23/22 brd 10.222.131.255 scope global br-eno1
| valid_lft forever preferred_lft forever

Obviously wrong and should be 10.222.128.23:

| ubuntu@comet:~$ sudo juju-run infra/0 "ip route get 8.8.8.8"
| 8.8.8.8 via 10.222.128.1 dev br-eno1 src 10.222.128.23
| cache

This is from an environment deployed using Juju 2.2.3.

Thanks,

Haw

[John A Meinel (jameinel)]

How are we to tell what address is "wrong" vs "right" when all addresses
are private. (10.220.254.1 is certainly an RFC 1918 address.)

Charms are recommended to move away from "unit-get private-address" to
"network-get" (see docs for more complete information), which allows juju
to know what space/binding/etc the charm is trying to talk about, and
allows us to have enough context to know which address of multiple possible
addresses is correct.

On Mon, Dec 4, 2017 at 6:45 AM, Haw Loeung <email address hidden>
wrote:

> Public bug reported:
>
> Hi,
>
> Nagios checks etc. uses results returned from private-address. Seems on
> hosts where there's multiple, this can sometimes be wrong:
>
> | ubuntu@comet:~$ sudo juju-run infra/0 "unit-get private-address"
> | 10.220.254.1
>
> Is it returning the first?
>
> | ubuntu@comet:~$ sudo juju-run infra/0 "ip -4 addr"
> | 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
> group default qlen 1
> | inet 127.0.0.1/8 scope host lo
> | valid_lft forever preferred_lft forever
> | 8: br-eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> state UP group default qlen 1000
> | inet 10.220.254.1/24 brd 10.220.254.255 scope global br-eno2
> | valid_lft forever preferred_lft forever
> | 9: br-eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> state UP group default qlen 1000
> | inet 10.222.128.23/22 brd 10.222.131.255 scope global br-eno1
> | valid_lft forever preferred_lft forever
>
> Obviously wrong and should be 10.222.128.23:
>
> | ubuntu@comet:~$ sudo juju-run infra/0 "ip route get 8.8.8.8"
> | 8.8.8.8 via 10.222.128.1 dev br-eno1 src 10.222.128.23
> | cache
>
> This is from an environment deployed using Juju 2.2.3.
>
>
> Thanks,
>
> Haw
>
> ** Affects: juju
> Importance: Undecided
> Status: New
>
> --
> You received this bug notification because you are subscribed to juju.
> Matching subscriptions: juju bugs
> https://bugs.launchpad.net/bugs/1736050
>
> Title:
> juju private-address returns wrong IP
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/1736050/+subscriptions
>

[Haw Loeung (hloeung)]

I would think it would return the source IP used for the default route (as per above it would be '10.222.128.23').

I'll target this bug to the NRPE charm then. I think it could use 'network-get' as advised to get the ingress-addresses for the configured "nagios_master" or from the Nagios relation.

[Haw Loeung (hloeung)]

Even with 'network-get', the IP address returned is clearly wrong:

| ubuntu@comet:~$ sudo juju-run nrpe-physical/19 "network-get nrpe-external-master"
| info:
| - macaddress: 2c:44:fd:92:61:11
| interfacename: eno2
| addresses:
| - address: 10.220.254.1
| cidr: 10.220.254.0/24
| - macaddress: 2c:44:fd:92:61:11
| interfacename: br-eno2
| addresses:
| - address: 10.220.254.1
| cidr: 10.220.254.0/24

Again, on this host, all outbound connections use the 10.222.128.23 address:

| ubuntu@comet:~$ ip route show | grep '^default'
| default via 10.222.128.1 dev br-eno1 onlink

| ubuntu@comet:~$ ip -o addr show dev br-eno1
| 9: br-eno1 inet 10.222.128.23/22 brd 10.222.131.255 scope global br-eno1\ valid_lft forever preferred_lft forever
| 9: br-eno1 inet6 fe80::2e44:fdff:fe92:6110/64 scope link \ valid_lft forever preferred_lft forever

[John A Meinel (jameinel)]

Is this a case of "without setting a specific binding, we end up selecting an arbitrary IP that you feel is incorrect"?
I suppose it could be "when guessing what IP to use, prefer the IP associated with the default gateway".
I think its still guessing, rather than doing something the user explicitly requested. It could be a more educated guess, but I'd like to make sure we have a way for the user to be specific before we do lots of work on heuristics that will always be wrong some of the time.

[Haw Loeung (hloeung)]

It's still unclear to me what "binding" means and is used by Juju. I think it's the endpoint of the provided relation (per above, "network-get nrpe-external-master").

I think you're right that it could be "when guessing what IP to use, prefer the IP associated with the default gateway". It's better than what it is right now of "when guessing what IP to use, use *the IP of the first non-loopback interface*".

[Tim Penhey (thumper)]

If there are no spaces, nor bindings specified, juju should report the address that would be used to contact the API servers.

This has been done in python by opening a UDP socket to the server, and looking at the client IP address, then closing the socket. Since it is a UDP socket, there is no traffic, just a file handle that is released quickly.