juju - reports incorrect ip address for k8s tigera enterprise deployments

[Adrian Flynn]

Internal bare metal.
juju version 3.3.1-genericlinux-amd64

For a charmed k8s deployment that uses tigera enterprise and dual TOR configuration with BGP the juju status reports the incorrect ip address for each node.

In dual tor bgp config the k8s ip address is on the loopback interface and sits behind two addresses on physical nics. Example - for a node we might have

tor1 <--> nic1 - 10.33.1.13
                           +---- bgp -->lo 10.33.5.13 # This is the k8s public ip address
tor2 <--> nic2 - 10.33.2.13

juju status reports 10.33.1.13 as the public ip address. The addresses on nic1 and nic2 are purely for bgp and physical redundancy. There are no reachable services on these addresses.

For accuracy juju status should report the 10.33.5.13 on the loopback interface as the public ip address of the node.

[Joseph Phillips (manadart)]

Setting a preferred public address for status display is a wart that we're working out of Juju for the upcoming version 4.0.

Juju sets it based on the first set of discovered addresses from the machine, I think based on a sort that falls back to being lexicographical. Subsequent updates from on-machine discovery won't change that address if there isn't a better candidate.

[Adrian Flynn (flynna)]

Thanks Joseph

Maybe some way to tell Juju which device or network to get the address from. Much like Tigera Enterprise Charm has a value

nic_autodetection_regex: "lo"

get the ip address from the loopback. Ignores 127.0.0.1. Though this will break if there is more than 1 additional address on the loopback. Could use a combination of

nic_autodetection_regex: "lo" # use the address on the "lo" that
public_address_regex: 10.33. # matches the regex. If more than 1 match use the first and user needs to fix this misconfig. If no match then we have a misconfig - report "missing".

or just

nic_autodetection_regex: "enp3s0f0" # use the address on this interface

or just

public_address_regex: 10.33. # use the first address that matches this regex

[John A Meinel (jameinel)]

Is the issue that this is impacting charms relating to each other, or is this only an issue that `juju status` is reporting only 1 address in tabular form.
You can also do `juju status --format=yaml` to see the full list of addresses for a node.

[Adrian Flynn (flynna)]

The issue is simply that 'juju status' is reporting the wrong public address. It appears that juju is reporting the first ip address on the first physical interface that it finds and "assuming" this is the public ip address. This is an unsafe assumption. It will be wrong in so many cases where nodes have more than 1 ip address. In our case our nodes have 4 ip addresses and the k8s public ip is on the "lo" as per the dual tor BGP configuration used by Charmed Tigera Enterprise.

I have had a look at 'juju status --format=yaml' and can see the full list of addresses for nodes. The first address is what juju is reporting and this is wrong.

In summary the main point of this "bug" is the "juju status" is reporting the incorrect address in the tabular output.