sunbeam uses the wrong ip address when joining cluster

[Andre Ruiz]

I have a machine with more than one IP. It's a cloud baremetal machine from equinix and they do this weird networking config where you get a private /31 ip and also a public ip on the main interface.

So I'm completely ignoring those ips and creating two more vlans, that are actually level 2 vlans across my cluster. First one is the management/everything and the second one is unconfigured to be used by OVN later.

Just to be more clear:

bond0 - has two ips, private/31 and public/31. I'm ignoring those.
bond0.1001 - has one ip on the 10.0.1.0/24 network
bond0.1002 - ha no IPs, will use network 10.0.2.0/24 later for Floating IPs.

When bootstrapping sunbeam, I answer that the control plane should use the network from bond0.1001 interface.

The bootstrap works, but later when I want to add more nodes and I do an cluster add command, it generates a token that contains the first private ip (which is different from the network I want to use) (I know this because I base64-decoded the token and saw the IP in the json output.

I believe sunbeam should be using the network I requested on node joins (the other IP could be firewalled or have some other limitation).

Actually, I suspect that the wrong IP is also being used when telling microk8s to cluster itself, and even microceph too. Althoug these ones I could not be completely sure.

[Andre Ruiz (andre-ruiz)]

Just confirming that the hostname does resolve back to the desired ips and is added uniquely to /etc/hosts, and it still uses the wrong ip (in case hostname resolution play any role here).