Loadbalancer for kube API endpoint not properly configured

[Jeff Hillman]

Kubernetes 1.19
Focal + Ussuri openstack
latest/stable charms

When applying the appropriate relations for octavia to take the place of the kubeapi-load-balancer charm, a loadbalancer is being created, but it does not get a proper FIP in front of it.

Workloads (ServiceType LoadBalancer) work fine, however using octavia for the kube API endpoint isn't creating proper load balancers.

Trying different combinations of setting the lb-floating-network, and lb-subnet have different outcomes, but none work.

Simply setting the lb-floating-network to the name/UUID of the FIP network in openstack and NOT setting the lb-subnet will cause an octavia loadbalancer to be created, however no FIP is assigned to it. instead the .kube/config has an internal tenant IP address.

Setting neither will cause the same behavior as above.

Setting lb-floating-network to the FIP network name/UUID and setting lb-subnet to the FIP subnet will cause a load balancer to be created, but with ONLY a FIP IP, and no private tenant net IP, so it is ultimately unreachable.

Lastly, in the oddest scenario, setting the lb-floating-network to the FIP network name/UUID and setting the lb-subnet, does create an octavia load balancer, but doing an `openstack loadbalancer show <uuid>` doesn't show a FIP. However, the kubeconfig provided by k8s-master has a FIP. When running `openstack floating ip list | grep <FIP in kubeconfig>` show a tenant IP address that isn't in use by kubernetes. in fact, going to /root/.kube/config on a k8s master shows a tenant address that is different from both the `openstack loadbalancer show <uuid>` tenant address and is different from the address associated with a floating IP address.

There is no workaround for this. Attaching openstack-integrator log.

[Jeff Hillman (jhillman)]

subscribed field-high

[Chris Sanders (chris.sanders)]

@Jeff can we get a bundle to see how things are configured. You can share it via private-fileshare if it has sensitive information.

[Nobuto Murata (nobuto)]

> doing an `openstack loadbalancer show <uuid>` doesn't show a FIP. However, the kubeconfig provided by k8s-master has a FIP.

This is expected. `loadbalancer show` outputs a vip of the loadbalancer and it's usually on a private network, and the vip will be associated with an floating IP for external access.

One thing I noticed when I was testing it was that the floating IP network must be specified before deploying and relating openstack-integrator with k8s-master. Looks like the config is used only when creating the loadbalancer.
https://github.com/juju-solutions/charm-openstack-integrator/blob/244b8d533f38f18fc6908dc8f9327c9b767a47cc/lib/charms/layer/openstack.py#L149-L163

[Jeff Hillman (jhillman)]

Bundle provided

[Jeff Hillman (jhillman)]

@nobuto, agreed. any change requires a full re-deploy. the charm states it is managing load balancers when you make a config change, but nothing actually happens.

[Jeff Hillman (jhillman)]

unsubscribed field-high

[Nobuto Murata (nobuto)]

Looks like the root cause was the same with:
https://bugs.launchpad.net/charm-openstack-integrator/+bug/1905008