setting floating-network-id has no impact on the subnet used for loadbalancer VIPs

[Michael Skalka]

Using latest openstack-integrator charm (102) on Focal Ussuri openstack and CDK 1.20

I set the floating-network-id to our external network ID:

ubuntu@production-cpe-21921d77-8c0c-444a-a37a-3998839499ee:~/project$ juju config openstack-integrator floating-network-id
c9f28ce7-edf8-4cde-bf4c-db522bf8dc86

ubuntu@production-cpe-21921d77-8c0c-444a-a37a-3998839499ee:~/project$ openstack network list
+--------------------------------------+-------------------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+-------------------+--------------------------------------+
| c9f28ce7-edf8-4cde-bf4c-db522bf8dc86 | provider-2734-net | e5b1a0b3-c34a-425e-a5be-e46dbe60e512 |
| ed4f3272-2639-4732-9a35-4246566989fe | lb-mgmt-net | f6e0183a-5f1e-489b-9ea4-f8a970ae3cbc |
| f7b047fa-0779-45ca-9416-921bc5d9207c | ubuntu-net | 65e5a313-d52b-46f3-b7a3-f6d5fe0ab1b2 |
+--------------------------------------+-------------------+--------------------------------------+

ubuntu@production-cpe-21921d77-8c0c-444a-a37a-3998839499ee:~/project$ openstack network show c9f28ce7-edf8-4cde-bf4c-db522bf8dc86
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2021-03-22T15:30:30Z |
| description | |
| dns_domain | |
| id | c9f28ce7-edf8-4cde-bf4c-db522bf8dc86 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| mtu | 1500 |
| name | provider-2734-net |
| port_security_enabled | True |
| project_id | 678e60a33ad14f34811b287825ac4016 |
| provider:network_type | vlan |
| provider:physical_network | physnet1 |
| provider:segmentation_id | 2734 |
| qos_policy_id | None |
| revision_number | 2 |
| router:external | External |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | e5b1a0b3-c34a-425e-a5be-e46dbe60e512 |
| tags | |
| updated_at | 2021-03-22T15:30:31Z |
+---------------------------+--------------------------------------+

ubuntu@production-cpe-21921d77-8c0c-444a-a37a-3998839499ee:~/project$ openstack subnet show e5b1a0b3-c34a-425e-a5be-e46dbe60e512
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| allocation_pools | 10.244.32.2-10.244.32.254 |
| cidr | 10.244.32.0/21 |
| created_at | 2021-03-22T15:30:31Z |
| description | |
| dns_nameservers | |
| dns_publish_fixed_ip | None |
| enable_dhcp | False |
| gateway_ip | 10.244.32.1 |
| host_routes | |
| id | e5b1a0b3-c34a-425e-a5be-e46dbe60e512 |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | provider-2734-subnet |
| network_id | c9f28ce7-edf8-4cde-bf4c-db522bf8dc86 |
| prefix_length | None |
| project_id | 678e60a33ad14f34811b287825ac4016 |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
| tags | |
| updated_at | 2021-03-22T15:30:31Z |
+----------------------+--------------------------------------+

However when I try to bootstrap into kubernetes using juju I get an address outside of that network (on the ubuntu-net in this case) and the bootstrap fails:

ubuntu@production-cpe-21921d77-8c0c-444a-a37a-3998839499ee:~/project$ openstack loadbalancer list
+--------------------------------------+--------------------------------------------------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+
| id | name | project_id | vip_address | provisioning_status | operating_status | provider |
+--------------------------------------+--------------------------------------------------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+
| 594afb97-b8f4-4c5b-b2f2-26e22e6d91c8 | kube_service_kubernetes-pbpdwh0lgfgtsv2cu7fvvzwn7tk1fjfq_controller-foundations-k8s_controller-service | 678e60a33ad14f34811b287825ac4016 | 172.16.0.163 | PENDING_CREATE | OFFLINE | amphora |
+--------------------------------------+--------------------------------------------------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+

[Michael Skalka (mskalka)]

Further, setting both the floating-network-id and subnet-id to appropriate values doesn't have seem to have changed the result:

ubuntu@production-cpe-21921d77-8c0c-444a-a37a-3998839499ee:~/project$ juju config -m foundation-openstack:kubernetes openstack-integrator subnet-id
e5b1a0b3-c34a-425e-a5be-e46dbe60e512

ubuntu@production-cpe-21921d77-8c0c-444a-a37a-3998839499ee:~/project$ openstack loadbalancer list
+--------------------------------------+--------------------------------------------------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+
| id | name | project_id | vip_address | provisioning_status | operating_status | provider |
+--------------------------------------+--------------------------------------------------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+
| 9f322137-5c8e-4e3d-bbf4-93c71dff910b | kube_service_kubernetes-pbpdwh0lgfgtsv2cu7fvvzwn7tk1fjfq_controller-foundations-k8s_controller-service | 678e60a33ad14f34811b287825ac4016 | 172.16.0.210 | PENDING_CREATE | OFFLINE | amphora |
+--------------------------------------+--------------------------------------------------------------------------------------------------------+----------------------------------+--------------+---------------------+------------------+----------+

[Cory Johns (johnsca)]

This might be related to https://bugs.launchpad.net/charm-kubernetes-master/+bug/1892164 in which case the following can be used as a workaround:

kubectl -n kube-system rollout restart ds/openstack-cloud-controller-manager

[George Kraft (cynerva)]

Michael confirmed in another channel that the rollout restart fixed this for him. Marking this as a duplicate of 1892164.