Multus DaemonSet should be restarted during cidr expansion

Bug #2020704 reported by Adam Dyess
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Multus Charm
Triaged
Medium
Unassigned

Bug Description

test_service_cidr_expansion [1] fails when deployed with kube-ovn and multus

This test attempts to expand the cidr range by updating config service-cidr the kubernetes-control-plane. The control-plane reacts to this config changed by restarting a known label `cdk-restart-on-ca-change=true` [2] of DaemonSets, Deployments, and such. However -- the multus charm has deployed a DaemonSet not in this white list.

Effect:
New pods scheduled after the change, will fail to contact the API service because it's address will have changed and cannot proceed out of the Pending State.
(eg -- 10.152.183.1 will no longer be the API service, but rather 10.152.182.1)

Workaround:
Manually restart the Daemonset `kube-multus-ds` in the `kube-system` namespace
* This can hang as well -- since the container install-multus-binary in the kube-multus-ds pods has trouble replacing running multus binaries
* In order to replace those binaries, run `killall multus` and `rm /opt/cni/bin/multus`

[1]: https://github.com/charmed-kubernetes/jenkins/blob/b003fe89fad19f5ad7a73e33d14d7b466fc5c52d/jobs/integration/validation.py#L1037
[2]: https://github.com/charmed-kubernetes/charm-kubernetes-control-plane/blob/30f3e6628577d09a5a1e44cc947b0f6d38917e90/reactive/kubernetes_control_plane.py#L2578

Revision history for this message
Adam Dyess (addyess) wrote :

Skip the cidr expansion test [1]

[1] https://github.com/charmed-kubernetes/jenkins/pull/1327

George Kraft (cynerva)
Changed in charm-multus:
importance: Undecided → Medium
status: New → Triaged
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.