Version check is needed before injecting DevicePlugins feature gate
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Kubernetes Control Plane Charm |
Fix Released
|
High
|
Chris Johnston | ||
Kubernetes Worker Charm |
Fix Released
|
High
|
Chris Johnston |
Bug Description
Charm: kubernetes-worker
Channel: 1.28/stable
Problem:
Newly deployed kubnernetes-worker unit/node is not starting up correctly, kubelet is crashing
with this error:
---
"command failed" err="failed to set feature gates from initial flags-based config: unrecognized feature gate: DevicePlugins"
snap.kubelet.
snap.kubelet.
---
The cause for this appears to be the following
1. The "DevicePlugins" feature gate has been incorporated into k8s as of 1.28 as a generally available feature
https:/
2. However, the kubernetes-worker charm still inserts an invalid DevicePlugins gate in the kubelet config, which is leading to the kubelet daemon failing to start
3. This configuration is passed here
https:/
The only check for this config to be added is for it to be a gpu enabled worker node. No check of the version is being made, hence the errors when trying to start kubelet on the node.
The charm needs to check that
----
A) The worker node has GPU enabled
B) The k8s version is at a version where the feature gate would be needed to be passed
summary: |
- Version checked is needed before injecting DevicePlugins feature gate + Version check is needed before injecting DevicePlugins feature gate |
Changed in charm-kubernetes-worker: | |
status: | New → Confirmed |
Changed in charm-kubernetes-master: | |
status: | New → Confirmed |
Changed in charm-kubernetes-worker: | |
milestone: | none → 1.29 |
Changed in charm-kubernetes-master: | |
milestone: | none → 1.29 |
tags: |
added: backport-needed removed: review-needed |
tags: | removed: backport-needed |
Changed in charm-kubernetes-master: | |
status: | Confirmed → Fix Committed |
Changed in charm-kubernetes-worker: | |
status: | Confirmed → Fix Committed |
Changed in charm-kubernetes-master: | |
assignee: | nobody → Chris Johnston (cjohnston) |
Changed in charm-kubernetes-worker: | |
assignee: | nobody → Chris Johnston (cjohnston) |
Changed in charm-kubernetes-master: | |
status: | Fix Committed → In Progress |
Changed in charm-kubernetes-worker: | |
status: | Fix Committed → In Progress |
Changed in charm-kubernetes-master: | |
importance: | Undecided → High |
Changed in charm-kubernetes-worker: | |
importance: | Undecided → High |
Changed in charm-kubernetes-worker: | |
milestone: | 1.29 → 1.28+ck2 |
Changed in charm-kubernetes-master: | |
milestone: | 1.29 → 1.28+ck2 |
Changed in charm-kubernetes-master: | |
status: | In Progress → Fix Committed |
Changed in charm-kubernetes-worker: | |
status: | In Progress → Fix Committed |
tags: | added: backport-needed |
tags: | removed: backport-needed |
Changed in charm-kubernetes-master: | |
status: | Fix Committed → Fix Released |
Changed in charm-kubernetes-worker: | |
status: | Fix Committed → Fix Released |
https:/ /github. com/charmed- kubernetes/ layer-kubernete s-common/ pull/47