containerd stuck with "Container runtime not available"
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Containerd Subordinate Charm |
Fix Released
|
High
|
Joseph Borg |
Bug Description
1/5 of our containerd units got blocked with the message "Container runtime not available".
This output is in all of the kubernetes unit logs but containerd/1 is the only unit that gets stuck in the blocked state.
---
The pod-logs contain the following:
pod-logs/
E0301 02:30:05.021003 1 manager.go:101] Error in scraping containers from kubelet_
---
Here's a link to the artifacts/
https:/
summary: |
- containerd failing to scrape containers from kubelet_summary + containerd stuck with "Container runtime not available" |
Changed in charm-containerd: | |
assignee: | nobody → Joseph Borg (joeborg) |
Changed in charm-containerd: | |
importance: | Undecided → High |
status: | Confirmed → In Progress |
tags: | added: review-needed |
Changed in charm-containerd: | |
status: | In Progress → Fix Committed |
tags: | removed: review-needed |
Changed in charm-containerd: | |
milestone: | none → 1.17+ck3 |
milestone: | 1.17+ck3 → 1.18 |
Changed in charm-containerd: | |
status: | Fix Committed → Fix Released |
containerd/1 is stuck in blocked status with "Container runtime not available". Looks like a status bug - I'm not seeing anything in the logs to indicate that containerd is actually failing, and it's pretty obvious looking at the charm code[1] that the status isn't being re-checked during the update-status hook like it should be.
[1]: https:/ /github. com/charmed- kubernetes/ charm-container d/blob/ d38de1f01dd0a4d 686c09300627442 28a332af38/ reactive/ containerd. py#L326- L331