containerd pause container image was delete during operation on EKS

[DingGGu]

https://cloud-images.ubuntu.com/aws-eks/

Since Ubuntu EKS 1.22 AMI, containerd has become the default container runtime.

When the node receives DiskPressure, kubelet gc the unused image. I faced pause container image has been removed during operation. After the pause container is deleted, the node can no longer create pods. (related with containerd does not support ecr native image pull)

kubelet has a pod-infra-container-image flag, which prevents gc to a pause container image.

Edit the bootstrap.sh file so that the pod-infra-container-image flag is set in all container runtimes.

I am using bootstrap.sh by modifying it.

mv /etc/eks/bootstrap.sh /etc/eks/bootstrap.sh.orig
sed "s,container-runtime-endpoint=unix:///run/containerd/containerd.sock,container-runtime-endpoint=unix:///run/containerd/containerd.sock pod-infra-container-image=" \$PAUSE_CONTAINER",g" < /etc/eks/bootstrap.sh.orig > /etc/eks/bootstrap.sh
chmod +x /etc/eks/bootstrap.sh

[Tomáš Virtus (virtustom)]

New AMIs were released yesterday with serial 20220419.1. Pause container is now configured via --pod-infra-container-image kubelet option for both dockershim and containerd container runtimes. Can you please try it?

[DingGGu (dinggggu)]

I confirm that flag was set correctly.

# snap get kubelet-eks
Key Value
...
container-runtime remote
container-runtime-endpoint unix:///run/containerd/containerd.sock
pod-infra-container-image 602401143452.dkr.ecr.ap-northeast-2.amazonaws.com/eks/pause:3.1-eksbuild.1
...