Comment 2 for bug 1970645

Revision history for this message
Alexandru Dimofte (adimofte) wrote :

I found this pods in Evicted state:
openstack cinder-volume-bb64cb587-lvqxh ● 0/0 0 Evicted n/a controller-1 172m │
│ openstack fm-rest-api-649cd5b99d-8f9v6 ● 0/0 0 Evicted n/a controller-1 172m │
│ openstack ingress-6b65fb7fc9-g6cf5 ● 0/0 0 Evicted n/a controller-1 172m │
│ openstack keystone-api-66b87d555b-xgxlm ● 0/0 0 Evicted n/a controller-1 172m │
│ openstack mariadb-ingress-bcd8fb475-r8tj7 ● 0/0 0 Evicted n/a controller-1 3h36m │
│ openstack mariadb-ingress-bcd8fb475-vpszs ● 0/0 0 Evicted n/a controller-0 3h36m │
│ openstack nova-api-proxy-78c6447cb-69txr ● 0/0 0 Evicted n/a controller-1 176m │
│ openstack nova-scheduler-7b6fd68499-5hwwh ● 0/0 0 Evicted n/a controller-1 176m │
│ openstack placement-api-68668d8dd5-mttwl ● 0/0 0 Evicted n/a controller-1 176m │

Describing the pods I found this kind of messages:
 Message: The node was low on resource: ephemeral-storage. Container ingress was using 112Ki, which exceeds its request of 0.
ex:
 Events: │
│ Type Reason Age From Message │
│ ---- ------ ---- ---- ------- │
│ Normal Scheduled 3h41m default-scheduler Successfully assigned openstack/mariadb-ingress-bcd8fb475-vpszs to controller-0 │
│ Normal AddedInterface 3h41m multus Add eth0 [172.16.192.99/32] │
│ Normal Pulled 3h41m kubelet, controller-0 Container image "registry.local:9001/quay.io/airshipit/kubernetes-entrypoint:v1.0.0" already present on machine │
│ Normal Created 3h41m kubelet, controller-0 Created container init │
│ Normal Started 3h41m kubelet, controller-0 Started container init │
│ Normal Pulled 3h41m kubelet, controller-0 Container image "registry.local:9001/k8s.gcr.io/ingress-nginx/controller:v0.42.0" already present on machine │
│ Normal Created 3h41m kubelet, controller-0 Created container ingress │
│ Normal Started 3h41m kubelet, controller-0 Started container ingress │
│ Warning Unhealthy 3h39m (x16 over 3h41m) kubelet, controller-0 Readiness probe failed: dial tcp 172.16.192.99:3306: connect: connection refused │
│ Normal RELOAD 3h39m (x2 over 3h41m) nginx-ingress-controller NGINX reload triggered due to a change in configuration │
│ Warning Evicted 178m kubelet, controller-0 The node was low on resource: ephemeral-storage. Container ingress was using 112Ki, which exceeds its request of 0. │
│ Normal Killing 178m kubelet, controller-0 Stopping container ingress