Comment 1 for bug 1884292

kube-state-metrics errors are a red herring.

test_dns_provider timed out after 15 minutes. The test waits for coredns pods to be removed, but it looks like a coredns pod is stuck terminating. The node that hosts the pod is in Unknown status.

Kubelet on that node is failing to communicate with kube-apiserver due to "use of closed network connection" errors:

E0619 04:57:09.410568 14620 server.go:269] Authorization error (user=system:kube-apiserver, verb=get, resource=nodes, subresource=metrics)%!(EXTRA *url.Error=Post https://172.31.39.158:443/apis/authorization.k8s.io/v1/subjectaccessreviews: write tcp 172.31.39.53:57246->172.31.39.158:443: use of closed network connection)
E0619 04:57:11.237701 14620 kubelet_node_status.go:402] Error updating node status, will retry: error getting node "ip-172-31-39-53.ec2.internal": an error on the server ("") has prevented the request from succeeding (get nodes ip-172-31-39-53.ec2.internal)
E0619 04:57:11.237742 14620 kubelet_node_status.go:389] Unable to update node status: update node status exceeds retry count

This is a known issue in Kubernetes/Golang:

https://github.com/kubernetes/kubernetes/issues/87615
https://github.com/golang/go/issues/39750