I've seen once a sanity lab in the reproduced state. The hello-kitty pod failed to respond to Liveness and Readiness probes.
"Liveness probe failed ..."
"Readiness probe failed ..."
Trying to set up a virtual IPv6 lab i've seen the exact same log for calico-node, didn't write the output somewhere :(.


I tried to reproduce the issue with a newer build the output changed from "Liveness probe failed" to the below. Same behavior for calico-node/hello-kitty depending on default route set on host.


fd00::1 is on a NAT64/DNS64 node.

ansible-playbook
	dns_servers:
	- fd00::1
	external_oam_subnet: fd00::/64
	external_oam_gateway_address: fd00::1
	external_oam_floating_address: fd00::2
	external_oam_node_0_address: fd00::3
	external_oam_node_1_address: fd00::4
	management_subnet: fd01::/64
	management_multicast_subnet: ff08::1:1:0/124
	cluster_host_subnet: fd02::/64
	cluster_pod_subnet: fd03::/64
	cluster_service_subnet: fd04::/112
	docker_no_proxy:
	  - tis-lab-registry.cumulus.wrs.com

Apply hello-kitty:
	kubectl  --kubeconfig /etc/kubernetes/admin.conf apply -f /home/sysadmin/custom_apps/hellokitty.yaml

No default route:
	  Warning  FailedCreatePodSandBox  74s                 kubelet, controller-0  Failed create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "0dbbc088b1baba5581533c73dd
	ff77f1605bfddb5c37a4cc8c7a396beeb183aea" network for pod "hellokitty": NetworkPlugin cni failed to set up pod "hellokitty_default" network: Multus: Err adding pod to network "chain": Multus: error in  
	invoke Conflist add - "chain": error in getting result from AddNetworkList: error getting ClusterInformation: Get https://[fd04::1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: dial  
	tcp [fd04::1]:443: connect: network is unreachable, failed to clean up sandbox container "0dbbc088b1baba5581533c73dff77f1605bfddb5c37a4cc8c7a396beeb183aea" network for pod "hellokitty": NetworkPlugin  
	cni failed to teardown pod "hellokitty_default" network: Multus: error in invoke Conflist Del - "chain": error in getting result from DelNetworkList: error getting ClusterInformation: Get https://[fd00
	4::1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: dial tcp [fd04::1]:443: connect: network is unreachable]
	  Warning  FailedCreatePodSandBox  67s                 kubelet, controller-0  Failed create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "7b94821cda718783348d63b822
	e0c16a170ee4b3841bdfa2e3e90d672042681a3" network for pod "hellokitty": NetworkPlugin cni failed to set up pod "hellokitty_default" network: Multus: Err adding pod to network "chain": Multus: error in  
	invoke Conflist add - "chain": error in getting result from AddNetworkList: error getting ClusterInformation: Get https://[fd04::1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: dial  
	tcp [fd04::1]:443: connect: network is unreachable, failed to clean up sandbox container "7b94821cda718783348d63b82e0c16a170ee4b3841bdfa2e3e90d672042681a3" network for pod "hellokitty": NetworkPlugin  
	cni failed to teardown pod "hellokitty_default" network: Multus: error in invoke Conflist Del - "chain": error in getting result from DelNetworkList: error getting ClusterInformation: Get https://[fd00
	4::1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: dial tcp [fd04::1]:443: connect: network is unreachable]



Adding default route: default via fd00::1 dev enp0s3 metric 1024 pref medium
	  Normal   SandboxChanged          66s (x9 over 2m3s)  kubelet, controller-0  Pod sandbox changed, it will be killed and re-created.
	  Normal   Pulling                 60s                 kubelet, controller-0  Pulling image "gwaines/hellokitty:v1.0"
	  Normal   Pulled                  50s                 kubelet, controller-0  Successfully pulled image "gwaines/hellokitty:v1.0"
	  Normal   Created                 50s                 kubelet, controller-0  Created container hellokitty
	  Normal   Started                 49s                 kubelet, controller-0  Started container hellokitty

No default route, when coredns was deployed:
	  Warning  FailedCreatePodSandBox  4m30s                  kubelet, controller-0  Failed create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "5e56f5b8c36f33c680da399
	543170fe95a8738ddbe7f38b6b339b910cea8a6c2c" network for pod "coredns-7cf476b5c8-thwk8": NetworkPlugin cni failed to set up pod "coredns-7cf476b5c8-thwk8_kube-system" network: Multus: Err adding pod too
	 network "chain": Multus: error in invoke Conflist add - "chain": error in getting result from AddNetworkList: error getting ClusterInformation: Get https://[fd04::1]:443/apis/crd.projectcalico.org/v11
	/clusterinformations/default: dial tcp [fd04::1]:443: connect: network is unreachable, failed to clean up sandbox container "5e56f5b8c36f33c680da39543170fe95a8738ddbe7f38b6b339b910cea8a6c2c" network ff
	or pod "coredns-7cf476b5c8-thwk8": NetworkPlugin cni failed to teardown pod "coredns-7cf476b5c8-thwk8_kube-system" network: Multus: error in invoke Conflist Del - "chain": error in getting result fromm
	 DelNetworkList: error getting ClusterInformation: Get https://[fd04::1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: dial tcp [fd04::1]:443: connect: network is unreachable]