Testrun https://solutions.qa.canonical.com/v2/testruns/e0687dc4-35e6-42b2-ae23-ffe0780bc367/ which is ck8s 1.27 on aws focal with kube ovn fails on the k8s test suite with:
====================
=================================== FAILURES ===================================
_______________________ test_clusterip_service_endpoint ________________________
Traceback (most recent call last):
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 107, in test_clusterip_service_endpoint
await setup_svc("ClusterIP")
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 45, in setup_svc
sh.kubectl.create(
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 1524, in __call__
return RunningCommand(cmd, call_args, stdin, stdout, stderr)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 788, in __init__
self.wait()
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 845, in wait
self.handle_command_exit_code(exit_code)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 869, in handle_command_exit_code
raise exc
sh.ErrorReturnCode_1:
RAN: /snap/bin/kubectl create deployment hello-world --image=rocks.canonical.com/cdk/google-samples/hello-app:1.0
STDOUT:
STDERR:
error: failed to create deployment: an error on the server ("<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.18.0 (Ubuntu)</center>\r\n</body>\r\n</html>") has prevented the request from succeeding (post deployments.apps)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 126, in test_clusterip_service_endpoint
await cleanup()
File "/home/ubuntu/k8s-validation/jobs/integration/test_service_endpoints.py", line 70, in cleanup
sh.kubectl.delete("deployment", "hello-world")
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 1524, in __call__
return RunningCommand(cmd, call_args, stdin, stdout, stderr)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 788, in __init__
self.wait()
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 845, in wait
self.handle_command_exit_code(exit_code)
File "/home/ubuntu/k8s-validation/.tox/py3/lib/python3.8/site-packages/sh.py", line 869, in handle_command_exit_code
raise exc
sh.ErrorReturnCode_1:
RAN: /snap/bin/kubectl delete deployment hello-world
STDOUT:
STDERR:
E0525 06:50:15.751378 81551 memcache.go:265] couldn't get current server API group list: an error on the server ("<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.18.0 (Ubuntu)</center>\r\n</body>\r\n</html>") has prevented the request from succeeding
E0525 06:50:15.754554 81551 memcache.go:265] couldn't get current server API group list: an error on the server ("<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.18.0 (Ubuntu)</center>\r\n</body>\r\n</html>") has prevented the request from succeeding
====================
I will investigate a bit more later on why this is happening.
Crashdumps and configs can be found here:
https://oil-jenkins.canonical.com/artifacts/e0687dc4-35e6-42b2-ae23-ffe0780bc367/index.html
