test_status_messages fails because Kubernetes master is not running

[Marian Gasparovic]

test_status_messages

Traceback (most recent call last):
  File "/home/ubuntu/k8s-validation/jobs/integration/validation.py", line 257, in test_status_messages
    assert unit.workload_status_message == message
AssertionError: assert 'Waiting for ... pod to start' == 'Kubernetes master running.'
  - Kubernetes master running.
  + Waiting for 1 kube-system pod to start

This is the first time we encountered this error.

charmed-kubernetes_1.20
juju_2.8.10

crashdump
https://oil-jenkins.canonical.com/artifacts/5ba2a55a-aac6-4a9b-aeb8-6ce24d227e43/generated/generated/kubernetes/juju-crashdump-kubernetes-2021-04-07-13.37.43.tar.gz

Other artifacts
https://oil-jenkins.canonical.com/artifacts/5ba2a55a-aac6-4a9b-aeb8-6ce24d227e43/index.html

[George Kraft (cynerva)]

By the time the crashdump was captured, all kube-system pods were in a Running state. Nothing really wrong with the cluster here, test_status_messages is just overly sensitive. It needs to be made more robust, it should do a `juju wait` before checking the status messages.