Comment 2 for bug 2003360

Reviewed: https://review.opendev.org/c/starlingx/config/+/871114
Committed: https://opendev.org/starlingx/config/commit/af012621885c637aea4c64aa043af8d353082d0a
Submitter: "Zuul (22348)"
Branch: master

commit af012621885c637aea4c64aa043af8d353082d0a
Author: Boovan Rajendran <email address hidden>
Date: Thu Jan 19 12:00:17 2023 -0500

    Fix getting kubelet versions during k8s api server down

    When k8s api server is down, "system kube-host-upgrade
    controller-0 kubelet" fails during k8s upgrade.

    The fix is to add retry when receives max retries
    exceeded exception, so that k8s upgrade will not fail.

    This handles an exception seen in kube_get_control_plane_versions
    when k8s server is down. We now retry the function when there are
    no versions, so that k8s upgrade will not fail.

    Test Plan:
    PASS: Manually disrupt kube-apiserver by temporarily removing and
    later adding the file: /etc/kubernetes/manifests/kube-apiserver.yaml.
    PASS: Tested by manually killing kube-apiserver process.
    PASS: Tested by deleting kube-apiserver pod.

    Closes-Bug: 2003360

    Signed-off-by: Boovan Rajendran <email address hidden>
    Change-Id: I86c231fdeec16abacf2d1667c21eea471b9ecc9e