Comment 4 for bug 1999095

Reviewed: https://review.opendev.org/c/starlingx/config/+/867587
Committed: https://opendev.org/starlingx/config/commit/804a31613820d8064292495637928e5b6a66ee00
Submitter: "Zuul (22348)"
Branch: master

commit 804a31613820d8064292495637928e5b6a66ee00
Author: Chris Friesen <email address hidden>
Date: Tue Dec 13 20:39:02 2022 -0600

    clean up feature gates on k8s upgrade

    During a K8s feature upgrade from 1.23 to 1.24 we need to remove
    the "RemoveSelfLink=false" feature gate from kube-apiserver.

    We had previously handled updating the kubeadm configmap, which
    was sufficient to handle the running system. However, in order
    to properly handle backup and restore after the K8s upgrade to
    1.24 (and just for general tidiness) we need to also remove the
    feature gate from the service parameters and from the
    last_kube_extra_config_bootstrap.yaml file. These changes must
    be done on the active controller, so we can't easily do them
    from puppet where the existing feature-gate changes were done.

    Note that there may be other kube-apiserver feature gates
    specified by the end user, this adds a bit of complexity to the
    code.

    https://review.opendev.org/c/starlingx/integ/+/866804 was intended
    to deal with this problem, but it didn't work if the K8s control
    plane was upgraded on the inactive controller first.

    Test Plan:
    PASS: Test K8s control-plane upgrade on AIO-SX and ensure that
          problematic feature gate is removed from service parameters
          and yaml file.
    PASS: Test K8s control-plane upgrade on AIO-DX upgrading control
          plane on standby controller first. Ensure that the
          problematic feature gate is removed from service parameters
          and yaml file.

    Change-Id: Ief45be6e1dbae9eee68bb4d2be9535cd4b09f322
    Closes-Bug: 1999095
    Signed-off-by: Chris Friesen <email address hidden>