remove fragile feature gate cleanup code in upgrade_k8s_config.sh

Bug #2017696 reported by Gleb Aronsky
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Medium
Gleb Aronsky

Bug Description

Brief Description
-----------------

In /usr/local/sbin/upgrade_k8s_config.sh we have some legacy cleanup code that does not support the "flow" style of YAML. This means that if something reformats the kubeadm configmap into "flow" style then on the upgrade to K8s 1.22 the script will corrupt the configmap.

The whole /usr/local/sbin/upgrade_k8s_config.sh script is no longer needed now that we fix up the configmap as needed in sysinv as part of the K8s upgrade. We can delete the file from kubernetes-unversioned and remove the puppet code that calls it, or else delete the guts of the script but leave it there in case we need it later.

Severity
--------
Major

Steps to Reproduce
------------------
upgrade K8s to 1.22 from 1.21

Expected Behavior
------------------
Kubeadm configmap is adjusted to remove unsupported feature gates.

Actual Behavior
----------------
Kubeadm configmap is corrupted due to the script removing the entire line containing TTLAfterFinished=true without being YAML-aware. This is okay if the configmap is in "block" style YAML, but in "flow" style it corrupts the configmap.

Reproducibility
---------------
Reproducible

System Configuration
--------------------
All

Branch/Pull Time/Commit
-----------------------

Last Pass
---------

Timestamp/Logs
--------------

Test Activity
-------------
Testing

Workaround
----------

Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to integ (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/integ/+/881510

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on integ (master)

Change abandoned by "Gleb Aronsky <email address hidden>" on branch: master
Review: https://review.opendev.org/c/starlingx/integ/+/881510

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to stx-puppet (master)

Reviewed: https://review.opendev.org/c/starlingx/stx-puppet/+/880547
Committed: https://opendev.org/starlingx/stx-puppet/commit/77e0c7c14fb489194960bd7bf5f9f5f0e9495020
Submitter: "Zuul (22348)"
Branch: master

commit 77e0c7c14fb489194960bd7bf5f9f5f0e9495020
Author: Gleb Aronsky <email address hidden>
Date: Fri Apr 14 13:17:11 2023 -0700

    Remove references to deprecated script

    upgrade_k8s_config.sh has been deprecated and
    removed due to lack of support for "flow" style YAML.
    Deprecated functionality has been superseded
    by better YAML-aware handling in sysinv.

    Updating how we invoke kubeadm, we will now use an explicit
    version of kubeadm when calling it. The version called
    will now match the version we are upgrading to in order to handle
    the format unsupported by previous versions of kubeadm.

    Test Plan
    PASS:
    - Manually update scripts on controllers and worker nodes based on
      https://review.opendev.org/c/starlingx/integ/+/880390
    - Perform manual upgrade from k8s v1.21.8 to v1.22.5
    - Verify kubernetes successfully upgraded to v1.22.5

    Test was performed in the lab with local changes
    to verify the code.
    Patch was not tested.

    Closes-Bug: 2017696
    Change-Id: I840eb566057be495fe0da3cae7604bf8055c0d4f
    Signed-off-by: Gleb Aronsky <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to integ (master)

Reviewed: https://review.opendev.org/c/starlingx/integ/+/880390
Committed: https://opendev.org/starlingx/integ/commit/653f65106eb2ee831cd63f3184c9e77812174bf0
Submitter: "Zuul (22348)"
Branch: master

commit 653f65106eb2ee831cd63f3184c9e77812174bf0
Author: Gleb Aronsky <email address hidden>
Date: Thu Apr 13 14:34:07 2023 -0700

    Remove broken/stale K8s upgrade script

    upgrade_k8s_config.sh does not support
    "flow" style of YAML. This results in
    the configmap becoming corrupted when
    upgrading from k8s 1.21 to 1.22.

    Removing the legacy code, the changes to the
    config map are now handled in sysinv.

    Test Plan
    PASS:
    - Manually update scripts on controllers and worker nodes based on
      https://review.opendev.org/c/starlingx/stx-puppet/+/880547?tab=comments
    - Perform manual upgrade from k8s v1.21.8 to v1.22.5
    - Verify kubernetes successfully upgraded to v1.22.5

    Test was performed in the lab with local changes
    to verify the code.
    Patch was not tested.

    Closes-Bug: 2017696
    Depends-On: https://review.opendev.org/c/starlingx/stx-puppet/+/880547
    Signed-off-by: Gleb Aronsky <email address hidden>
    Change-Id: Ic47c3e40418898917934309f69b35ad1f401eb06

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to stx-puppet (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/stx-puppet/+/881917

Ghada Khalil (gkhalil)
Changed in starlingx:
assignee: nobody → Gleb Aronsky (gleb-aronsky)
importance: Undecided → Medium
tags: added: stx.conta
tags: added: stx.9.0 stx.containers
removed: stx.conta
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to stx-puppet (master)

Reviewed: https://review.opendev.org/c/starlingx/stx-puppet/+/881917
Committed: https://opendev.org/starlingx/stx-puppet/commit/b9ff315ff7425a166d488b4f167e8cb94b5e1fbe
Submitter: "Zuul (22348)"
Branch: master

commit b9ff315ff7425a166d488b4f167e8cb94b5e1fbe
Author: Chris Friesen <email address hidden>
Date: Mon May 1 10:25:12 2023 -0600

    remove reference to deleted exec resource

    In commit 77e0c7c1 we removed an exec resource that called out to an
    obsolete script. However, we neglected to remove a "require"
    metaparameter which referenced the removed script. It's unclear how
    this was missed since the previous change was tested in VirtualBox.

    This causes a puppet error when trying to upgrade K8s:

    Could not find resource 'Exec[update kubeadm-config]' in parameter
    'require' (file:
    /usr/share/puppet/modules/platform/manifests/kubernetes.pp, line: 813)

    The fix is to remove the metaparameter.

    TEST PLAN:
    PASS: While running the dev branch on AIO-DX, upgrade K8s from 1.21
          to 1.22.
          (note, a workaround was required to deal with
           https://bugs.launchpad.net/starlingx/+bug/2018247)

    Partial-Bug: 2017696
    Change-Id: I66c0e88f0f0a3acc3326391263123e60667561cc
    Signed-off-by: Chris Friesen <email address hidden>

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.