Need to sanitize SCTPSupport feature gate before upgrading to K8s 1.22

Bug #2015917 reported by Saba Touheed Mujawar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Medium
Saba Touheed Mujawar

Bug Description

Brief Description
-----------------
In some (possibly most) circumstances if upgrading from 21.12 to 22.12 we can be left with the "SCTPSupport" feature gate, which is not supported by Kubernetes 1.22. In this scenario, attempting to upgrade to K8s 1.22 will result in kube-apiserver failing to start, which will result in a non-functional K8s control plane.

In the 22.06 CentOS-based load we had a workaround for this, but the Debian implementation will need to be different to fully account for the custom Kubernetes configuration support.

Severity
--------
Major

Steps to Reproduce
------------------
Per CGTS-45939, Perform Platform Upgrade from 21.12 Patch 9 to 22.12 Patch 1 as per procedure and then attempt to upgrade K8s to 1.22.5.

Expected Behavior
------------------
K8s should be upgraded to 1.22.5.

Actual Behavior
----------------
kube-apiserver refuses to start up due to the "SCTPSupport" feature gate no longer being supported.

Reproducibility
---------------
Reproducible

System Configuration
--------------------
Could affect any configuration.

Last Pass
---------
Likely a day-one issue with the 21.12->22.12 upgrade.

Timestamp/Logs
--------------
From a dedicated test run in VBOX:

root@controller-0:/var/log/puppet# cat /var/log/containers/kube-apiserver-controller-0_kube-system_kube-apiserver-f0018ab572c3541221009514ca40d8677a45a6f9e10422ca0770937e796d5482.log
2023-04-10T17:33:24.2701004Z stderr F Error: invalid argument "SCTPSupport=true,TTLAfterFinished=true,HugePageStorageMediumSize=true" for "--feature-gates" flag: unrecognized feature gate: SCTPSupport

Test Activity
-------------
Developer Testing

Workaround
----------
Manually editing the configmap prior to upgrading K8s.

Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to config (master)

Reviewed: https://review.opendev.org/c/starlingx/config/+/880063
Committed: https://opendev.org/starlingx/config/commit/2afa67e8730b62465b02041865934b4d42ded3f7
Submitter: "Zuul (22348)"
Branch: master

commit 2afa67e8730b62465b02041865934b4d42ded3f7
Author: Saba Touheed Mujawar <email address hidden>
Date: Tue Apr 11 11:18:11 2023 -0400

    Sanitize SCTPSupport feature gate before upgrading to K8s 1.22

    k8s 1.22 does not support "SCTPSupport" feature gate.
    Upgrading from 21.12 to 22.12 and then upgrading to k8s 1.22
    will result in kube-apiserver failing to start, which will result
    in a non-functional K8s control plane.

    Changes are made to remove this feature gate.

    Also, in k8s version 1.24 and later, name of the kubelet configmap
    is changed to "kubelet-config" from kubelet-config-<version>.

    Test Plan:
    PASS : k8s 1.21.8 upgrade with custom extraArgs removes SCTPSupport
           Confirmed kubeadm-configmap is updated.
    PASS : Added featureGate "SCTPSupport" manually to
           kubelet_configurations and upgarded k8s to 1.22.
           kubelet-configmap is updated
    PASS : Package builds successfully.

    Closes-Bug: 2015917

    Change-Id: Ief8791fe4307e2d8b70cc08f08a5bdd7c452b662
    Signed-off-by: Saba Touheed Mujawar <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Medium
tags: added: stx.9.0 stx.containers
Changed in starlingx:
assignee: nobody → Saba Touheed Mujawar (smujawar)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.