Brief Description
Upgrade activation fails on script update-k8s-feature-gates.sh.
Severity
Major: System/Feature is usable but degraded
Steps to Reproduce
Execute upgrade procedure
Expected Behavior
Upgrade activation works
Actual Behavior
Upgrade activation fails
Reproducibility
Intermittent
System Configuration
AIO-DX
Did this test scenario pass previously? If so, please indicate the load/pull time info of the last pass.
Yes, same load
Timestamp/Logs
```
2022-09-20T07:33:24.813 Debug: 2022-09-20 07:33:23 +0000 Exec[update kubeadm feature-gates](provider=posix): Executing '/usr/local/sbin/update-k8s-feature-gates.sh'
2022-09-20T07:33:24.815 Debug: 2022-09-20 07:33:23 +0000 Executing: '/usr/local/sbin/update-k8s-feature-gates.sh'
2022-09-20T07:33:24.818 Notice: 2022-09-20 07:33:24 +0000 /Stage[main]/Platform::Kubernetes::Master::Update_kubeadm_feature_gates/Exec[update kubeadm feature-gates]/returns: Error from server (Conflict): error when replacing "/tmp/kubeadm_cm": Operation cannot be fulfilled on configmaps "kubeadm-config": the object has been modified; please apply your changes to the latest version and try again
2022-09-20T07:33:24.820 Error: 2022-09-20 07:33:24 +0000 /usr/local/sbin/update-k8s-feature-gates.sh returned 1 instead of one of [0]
2022-09-20T07:33:24.826 /usr/share/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail'
2022-09-20T07:33:24.828 /usr/share/ruby/vendor_ruby/puppet/type/exec.rb:160:in `sync'
2022-09-20T07:33:24.833 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync'
2022-09-20T07:33:24.837 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed'
2022-09-20T07:33:24.842 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes'
2022-09-20T07:33:24.847 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each'
2022-09-20T07:33:24.850 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes'
2022-09-20T07:33:24.852 /usr/share/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate'
2022-09-20T07:33:24.858 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:230:in `apply'
2022-09-20T07:33:24.860 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:246:in `eval_resource'
2022-09-20T07:33:24.862 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `call'
2022-09-20T07:33:24.867 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate'
2022-09-20T07:33:24.869 /usr/share/ruby/vendor_ruby/puppet/util.rb:386:in `block in thinmark'
2022-09-20T07:33:24.871 /usr/share/ruby/benchmark.rb:296:in `realtime'
2022-09-20T07:33:24.873 /usr/share/ruby/vendor_ruby/puppet/util.rb:385:in `thinmark'
2022-09-20T07:33:24.875 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate'
2022-09-20T07:33:24.878 /usr/share/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
2022-09-20T07:33:24.880 /usr/share/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate'
2022-09-20T07:33:24.882 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply'
2022-09-20T07:33:24.884 /usr/share/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination'
2022-09-20T07:33:24.887 /usr/share/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination'
2022-09-20T07:33:24.889 /usr/share/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply'
2022-09-20T07:33:24.894 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog'
2022-09-20T07:33:24.898 /usr/share/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark'
2022-09-20T07:33:24.900 /usr/share/ruby/benchmark.rb:296:in `realtime'
2022-09-20T07:33:24.902 /usr/share/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark'
2022-09-20T07:33:24.904 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog'
2022-09-20T07:33:24.907 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal'
2022-09-20T07:33:24.909 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run'
2022-09-20T07:33:24.912 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2022-09-20T07:33:24.916 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'
2022-09-20T07:33:24.926 /usr/share/ruby/vendor_ruby/puppet/configurer.rb:195:in `run'
2022-09-20T07:33:24.931 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog'
2022-09-20T07:33:24.934 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main'
2022-09-20T07:33:24.936 /usr/share/ruby/vendor_ruby/puppet/context.rb:65:in `override'
2022-09-20T07:33:24.938 /usr/share/ruby/vendor_ruby/puppet.rb:241:in `override'
2022-09-20T07:33:24.940 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main'
2022-09-20T07:33:24.941 /usr/share/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command'
2022-09-20T07:33:24.944 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `block in run'
2022-09-20T07:33:24.946 /usr/share/ruby/vendor_ruby/puppet/util.rb:540:in `exit_on_fail'
2022-09-20T07:33:24.949 /usr/share/ruby/vendor_ruby/puppet/application.rb:344:in `run'
2022-09-20T07:33:24.951 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
2022-09-20T07:33:24.954 /usr/share/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
2022-09-20T07:33:24.956 /usr/bin/puppet:5:in `<main>'
2022-09-20T07:33:24.958 Error: 2022-09-20 07:33:24 +0000 /Stage[main]/Platform::Kubernetes::Master::Update_kubeadm_feature_gates/Exec[update kubeadm feature-gates]/returns: change from notrun to 0 failed: /usr/local/sbin/update-k8s-feature-gates.sh returned 1 instead of one of [0]
```
Alarms
Test Activity
Sanity
Workaround
---
Reviewed: https:/
Committed: https:/
Submitter: "Zuul (22348)"
Branch: master
commit 6e7736059ac9278
Author: Kaustubh Dhokte <email address hidden>
Date: Mon Nov 14 16:28:47 2022 +0000
update feature-gates for specific k8s version
This change modifies upgrade_
k8s feature-gates for different k8s versions. With every k8s release,
default values of some feature-gate are changed and usage of some
feature-gates often gets deprecated.
The script runs during each k8s control plane upgrade before
upgrading first master. It modifies kubeadm-config configmap
with features-gates as required for the specific k8s version
we are upgrading to.
The set of changes here, https:/
(k8s-1.22.5: remove feature-gates), https:/
(Add a puppet class to support k8s feature-gate update), and https:/
(apply feature-gate update during upgrade-activate) were added for
stx 6.0 to stx 7.0 upgrade (CentOS) for changes in feature-gates with
respect to k8s 1.22. Instead of adding that script to Debian and
maintaining two different scripts, going forward we can maintain this
single script to accommodate any change in feature-gates
(or any other config in kubeadm-config) with respect to the specific
k8s version we are upgrading to.
Test Plan:
PASS: K8s upgrade 1.21.8 to 1.22.5
PASS: k8s upgrade 1.23.1 to 1.24.4
PASS: shellcheck run
PASS: replace_configmap function was unit tested separately.
Closes-Bug: 1996546
Closes-Bug: 1990880
Signed-off-by: Kaustubh Dhokte <email address hidden>
Change-Id: Ib693d7892aee2d