| 2023-10-03 13:43:18 |
Boovan Rajendran |
bug |
|
|
added bug |
| 2023-10-03 13:43:24 |
Boovan Rajendran |
starlingx: assignee |
|
Boovan Rajendran (brajendr) |
|
| 2023-10-03 13:44:45 |
Boovan Rajendran |
description |
Brief Description:
K8s upgrade-abort in a SX stuck at upgrade-aborting state
Severity:
Major
Steps to Reproduce:
system kube-upgrade-start v1.22.5
system kube-upgrade-download-images
system kube-upgrade-networking
system kube-host-upgrade controller-0 control-plane
system kube-upgrade-abort
After controller-0 is upgraded abort the upgrade
Expected Behavior:
Upgrade should be in upgrade-aborted state
Actual Behavior:
sysadmin@controller-0 (keystone_admin)]$ system kube-upgrade-show
-----------{}{}-----------------------------------+
Property Value
-----------{}{}-----------------------------------+
uuid aa3bf9f5-e677-4e09-8da8-fd1fad29bd24
from_version v1.21.8
to_version v1.22.5
state upgrade-aborting
created_at 2023-09-29T01:52:32.964855+00:00
updated_at 2023-09-29T02:02:57.925764+00:00
-----------{}{}-----------------------------------+
Reproducibility:
100%
System Configuration
AIO-SX
Last Pass
NA
Timestamp/Logs
sysinv 2023-09-29 02:06:15.360 20601 WARNING sysinv.puppet.kubernetes [-] Exception generating bootstrap token: subprocess.CalledProcessError: Command '['/usr/local/kubernetes/1.21.8/stage1/usr/bin/kubeadm', 'init', 'phase', 'upload-certs', '--upload-certs', '--config', '/tmp/tmp4c83zp55.yaml']' returned non-zero exit status 1.
sysinv 2023-09-29 02:06:15.368 20601 WARNING sysinv.puppet.kubernetes [-] Retrying in _get_kubernetes_join_cmd
sysinv 2023-09-29 02:06:15.368 20601 ERROR sysinv.conductor.manager [-] _config_update_puppet Failed to generate bootstrap token: sysinv.common.exception.SysinvException: Failed to generate bootstrap token
Alarms
NA
Test Activity
Feature Testing
Workaround
Describe workaround if available |
Brief Description:
K8s upgrade-abort in a SX stuck at upgrade-aborting state
Severity:
Major
Steps to Reproduce:
system kube-upgrade-start v1.22.5
system kube-upgrade-download-images
system kube-upgrade-networking
system kube-host-upgrade controller-0 control-plane
system kube-upgrade-abort
After controller-0 is upgraded abort the upgrade
Expected Behavior:
Upgrade should be in upgrade-aborted state
Actual Behavior:
sysadmin@controller-0 (keystone_admin)]$ system kube-upgrade-show
-----------{}{}-----------------------------------+
Property Value
-----------{}{}-----------------------------------+
uuid aa3bf9f5-e677-4e09-8da8-fd1fad29bd24
from_version v1.21.8
to_version v1.22.5
state upgrade-aborting
created_at 2023-09-29T01:52:32.964855+00:00
updated_at 2023-09-29T02:02:57.925764+00:00
-----------{}{}-----------------------------------+
Reproducibility:
100%
System Configuration
AIO-SX
Last Pass
NA
Timestamp/Logs
sysinv 2023-09-29 02:06:15.360 20601 WARNING sysinv.puppet.kubernetes [-] Exception generating bootstrap token: subprocess.CalledProcessError: Command '['/usr/local/kubernetes/1.21.8/stage1/usr/bin/kubeadm', 'init', 'phase', 'upload-certs', '--upload-certs', '--config', '/tmp/tmp4c83zp55.yaml']' returned non-zero exit status 1.
sysinv 2023-09-29 02:06:15.368 20601 WARNING sysinv.puppet.kubernetes [-] Retrying in _get_kubernetes_join_cmd
sysinv 2023-09-29 02:06:15.368 20601 ERROR sysinv.conductor.manager [-] _config_update_puppet Failed to generate bootstrap token: sysinv.common.exception.SysinvException: Failed to generate bootstrap token
Alarms
NA
Test Activity
Feature Testing
Workaround
Describe workaround if available |
|
| 2023-10-03 13:46:25 |
Boovan Rajendran |
description |
Brief Description:
K8s upgrade-abort in a SX stuck at upgrade-aborting state
Severity:
Major
Steps to Reproduce:
system kube-upgrade-start v1.22.5
system kube-upgrade-download-images
system kube-upgrade-networking
system kube-host-upgrade controller-0 control-plane
system kube-upgrade-abort
After controller-0 is upgraded abort the upgrade
Expected Behavior:
Upgrade should be in upgrade-aborted state
Actual Behavior:
sysadmin@controller-0 (keystone_admin)]$ system kube-upgrade-show
-----------{}{}-----------------------------------+
Property Value
-----------{}{}-----------------------------------+
uuid aa3bf9f5-e677-4e09-8da8-fd1fad29bd24
from_version v1.21.8
to_version v1.22.5
state upgrade-aborting
created_at 2023-09-29T01:52:32.964855+00:00
updated_at 2023-09-29T02:02:57.925764+00:00
-----------{}{}-----------------------------------+
Reproducibility:
100%
System Configuration
AIO-SX
Last Pass
NA
Timestamp/Logs
sysinv 2023-09-29 02:06:15.360 20601 WARNING sysinv.puppet.kubernetes [-] Exception generating bootstrap token: subprocess.CalledProcessError: Command '['/usr/local/kubernetes/1.21.8/stage1/usr/bin/kubeadm', 'init', 'phase', 'upload-certs', '--upload-certs', '--config', '/tmp/tmp4c83zp55.yaml']' returned non-zero exit status 1.
sysinv 2023-09-29 02:06:15.368 20601 WARNING sysinv.puppet.kubernetes [-] Retrying in _get_kubernetes_join_cmd
sysinv 2023-09-29 02:06:15.368 20601 ERROR sysinv.conductor.manager [-] _config_update_puppet Failed to generate bootstrap token: sysinv.common.exception.SysinvException: Failed to generate bootstrap token
Alarms
NA
Test Activity
Feature Testing
Workaround
Describe workaround if available |
Brief Description:
K8s upgrade-abort in a SX stuck at upgrade-aborting state
Severity:
Major
Steps to Reproduce:
system kube-upgrade-start v1.22.5
system kube-upgrade-download-images
system kube-upgrade-networking
system kube-host-upgrade controller-0 control-plane
system kube-upgrade-abort
After controller-0 is upgraded abort the upgrade
Expected Behavior:
Upgrade should be in upgrade-aborted state
Actual Behavior:
[sysadmin@controller-0 ~(keystone_admin)]$ system kube-upgrade-abort
+--------------+--------------------------------------+
| Property | Value |
+--------------+--------------------------------------+
| uuid | 527d95e1-307c-4836-9614-f8ebd844691f |
| from_version | v1.21.8 |
| to_version | v1.22.5 |
| state | upgrade-aborting |
| created_at | 2023-10-03T07:05:11.711079+00:00 |
| updated_at | 2023-10-03T07:29:36.126765+00:00 |
+--------------+--------------------------------------+
Reproducibility:
100%
System Configuration
AIO-SX
Last Pass
NA
Timestamp/Logs
sysinv 2023-09-29 02:06:15.360 20601 WARNING sysinv.puppet.kubernetes [-] Exception generating bootstrap token: subprocess.CalledProcessError: Command '['/usr/local/kubernetes/1.21.8/stage1/usr/bin/kubeadm', 'init', 'phase', 'upload-certs', '--upload-certs', '--config', '/tmp/tmp4c83zp55.yaml']' returned non-zero exit status 1.
sysinv 2023-09-29 02:06:15.368 20601 WARNING sysinv.puppet.kubernetes [-] Retrying in _get_kubernetes_join_cmd
sysinv 2023-09-29 02:06:15.368 20601 ERROR sysinv.conductor.manager [-] _config_update_puppet Failed to generate bootstrap token: sysinv.common.exception.SysinvException: Failed to generate bootstrap token
Alarms
NA
Test Activity
Feature Testing
Workaround
Describe workaround if available |
|
| 2023-10-03 14:06:11 |
OpenStack Infra |
starlingx: status |
New |
In Progress |
|
| 2023-10-06 16:58:18 |
OpenStack Infra |
starlingx: status |
In Progress |
Fix Released |
|
| 2023-10-11 01:04:19 |
Ghada Khalil |
starlingx: importance |
Undecided |
Medium |
|
| 2023-10-11 01:04:33 |
Ghada Khalil |
tags |
|
stx.9.0 stx.containers stx.update |
|
