New version of oidc-auth-apps upload was mistakenly disabled during upgrade activation

Bug #1923004 reported by Bob Church on 2021-04-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Medium
Bob Church

Bug Description

Brief Description
-----------------
The upload of new version of oidc-auth-apps threw an exception resulting in upgrade activation failure

sysinv 2021-04-06 11:31:23.722 135021 INFO sysinv.conductor.kube_app [-] Helm chart oidc-client-0.1.0.tgz uploaded
sysinv 2021-04-06 11:31:23.861 135021 INFO sysinv.conductor.kube_app [-] Helm chart dex-0.8.0.tgz uploaded
sysinv 2021-04-06 11:31:23.902 135021 INFO sysinv.conductor.kube_app [-] PluginHelper: Disabled plugin directory /opt/platform/helm/21.05/oidc-auth-apps/1.0-37/plugins: removed /usr/lib64/python2.7/site-packages/stx_app-oidc-auth-apps-1.0-37.pth
sysinv 2021-04-06 11:31:23.902 135021 INFO sysinv.conductor.kube_app [-] PluginHelper: Disabling distributions: [u'k8sapp-oidc']
sysinv 2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app [-] u'k8sapp-oidc': KeyError: u'k8sapp-oidc'
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app Traceback (most recent call last):
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 1768, in perform_app_upload
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app self._upload_helm_charts(app)
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 878, in _upload_helm_charts
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app self._plugins.deactivate_plugins(app)
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 3806, in deactivate_plugins
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app del pkg_resources.working_set.by_key[distribution]
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app KeyError: u'k8sapp-oidc'
2021-04-06 11:31:23.910 135021 ERROR sysinv.conductor.kube_app
sysinv 2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app [-] Application upload aborted!.
sysinv 2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app [-] Upload of application oidc-auth-apps (1.0-37) failed: u'k8sapp-oidc': KubeAppUploadFailure: Upload of application oidc-auth-apps (1.0-37) failed: u'k8sapp-oidc'
2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app Traceback (most recent call last):
2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 2412, in perform_app_update
2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app lifecycle_hook_info_app_upload=lifecycle_hook_info_app_update)
2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 1790, in perform_app_upload
2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app name=app.name, version=app.version, reason=e)
2021-04-06 11:31:24.133 135021 ERROR sysinv.conductor.kube_app KubeAppUploadFailure: Upload of application oidc-auth-apps (1.0-37) failed:

Severity
--------
Provide the severity of the defect.
Major: System/Feature is usable but degraded>

Steps to Reproduce
------------------
Start with a previous version of oidc-auth-apps and ensure that one or more of the images required by oidc-auth-apps-1.0-37 are not available. This will trigger an update failure and and application rollback. Perform the application-update twice

1. system application-update /usr/local/share/applications/helm/oidc-auth-apps-1.0-37.tgz
2. system application-update /usr/local/share/applications/helm/oidc-auth-apps-1.0-37.tgz

Expected Behavior
------------------
In both attempts, the application update will be attempted, fail, and rolled back to the previous version without the traceback being thrown in the sysinv.log as shown in the description

Actual Behavior
----------------
See descritpion

Reproducibility
---------------
100% reproducible

System Configuration
--------------------
AIOSX, AIODX, Standard, DC

Branch/Pull Time/Commit
-----------------------
Build from 4/5/2021

Last Pass
---------
I don't think this specifc scenario has been previously tested

Timestamp/Logs
--------------

Test Activity
-------------
Developer Testing

Workaround
----------
N/A

Bob Church (rchurch) on 2021-04-08
Changed in starlingx:
assignee: nobody → Bob Church (rchurch)
status: New → In Progress
Ghada Khalil (gkhalil) on 2021-04-08
Changed in starlingx:
importance: Undecided → Medium
tags: added: stx.5.0 stx.containers stx.update
Bob Church (rchurch) wrote :
Changed in starlingx:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers