The issue can be reproduced on AIO-SX after lock/unlock controller with "20210401T032802Z" load.
2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app Traceback (most recent call last): 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 2294, in perform_app_apply 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app self.app_lifecycle_actions(None, None, rpc_app, lifecycle_hook_info_app_apply) 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 1891, in app_lifecycle_actions 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app lifecycle_op.app_lifecycle_actions(context, conductor_obj, self, app, hook_info) 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app File "/opt/platform/helm/21.05/stx-openstack/1.0-78-centos-stable-versioned/plugins/k8sapp_openstack/lifecycle/lifecycle_openstack.py", line 56, in app_lifecycle_actions 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app return self.pre_manifest_apply(app, app_op, hook_info) 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app File "/opt/platform/helm/21.05/stx-openstack/1.0-78-centos-stable-versioned/plugins/k8sapp_openstack/lifecycle/lifecycle_openstack.py", line 144, in pre_manifest_apply 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app result = helm_utils.get_openstack_pending_install_charts() 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/helm/utils.py", line 205, in get_openstack_pending_install_charts 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app reason="Failed to obtain pending charts list: %s" % e) 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app HelmTillerFailure: Helm operation failure: Failed to obtain pending charts list: Helm operation failure: Error: write tcp 172.16.192.111:33372->192.168.204.1:5432: write: connection reset by peer 2021-04-01 17:11:01.794 95532 ERROR sysinv.conductor.kube_app command terminated with exit code 1
The issue can be reproduced on AIO-SX after lock/unlock controller with "20210401T032802Z" load.
2021-04-01 17:11:01.794 95532 ERROR sysinv. conductor. kube_app Traceback (most recent call last): conductor. kube_app File "/usr/lib64/ python2. 7/site- packages/ sysinv/ conductor/ kube_app. py", line 2294, in perform_app_apply conductor. kube_app self.app_ lifecycle_ actions( None, None, rpc_app, lifecycle_ hook_info_ app_apply) conductor. kube_app File "/usr/lib64/ python2. 7/site- packages/ sysinv/ conductor/ kube_app. py", line 1891, in app_lifecycle_ actions conductor. kube_app lifecycle_ op.app_ lifecycle_ actions( context, conductor_obj, self, app, hook_info) conductor. kube_app File "/opt/platform/ helm/21. 05/stx- openstack/ 1.0-78- centos- stable- versioned/ plugins/ k8sapp_ openstack/ lifecycle/ lifecycle_ openstack. py", line 56, in app_lifecycle_ actions conductor. kube_app return self.pre_ manifest_ apply(app, app_op, hook_info) conductor. kube_app File "/opt/platform/ helm/21. 05/stx- openstack/ 1.0-78- centos- stable- versioned/ plugins/ k8sapp_ openstack/ lifecycle/ lifecycle_ openstack. py", line 144, in pre_manifest_apply conductor. kube_app result = helm_utils. get_openstack_ pending_ install_ charts( ) conductor. kube_app File "/usr/lib64/ python2. 7/site- packages/ sysinv/ helm/utils. py", line 205, in get_openstack_ pending_ install_ charts conductor. kube_app reason="Failed to obtain pending charts list: %s" % e) conductor. kube_app HelmTillerFailure: Helm operation failure: Failed to obtain pending charts list: Helm operation failure: Error: write tcp 172.16. 192.111: 33372-> 192.168. 204.1:5432: write: connection reset by peer conductor. kube_app command terminated with exit code 1
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.
2021-04-01 17:11:01.794 95532 ERROR sysinv.