Brief Description
-----------------
Job stx-admin-ks-user is not ready on AIO-SX.
Severity
------------------------------
Major
Steps to Reproduce
-----------------------------
Apply stx-openstack
Expected Behavior
------------------
stx-openstack is applied successfully
Actual Behavior
----------------
stx-openstack is apply-failed
Reproducibility
----------------------------
Intermittent
System Configuration
----------------------------
AIO-SX
Timestamp/Logs
----------------------------
2021-03-05 16:25:42.977 627 ERROR armada.handlers.wait [-] [chart=openstack-keystone]: Timed out waiting for jobs (namespace=openstack, labels=(release_group=osh-openstack-keystone)). These jobs were not ready=['keystone-bootstrap', 'stx-admin-ks-user']^[[00m
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada [-] Chart deploy [openstack-keystone] failed: armada.exceptions.k8s_exceptions.KubernetesWatchTimeoutException: Timed out waiting for jobs (namespace=openstack, labels=(release_group=osh-openstack-keystone)). These jobs were not ready=['keystone-bootstrap', 'stx-admin-ks-user']
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada Traceback (most recent call last):
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 225, in handle_result
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada result = get_result()
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 236, in <lambda>
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada if (handle_result(chart, lambda: deploy_chart(chart))):
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 214, in deploy_chart
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada chart, cg_test_all_charts, prefix, known_releases)
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada File "/usr/local/lib/python3.6/dist-packages/armada/handlers/chart_deploy.py", line 248, in execute
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada chart_wait.wait(timer)
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada File "/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 134, in wait
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada wait.wait(timeout=timeout)
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada File "/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 294, in wait
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada modified = self._wait(deadline)
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada File "/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 354, in _wait
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada raise k8s_exceptions.KubernetesWatchTimeoutException(error)
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada armada.exceptions.k8s_exceptions.KubernetesWatchTimeoutException: Timed out waiting for jobs (namespace=openstack, labels=(release_group=osh-openstack-keystone)). These jobs were not ready=['keystone-bootstrap', 'stx-admin-ks-user']
2021-03-05 16:25:42.978 627 ERROR armada.handlers.armada ^[[00m
2021-03-05 16:25:42.979 627 ERROR armada.handlers.armada [-] Chart deploy(s) failed: ['openstack-keystone']^[[00m
2021-03-05 16:25:43.817 627 INFO armada.handlers.lock [-] Releasing lock^[[00m
2021-03-05 16:25:43.841 627 ERROR armada.cli [-] Caught internal exception: armada.exceptions.armada_exceptions.ChartDeployException: Exception deploying charts: ['openstack-keystone']
2021-03-05 16:25:43.841 627 ERROR armada.cli Traceback (most recent call last):
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/__init__.py", line 38, in safe_invoke
2021-03-05 16:25:43.841 627 ERROR armada.cli self.invoke()
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 213, in invoke
2021-03-05 16:25:43.841 627 ERROR armada.cli resp = self.handle(documents, tiller)
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py", line 81, in func_wrapper
2021-03-05 16:25:43.841 627 ERROR armada.cli return future.result()
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2021-03-05 16:25:43.841 627 ERROR armada.cli return self.__get_result()
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2021-03-05 16:25:43.841 627 ERROR armada.cli raise self._exception
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2021-03-05 16:25:43.841 627 ERROR armada.cli result = self.fn(*self.args, **self.kwargs)
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 256, in handle
2021-03-05 16:25:43.841 627 ERROR armada.cli return armada.sync()
2021-03-05 16:25:43.841 627 ERROR armada.cli File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 252, in sync
2021-03-05 16:25:43.841 627 ERROR armada.cli raise armada_exceptions.ChartDeployException(failures)
2021-03-05 16:25:43.841 627 ERROR armada.cli armada.exceptions.armada_exceptions.ChartDeployException: Exception deploying charts: ['openstack-keystone']
2021-03-05 16:25:43.841 627 ERROR armada.cli ^[[00m
Test Activity
-------------
STX regression
Screening note: Intermittent issue so tagging for the next STX release.