Please provide full collect logs for the second failure.
The sysinv logs from the 23rd indicate that the service running on port 9001 (local docker registry?) may not have been running or configured properly.
2019-05-23 22:02:14.120 99896 INFO sysinv.conductor.kube_app [-] Application overrides generated.
2019-05-23 22:02:14.165 99896 INFO sysinv.conductor.kube_app [-] Armada manifest file has no img tags for chart helm-toolkit
2019-05-23 22:02:14.183 99896 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/external_storage/rbd-provisioner:v2.1.1-k8s1.11 download started from local registry
2019-05-23 22:02:14.212 99896 INFO sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/port/ceph-config-helper:v1.10.3 download started from local registry
2019-05-23 22:02:14.384 92020 INFO sysinv.agent.manager [req-9ba7d1b3-c924-4da3-9f92-535efb16040a admin None] Runtime manifest apply completed for classes [u'openstack::keystone::endpoint::runtime', u'platform::firewall::runtime', u'platform::sysinv::runtime'].
2019-05-23 22:02:14.385 92020 INFO sysinv.agent.manager [req-9ba7d1b3-c924-4da3-9f92-535efb16040a admin None] Agent config applied 79e3d68c-85bc-4f58-854e-aa0dc51bf3fa
2019-05-23 22:02:14.409 99896 INFO sysinv.conductor.manager [req-9ba7d1b3-c924-4da3-9f92-535efb16040a admin None] SYS_I Clear system config alarm: controller-0 target config 79e3d68c-85bc-4f58-854e-aa0dc51bf3fa
2019-05-23 22:02:24.469 99896 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/docker.io/port/ceph-config-helper:v1.10.3 download failed from local registry: 500 Server Error: Internal Server Error ("Get https://192.168.204.2:9001/v2/: net/http: TLS handshake timeout")
2019-05-23 22:02:24.480 99896 ERROR sysinv.conductor.kube_app [-] Image 192.168.204.2:9001/quay.io/external_storage/rbd-provisioner:v2.1.1-k8s1.11 download failed from local registry: 500 Server Error: Internal Server Error ("Get https://192.168.204.2:9001/v2/: net/http: TLS handshake timeout")
2019-05-23 22:02:24.480 99896 ERROR sysinv.conductor.kube_app [-] Deployment of application platform-integ-apps (1.0-5) failed: failed to download one or more image(s).
2019-05-23 22:02:24.480 99896 TRACE sysinv.conductor.kube_app Traceback (most recent call last):
2019-05-23 22:02:24.480 99896 TRACE sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 1197, in perform_app_apply
2019-05-23 22:02:24.480 99896 TRACE sysinv.conductor.kube_app self._download_images(app)
2019-05-23 22:02:24.480 99896 TRACE sysinv.conductor.kube_app File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 534, in _download_images
2019-05-23 22:02:24.480 99896 TRACE sysinv.conductor.kube_app reason="failed to download one or more image(s).")
2019-05-23 22:02:24.480 99896 TRACE sysinv.conductor.kube_app KubeAppApplyFailure: Deployment of application platform-integ-apps (1.0-5) failed: failed to download one or more image(s).
2019-05-23 22:02:24.480 99896 TRACE sysinv.conductor.kube_app
2019-05-23 22:02:24.491 99896 ERROR sysinv.conductor.kube_app [-] Application apply aborted!.
Please provide full collect logs for the second failure.
The sysinv logs from the 23rd indicate that the service running on port 9001 (local docker registry?) may not have been running or configured properly.
2019-05-23 22:02:14.120 99896 INFO sysinv. conductor. kube_app [-] Application overrides generated. conductor. kube_app [-] Armada manifest file has no img tags for chart helm-toolkit conductor. kube_app [-] Image 192.168. 204.2:9001/ quay.io/ external_ storage/ rbd-provisioner :v2.1.1- k8s1.11 download started from local registry conductor. kube_app [-] Image 192.168. 204.2:9001/ docker. io/port/ ceph-config- helper: v1.10.3 download started from local registry agent.manager [req-9ba7d1b3- c924-4da3- 9f92-535efb1604 0a admin None] Runtime manifest apply completed for classes [u'openstack: :keystone: :endpoint: :runtime' , u'platform: :firewall: :runtime' , u'platform: :sysinv: :runtime' ]. agent.manager [req-9ba7d1b3- c924-4da3- 9f92-535efb1604 0a admin None] Agent config applied 79e3d68c- 85bc-4f58- 854e-aa0dc51bf3 fa conductor. manager [req-9ba7d1b3- c924-4da3- 9f92-535efb1604 0a admin None] SYS_I Clear system config alarm: controller-0 target config 79e3d68c- 85bc-4f58- 854e-aa0dc51bf3 fa conductor. kube_app [-] Image 192.168. 204.2:9001/ docker. io/port/ ceph-config- helper: v1.10.3 download failed from local registry: 500 Server Error: Internal Server Error ("Get https:/ /192.168. 204.2:9001/ v2/: net/http: TLS handshake timeout") conductor. kube_app [-] Image 192.168. 204.2:9001/ quay.io/ external_ storage/ rbd-provisioner :v2.1.1- k8s1.11 download failed from local registry: 500 Server Error: Internal Server Error ("Get https:/ /192.168. 204.2:9001/ v2/: net/http: TLS handshake timeout") conductor. kube_app [-] Deployment of application platform-integ-apps (1.0-5) failed: failed to download one or more image(s). conductor. kube_app Traceback (most recent call last): conductor. kube_app File "/usr/lib64/ python2. 7/site- packages/ sysinv/ conductor/ kube_app. py", line 1197, in perform_app_apply conductor. kube_app self._download_ images( app) conductor. kube_app File "/usr/lib64/ python2. 7/site- packages/ sysinv/ conductor/ kube_app. py", line 534, in _download_images conductor. kube_app reason="failed to download one or more image(s).") conductor. kube_app KubeAppApplyFai lure: Deployment of application platform-integ-apps (1.0-5) failed: failed to download one or more image(s). conductor. kube_app conductor. kube_app [-] Application apply aborted!.
2019-05-23 22:02:14.165 99896 INFO sysinv.
2019-05-23 22:02:14.183 99896 INFO sysinv.
2019-05-23 22:02:14.212 99896 INFO sysinv.
2019-05-23 22:02:14.384 92020 INFO sysinv.
2019-05-23 22:02:14.385 92020 INFO sysinv.
2019-05-23 22:02:14.409 99896 INFO sysinv.
2019-05-23 22:02:24.469 99896 ERROR sysinv.
2019-05-23 22:02:24.480 99896 ERROR sysinv.
2019-05-23 22:02:24.480 99896 ERROR sysinv.
2019-05-23 22:02:24.480 99896 TRACE sysinv.
2019-05-23 22:02:24.480 99896 TRACE sysinv.
2019-05-23 22:02:24.480 99896 TRACE sysinv.
2019-05-23 22:02:24.480 99896 TRACE sysinv.
2019-05-23 22:02:24.480 99896 TRACE sysinv.
2019-05-23 22:02:24.480 99896 TRACE sysinv.
2019-05-23 22:02:24.480 99896 TRACE sysinv.
2019-05-23 22:02:24.491 99896 ERROR sysinv.