This is also failing for me in a 2+2 virtual box configuration. This is a non-openstack install - I am not applying the openstack related labels to any of the hosts.

The platform-integ-apps application repeatedly fails to apply. The signature is slightly different than in the collect from Christopher. I will attach a collect.

Here are the armada logs for the failed apply:

2019-05-28 19:50:27.869 41 DEBUG armada.handlers.document [-] Resolving reference /manifests/platform-integ-apps/1.0-5/platform-integ-apps-manifest.yaml. resolve_reference /usr/local/lib/python3.6/dist-packages/armada/handlers/document.py:49
2019-05-28 19:50:27.904 41 DEBUG armada.handlers.tiller [-] Using Tiller namespace: kube-system _get_tiller_namespace /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:174
2019-05-28 19:50:27.963 41 DEBUG armada.handlers.tiller [-] Found at least one Running Tiller pod. _get_tiller_pod /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:150
2019-05-28 19:50:27.963 41 DEBUG armada.handlers.tiller [-] Using Tiller pod IP: 192.168.204.3 _get_tiller_ip /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:165
2019-05-28 19:50:27.963 41 DEBUG armada.handlers.tiller [-] Using Tiller host port: 44134 _get_tiller_port /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:170
2019-05-28 19:50:27.964 41 DEBUG armada.handlers.tiller [-] Tiller getting gRPC insecure channel at 192.168.204.3:44134 with options: [grpc.max_send_message_length=429496729, grpc.max_receive_message_length=429496729] get_channel /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:124
2019-05-28 19:50:27.999 41 DEBUG armada.handlers.tiller [-] Armada is using Tiller at: None:44134, namespace=kube-system, timeout=300 __init__ /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:104
2019-05-28 19:50:28.010 41 INFO armada.handlers.lock [-] Acquiring lock
2019-05-28 19:50:28.024 41 INFO armada.handlers.lock [-] Lock Custom Resource Definition not found, creating now
2019-05-28 19:50:28.047 41 DEBUG armada.handlers.lock [-] Encountered known issue while creating CRD, continuing create_definition /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:297
2019-05-28 19:50:28.050 41 INFO armada.handlers.lock [-] Lock Custom Resource Definition not found, creating now
2019-05-28 19:50:28.109 41 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] helm-toolkit validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.111 41 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rbd-provisioner validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.111 41 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-ceph-pools-audit validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.112 41 DEBUG armada.utils.validate [-] Validating document [armada/ChartGroup/v1] starlingx-ceph-charts validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.112 41 DEBUG armada.utils.validate [-] Validating document [armada/Manifest/v1] platform-integration-manifest validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.114 41 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] helm-toolkit validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.114 41 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rbd-provisioner validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.116 41 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-ceph-pools-audit validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.117 41 DEBUG armada.utils.validate [-] Validating document [armada/ChartGroup/v1] starlingx-ceph-charts validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.117 41 DEBUG armada.utils.validate [-] Validating document [armada/Manifest/v1] platform-integration-manifest validate_armada_document /usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:152
2019-05-28 19:50:28.118 41 INFO armada.handlers.armada [-] Performing pre-flight operations.
2019-05-28 19:50:28.118 41 DEBUG armada.handlers.tiller [-] Using Tiller namespace: kube-system _get_tiller_namespace /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:174
2019-05-28 19:50:28.140 41 DEBUG armada.handlers.tiller [-] Found at least one Running Tiller pod. _get_tiller_pod /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:150
2019-05-28 19:50:28.141 41 DEBUG armada.handlers.tiller [-] Using Tiller pod IP: 192.168.204.3 _get_tiller_ip /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:165
2019-05-28 19:50:28.141 41 DEBUG armada.handlers.tiller [-] Getting Tiller Status: Tiller exists tiller_status /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:182
2019-05-28 19:50:28.142 41 INFO armada.handlers.armada [-] Downloading tarball from: http://controller:8080/helm_charts/stx-platform/rbd-provisioner-0.1.0.tgz
2019-05-28 19:50:28.142 41 WARNING armada.handlers.armada [-] Disabling server validation certs to extract charts
2019-05-28 19:50:28.194 41 INFO armada.handlers.armada [-] Downloading tarball from: http://controller:8080/helm_charts/stx-platform/helm-toolkit-0.1.0.tgz
2019-05-28 19:50:28.195 41 WARNING armada.handlers.armada [-] Disabling server validation certs to extract charts
2019-05-28 19:50:28.228 41 INFO armada.handlers.armada [-] Downloading tarball from: http://controller:8080/helm_charts/stx-platform/ceph-pools-audit-0.1.0.tgz
2019-05-28 19:50:28.229 41 WARNING armada.handlers.armada [-] Disabling server validation certs to extract charts
2019-05-28 19:50:28.262 41 DEBUG armada.handlers.tiller [-] Tiller ListReleases() with timeout=300, request=limit: 32
status_codes: UNKNOWN
status_codes: DEPLOYED
status_codes: DELETED
status_codes: DELETING
status_codes: FAILED
status_codes: PENDING_INSTALL
status_codes: PENDING_UPGRADE
status_codes: PENDING_ROLLBACK
 get_results /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:210
2019-05-28 19:50:28.293 41 INFO armada.handlers.armada [-] Processing ChartGroup: starlingx-ceph-charts (StarlingX Ceph Charts), sequenced=True
2019-05-28 19:50:28.293 41 INFO armada.handlers.chart_deploy [-] [chart=rbd-provisioner]: Processing Chart, release=stx-rbd-provisioner
2019-05-28 19:50:28.294 41 INFO armada.handlers.chart_deploy [-] [chart=rbd-provisioner]: known: [], release_name: stx-rbd-provisioner
2019-05-28 19:50:28.295 41 INFO armada.handlers.chartbuilder [-] [chart=rbd-provisioner]: Building dependency chart helm-toolkit for release rbd-provisioner.
2019-05-28 19:50:28.319 41 INFO armada.handlers.chart_deploy [-] [chart=rbd-provisioner]: Installing release stx-rbd-provisioner in namespace kube-system, wait=True, timeout=1800s
2019-05-28 19:50:28.324 41 INFO armada.handlers.tiller [-] [chart=rbd-provisioner]: Helm install release: wait=True, timeout=1800
2019-05-28 19:51:25.695 41 INFO armada.handlers.chart_deploy [-] [chart=rbd-provisioner]: Install completed with results from Tiller: {'release': 'stx-rbd-provisioner', 'namespace': 'kube-system', 'status': 'DEPLOYED', 'description': 'Install complete', 'version': 1}
2019-05-28 19:51:25.695 41 INFO armada.handlers.wait [-] [chart=rbd-provisioner]: Waiting for resource type=job, namespace=kube-system labels=app=rbd-provisioner for 1743s (k8s wait 1 times, sleep 1s)
2019-05-28 19:51:25.696 41 DEBUG armada.handlers.wait [-] [chart=rbd-provisioner]: Starting to wait on: namespace=kube-system, resource type=job, label_selector=(app=rbd-provisioner), timeout=1743 _watch_resource_completions /usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:273
2019-05-28 19:51:25.720 41 DEBUG armada.handlers.wait [-] [chart=rbd-provisioner]: Skipping wait, no job resources found. _watch_resource_completions /usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:292
2019-05-28 19:51:25.720 41 INFO armada.handlers.wait [-] [chart=rbd-provisioner]: Waiting for resource type=pod, namespace=kube-system labels=app=rbd-provisioner for 1743s (k8s wait 1 times, sleep 1s)
2019-05-28 19:51:25.720 41 DEBUG armada.handlers.wait [-] [chart=rbd-provisioner]: Starting to wait on: namespace=kube-system, resource type=pod, label_selector=(app=rbd-provisioner), timeout=1743 _watch_resource_completions /usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:273
2019-05-28 19:51:28.180 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:52:28.263 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:53:28.356 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:54:28.445 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:55:28.540 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:56:28.631 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:57:28.717 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:58:28.814 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 19:59:28.901 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:00:28.997 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:01:29.104 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:02:29.194 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:03:29.290 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:04:29.375 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:05:29.465 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:06:29.543 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:07:29.632 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:08:29.727 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:09:29.815 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:10:29.906 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:11:29.989 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:12:30.074 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:13:30.169 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:14:30.252 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:15:30.331 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:16:30.410 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:17:30.513 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:18:30.612 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:19:30.691 41 DEBUG armada.handlers.lock [-] Updating lock update_lock /usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:155
2019-05-28 20:20:28.756 41 ERROR armada.handlers.wait [-] [chart=rbd-provisioner]: Timed out waiting for pods (namespace=kube-system, labels=(app=rbd-provisioner)). None found! Are `wait.labels` correct? Does `wait.resources` need to exclude %s?
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada [-] Chart deploy [rbd-provisioner] failed: armada.exceptions.k8s_exceptions.KubernetesWatchTimeoutException: Timed out waiting for pods (namespace=kube-system, labels=(app=rbd-provisioner)). None found! Are `wait.labels` correct? Does `wait.resources` need to exclude {}?
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada Traceback (most recent call last):
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 222, in handle_result
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada     result = get_result()
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 233, in <lambda>
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada     if (handle_result(chart, lambda: deploy_chart(chart))):
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 211, in deploy_chart
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada     prefix, known_releases)
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/chart_deploy.py", line 232, in execute
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada     chart_wait.wait(timer)
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 104, in wait
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada     wait.wait(timeout=timeout)
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 244, in wait
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada     raise k8s_exceptions.KubernetesWatchTimeoutException(error)
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada armada.exceptions.k8s_exceptions.KubernetesWatchTimeoutException: Timed out waiting for pods (namespace=kube-system, labels=(app=rbd-provisioner)). None found! Are `wait.labels` correct? Does `wait.resources` need to exclude {}?
2019-05-28 20:20:28.756 41 ERROR armada.handlers.armada
2019-05-28 20:20:28.796 41 ERROR armada.handlers.armada [-] Chart deploy(s) failed: ['rbd-provisioner']
2019-05-28 20:20:29.776 41 INFO armada.handlers.lock [-] Releasing lock
2019-05-28 20:20:29.787 41 ERROR armada.cli [-] Caught internal exception: armada.exceptions.armada_exceptions.ChartDeployException: Exception deploying charts: ['rbd-provisioner']
2019-05-28 20:20:29.787 41 ERROR armada.cli Traceback (most recent call last):
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/local/lib/python3.6/dist-packages/armada/cli/__init__.py", line 39, in safe_invoke
2019-05-28 20:20:29.787 41 ERROR armada.cli     self.invoke()
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 209, in invoke
2019-05-28 20:20:29.787 41 ERROR armada.cli     resp = self.handle(documents, tiller)
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py", line 63, in func_wrapper
2019-05-28 20:20:29.787 41 ERROR armada.cli     return future.result()
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2019-05-28 20:20:29.787 41 ERROR armada.cli     return self.__get_result()
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2019-05-28 20:20:29.787 41 ERROR armada.cli     raise self._exception
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2019-05-28 20:20:29.787 41 ERROR armada.cli     result = self.fn(*self.args, **self.kwargs)
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 252, in handle
2019-05-28 20:20:29.787 41 ERROR armada.cli     return armada.sync()
2019-05-28 20:20:29.787 41 ERROR armada.cli   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 249, in sync
2019-05-28 20:20:29.787 41 ERROR armada.cli     raise armada_exceptions.ChartDeployException(failures)
2019-05-28 20:20:29.787 41 ERROR armada.cli armada.exceptions.armada_exceptions.ChartDeployException: Exception deploying charts: ['rbd-provisioner']
2019-05-28 20:20:29.787 41 ERROR armada.cli