Comment 0 for bug 1859641

Brief Description
-----------------
During the initial regular system setup, apply stx-openstack application failed. armada/stx-openstack-apply log shows that there a time out for waiting for the condition "grpc_status".

And the mariadb-ingress pods are not ready:
mariadb-ingress-5bb8b69fc8-czb7z 0/1 Running 0 5h18m dead:beef::8e22:765f:6121:eb4e controller-0 <none> <none>
mariadb-ingress-5bb8b69fc8-h8zx5 0/1 Running 0 5h18m dead:beef::a4ce:fec1:5423:e306 controller-1 <none> <none>

Severity
--------
Critical

Steps to Reproduce
------------------
install a regular system
apply stx-openstack application

TC-name: installation

Expected Behavior
------------------
applied

Actual Behavior
----------------
apply failed

Reproducibility
---------------
Unknown - first time this is seen it

System Configuration
--------------------
Multi-node system

Lab-name: WCp_71-75

Branch/Pull Time/Commit
-----------------------
20200111T023000Z

Last Pass
---------
unknown

Timestamp/Logs
--------------
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller [-] [chart=openstack-mariadb]: Error while installing release osh-openstack-mariadb: grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "release osh-openstack-mariadb failed: timed out waiting for the condition"
        debug_error_string = "{"created":"@1578993242.085407734","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"release osh-openstack-mariadb failed: timed out waiting for the condition","grpc_status":2}"
>
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller Traceback (most recent call last):
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py", line 473, in install_release
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller metadata=self.metadata)
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 533, in __call__
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller return _end_unary_response_blocking(state, call, False, None)
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller raise _Rendezvous(state, None, None, deadline)
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller status = StatusCode.UNKNOWN
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller details = "release osh-openstack-mariadb failed: timed out waiting for the condition"
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller debug_error_string = "{"created":"@1578993242.085407734","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"release osh-openstack-mariadb failed: timed out waiting for the condition","grpc_status":2}"
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller >
2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller ^[[00m
2020-01-14 09:14:02.087 458 DEBUG armada.handlers.tiller [-] [chart=openstack-mariadb]: Helm getting release status for release=osh-openstack-mariadb, version=0 get_release_status /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:539^[[00m
2020-01-14 09:14:02.230 458 DEBUG armada.handlers.tiller [-] [chart=openstack-mariadb]: GetReleaseStatus= name: "osh-openstack-mariadb"
info {
  status {
    code: FAILED
  }
  first_deployed {
    seconds: 1578991441
    nanos: 428638329
  }
  last_deployed {
    seconds: 1578991441
    nanos: 428638329
  }
  Description: "Release \"osh-openstack-mariadb\" failed: timed out waiting for the condition"
}

Test Activity
-------------
installation