Activity log for bug #1859641

Date Who What changed Old value New value Message
2020-01-14 16:11:51 Peng Peng bug added bug
2020-01-14 16:18:22 Yang Liu summary stx-openstack app apply failed by time out waiting for the condition "grpc_status" IPv6: stx-openstack app apply failed by time out waiting for the condition "grpc_status"
2020-01-14 16:19:56 Yang Liu description Brief Description ----------------- During the initial regular system setup, apply stx-openstack application failed. armada/stx-openstack-apply log shows that there a time out for waiting for the condition "grpc_status". And the mariadb-ingress pods are not ready: mariadb-ingress-5bb8b69fc8-czb7z 0/1 Running 0 5h18m dead:beef::8e22:765f:6121:eb4e controller-0 <none> <none> mariadb-ingress-5bb8b69fc8-h8zx5 0/1 Running 0 5h18m dead:beef::a4ce:fec1:5423:e306 controller-1 <none> <none> Severity -------- Critical Steps to Reproduce ------------------ install a regular system apply stx-openstack application TC-name: installation Expected Behavior ------------------ applied Actual Behavior ---------------- apply failed Reproducibility --------------- Unknown - first time this is seen it System Configuration -------------------- Multi-node system Lab-name: WCp_71-75 Branch/Pull Time/Commit ----------------------- 20200111T023000Z Last Pass --------- unknown Timestamp/Logs -------------- 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller [-] [chart=openstack-mariadb]: Error while installing release osh-openstack-mariadb: grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: status = StatusCode.UNKNOWN details = "release osh-openstack-mariadb failed: timed out waiting for the condition" debug_error_string = "{"created":"@1578993242.085407734","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"release osh-openstack-mariadb failed: timed out waiting for the condition","grpc_status":2}" > 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller Traceback (most recent call last): 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py", line 473, in install_release 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller metadata=self.metadata) 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 533, in __call__ 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller return _end_unary_response_blocking(state, call, False, None) 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller raise _Rendezvous(state, None, None, deadline) 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller status = StatusCode.UNKNOWN 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller details = "release osh-openstack-mariadb failed: timed out waiting for the condition" 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller debug_error_string = "{"created":"@1578993242.085407734","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"release osh-openstack-mariadb failed: timed out waiting for the condition","grpc_status":2}" 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller > 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller ^[[00m 2020-01-14 09:14:02.087 458 DEBUG armada.handlers.tiller [-] [chart=openstack-mariadb]: Helm getting release status for release=osh-openstack-mariadb, version=0 get_release_status /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:539^[[00m 2020-01-14 09:14:02.230 458 DEBUG armada.handlers.tiller [-] [chart=openstack-mariadb]: GetReleaseStatus= name: "osh-openstack-mariadb" info { status { code: FAILED } first_deployed { seconds: 1578991441 nanos: 428638329 } last_deployed { seconds: 1578991441 nanos: 428638329 } Description: "Release \"osh-openstack-mariadb\" failed: timed out waiting for the condition" } Test Activity ------------- installation Brief Description ----------------- During the initial regular system setup, apply stx-openstack application failed. armada/stx-openstack-apply log shows that there a time out for waiting for the condition "grpc_status". And the mariadb-ingress pods are not ready: mariadb-ingress-5bb8b69fc8-czb7z 0/1 Running 0 5h18m dead:beef::8e22:765f:6121:eb4e controller-0 <none> <none> mariadb-ingress-5bb8b69fc8-h8zx5 0/1 Running 0 5h18m dead:beef::a4ce:fec1:5423:e306 controller-1 <none> <none> Severity -------- Critical Steps to Reproduce ------------------ install a Ipv6 system apply stx-openstack application TC-name: installation Expected Behavior ------------------ applied Actual Behavior ---------------- - apply failed - mariadb-ingress pods are not ready: mariadb-ingress-5bb8b69fc8-czb7z 0/1 Running 0 5h18m dead:beef::8e22:765f:6121:eb4e controller-0 <none> <none> mariadb-ingress-5bb8b69fc8-h8zx5 0/1 Running 0 5h18m dead:beef::a4ce:fec1:5423:e306 controller-1 <none> <none> Reproducibility --------------- Unknown - first time this is tried on IPv6 system System Configuration -------------------- Multi-node system Lab-name: WCp_71-75 Branch/Pull Time/Commit ----------------------- 20200111T023000Z Last Pass --------- unknown Timestamp/Logs -------------- 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller [-] [chart=openstack-mariadb]: Error while installing release osh-openstack-mariadb: grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:         status = StatusCode.UNKNOWN         details = "release osh-openstack-mariadb failed: timed out waiting for the condition"         debug_error_string = "{"created":"@1578993242.085407734","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"release osh-openstack-mariadb failed: timed out waiting for the condition","grpc_status":2}" > 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller Traceback (most recent call last): 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py", line 473, in install_release 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller metadata=self.metadata) 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 533, in __call__ 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller return _end_unary_response_blocking(state, call, False, None) 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller raise _Rendezvous(state, None, None, deadline) 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with: 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller status = StatusCode.UNKNOWN 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller details = "release osh-openstack-mariadb failed: timed out waiting for the condition" 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller debug_error_string = "{"created":"@1578993242.085407734","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"release osh-openstack-mariadb failed: timed out waiting for the condition","grpc_status":2}" 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller > 2020-01-14 09:14:02.085 458 ERROR armada.handlers.tiller ^[[00m 2020-01-14 09:14:02.087 458 DEBUG armada.handlers.tiller [-] [chart=openstack-mariadb]: Helm getting release status for release=osh-openstack-mariadb, version=0 get_release_status /usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:539^[[00m 2020-01-14 09:14:02.230 458 DEBUG armada.handlers.tiller [-] [chart=openstack-mariadb]: GetReleaseStatus= name: "osh-openstack-mariadb" info {   status {     code: FAILED   }   first_deployed {     seconds: 1578991441     nanos: 428638329   }   last_deployed {     seconds: 1578991441     nanos: 428638329   }   Description: "Release \"osh-openstack-mariadb\" failed: timed out waiting for the condition" } Test Activity ------------- installation
2020-01-16 14:08:26 Yang Liu tags stx.retestneeded
2020-01-20 18:34:02 Ghada Khalil tags stx.retestneeded stx.containers stx.retestneeded
2020-01-20 18:34:30 Ghada Khalil tags stx.containers stx.retestneeded stx.containers stx.distro.openstack stx.retestneeded
2020-01-20 18:35:28 Ghada Khalil starlingx: importance Undecided High
2020-01-20 18:35:30 Ghada Khalil starlingx: status New Triaged
2020-01-20 18:35:51 Ghada Khalil starlingx: assignee yong hu (yhu6)
2020-01-20 18:40:37 Ghada Khalil tags stx.containers stx.distro.openstack stx.retestneeded stx.4.0 stx.containers stx.distro.openstack stx.retestneeded
2020-02-11 07:06:18 zhipeng liu starlingx: assignee yong hu (yhu6) zhipeng liu (zhipengs)
2020-02-18 08:43:13 zhipeng liu starlingx: status Triaged Incomplete
2020-02-20 04:50:57 Ghada Khalil bug added subscriber Joseph Richard
2020-02-20 13:47:50 Yang Liu starlingx: status Incomplete Confirmed
2020-02-26 08:22:09 zhipeng liu starlingx: status Confirmed In Progress
2020-03-31 14:36:03 OpenStack Infra tags stx.4.0 stx.containers stx.distro.openstack stx.retestneeded in-f-centos8 stx.4.0 stx.containers stx.distro.openstack stx.retestneeded
2020-03-31 14:36:04 OpenStack Infra bug watch added https://github.com/kubernetes/kubernetes/issues/80745
2020-03-31 14:36:04 OpenStack Infra bug watch added https://github.com/kubernetes/kubernetes/issues/85334
2020-04-21 14:33:05 OpenStack Infra starlingx: status In Progress Fix Released
2020-04-21 15:37:32 Peng Peng tags in-f-centos8 stx.4.0 stx.containers stx.distro.openstack stx.retestneeded in-f-centos8 stx.4.0 stx.containers stx.distro.openstack
2020-04-28 03:31:19 Yang Liu tags in-f-centos8 stx.4.0 stx.containers stx.distro.openstack in-f-centos8 stx.4.0 stx.containers stx.distro.openstack stx.retestneeded
2020-04-28 13:13:32 Bill Zvonar bug added subscriber Bill Zvonar
2020-06-04 17:19:12 Peng Peng tags in-f-centos8 stx.4.0 stx.containers stx.distro.openstack stx.retestneeded in-f-centos8 stx.4.0 stx.containers stx.distro.openstack