Looking at the logs, we see the mariadb-server-1 logs ending here: {"log":"2019-02-20 01:50:51,545 - OpenStack-Helm Mariadb - INFO - Updating grastate configmap\n","stream":"stderr","time":"2019-02-20T01:50:51.54616385Z"} {"log":"2019-02-20 01:51:01,531 - OpenStack-Helm Mariadb - INFO - Cluster info has been uptodate 0 times out of the required 12\n","stream":"stderr","time":"2019-02-20T01:51:01.531861929Z"} {"log":"2019-02-20 01:51:01,531 - OpenStack-Helm Mariadb - INFO - Checking to see if cluster data is fresh\n","stream":"stderr","time":"2019-02-20T01:51:01.531891753Z"} {"log":"2019-02-20 01:51:01,568 - OpenStack-Helm Mariadb - INFO - Updating grastate configmap\n","stream":"stderr","time":"2019-02-20T01:51:01.568883913Z"} The mariadb-server-0 logs show another story: {"log":"2019-02-20 01:50:58,967 - OpenStack-Helm Mariadb - INFO - The data we have from the cluster is ok for node mariadb-server-0\n","stream":"stderr","time":"2019-02-20T01:50:58.967962473Z"} {"log":"2019-02-20 01:51:08,978 - OpenStack-Helm Mariadb - INFO - Cluster info has been uptodate 1 times out of the required 12\n","stream":"stderr","time":"2019-02-20T01:51:08.978298167Z"} {"log":"2019-02-20 01:51:08,978 - OpenStack-Helm Mariadb - INFO - Checking to see if cluster data is fresh\n","stream":"stderr","time":"2019-02-20T01:51:08.97833839Z"} {"log":"2019-02-20 01:51:08,979 - OpenStack-Helm Mariadb - INFO - Updating grastate configmap\n","stream":"stderr","time":"2019-02-20T01:51:08.979132811Z"} {"log":"2019-02-20 01:51:21,870 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /api/v1/namespaces/openst ack/configmaps/osh-openstack-mariadb-mariadb-state\n","stream":"stderr","time":"2019-02-20T01:51:21.870759801Z"} {"log":"2019-02-20 01:51:21,870 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /api/v1/namespaces/openst ack/configmaps/osh-openstack-mariadb-mariadb-state\n","stream":"stderr","time":"2019-02-20T01:51:21.871024039Z"} {"log":"2019-02-20 01:51:21,882 - OpenStack-Helm Mariadb - INFO - The data we have from the cluster is too old to make a decision for node mariadb-server-1\n","stream":"stderr","time":"2019-02-20T01:51:21.883047989Z"} {"log":"2019-02-20 01:51:21,883 - OpenStack-Helm Mariadb - INFO - The data we have from the cluster is too old to make a decision for node mariadb-server-0\n","stream":"stderr","time":"2019-02-20T01:51:21.88352455Z"} {"log":"2019-02-20 01:51:31,893 - OpenStack-Helm Mariadb - INFO - Cluster info has been uptodate 0 times out of the required 12\n","stream":"stderr","time":"2019-02-20T01:51:31.893865638Z"} {"log":"2019-02-20 01:51:31,893 - OpenStack-Helm Mariadb - INFO - Checking to see if cluster data is fresh\n","stream":"stderr","time":"2019-02-20T01:51:31.893908681Z"} {"log":"2019-02-20 01:51:31,898 - OpenStack-Helm Mariadb - INFO - The data we have from the cluster is too old to make a decision for node mariadb-server-1\n","stream":"stderr","time":"2019-02-20T01:51:31.898558239Z"} {"log":"2019-02-20 01:51:31,898 - OpenStack-Helm Mariadb - INFO - The data we have from the cluster is too old to make a decision for node mariadb-server-0\n","stream":"stderr","time":"2019-02-20T01:51:31.898631348Z"} {"log":"2019-02-20 01:51:31,909 - OpenStack-Helm Mariadb - INFO - Updating grastate configmap\n","stream":"stderr","time":"2019-02-20T01:51:31.909790831Z"} {"log":"2019-02-20 01:51:41,908 - OpenStack-Helm Mariadb - INFO - Cluster info has been uptodate 0 times out of the required 12\n","stream":"stderr","time":"2019-02-20T01:51:41.908430614Z"} {"log":"2019-02-20 01:51:41,908 - OpenStack-Helm Mariadb - INFO - Checking to see if cluster data is fresh\n","stream":"stderr","time":"2019-02-20T01:51:41.908466489Z"} {"log":"2019-02-20 01:51:41,912 - OpenStack-Helm Mariadb - INFO - The data we have from the cluster is too old to make a decision for node mariadb-server-1\n","stream":"stderr","time":"2019-02-20T01:51:41.912964648Z"} {"log":"2019-02-20 01:51:41,912 - OpenStack-Helm Mariadb - INFO - The data we have from the cluster is ok for node mariadb-server-0\n","stream":"stderr","time":"2019-02-20T01:51:41.913080719Z"} {"log":"2019-02-20 01:51:41,931 - OpenStack-Helm Mariadb - INFO - Updating grastate configmap\n","stream":"stderr","time":"2019-02-20T01:51:41.931925188Z"} We see a broken connection at 2019-02-20 01:51:21, after which only mariadb-server-0 is updating its state and mariadb-server-1 appears hung.