mariadb bootstrap unconditionally check for wsrep cluster

Bug #1623662 reported by Serguei Bezverkhi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Critical
Serguei Bezverkhi

Bug Description

On August 28th, 2016 this change c7c87909 introduced unconditional check for wsrep cluster during bootstrap process whether is ready or not and if it is not it, bootstrap fails.
In kubernetes environment wsrep driver is set to none, cluster will NEVER reach ready state hence causing mariadb bootstrap job failure in kubernetes.

This code should run ONLY for Kolla and never for Kolla Kubernetes

CLUSTER_READY=$(mysql -u root --exec="SHOW STATUS LIKE 'wsrep_cluster_status'" | grep Primary)
TIMEOUT=${DB_MAX_TIMEOUT:-60}
    while [[ -z "${CLUSTER_READY}" ]]; do
        if [[ ${TIMEOUT} -gt 0 ]]; then
            let TIMEOUT-=1
            sleep 1
        else
            exit 1

Revision history for this message
Serguei Bezverkhi (sbezverk) wrote :

In lubernetes we always get this:
()[mysql@mariadb-bootstrap-adlhn /]$ mysql -u root --exec="SHOW STATUS LIKE 'wsrep_cluster_status'"
+----------------------+--------------+
| Variable_name | Value |
+----------------------+--------------+
| wsrep_cluster_status | Disconnected |
+----------------------+--------------+
()[mysql@mariadb-bootstrap-adlhn /]$
()[mysql@mariadb-bootstrap-adlhn /]$ mysql -u root --exec="SHOW GLOBAL STATUS LIKE 'wsrep_%';"
+--------------------------+----------------------+
| Variable_name | Value |
+--------------------------+----------------------+
| wsrep_cluster_conf_id | 18446744073709551615 |
| wsrep_cluster_size | 0 |
| wsrep_cluster_state_uuid | |
| wsrep_cluster_status | Disconnected |
| wsrep_connected | OFF |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 18446744073709551615 |
| wsrep_provider_name | |
| wsrep_provider_vendor | |
| wsrep_provider_version | |
| wsrep_ready | ON |
| wsrep_thread_count | 0 |
+--------------------------+----------------------+

Changed in kolla:
assignee: nobody → Serguei Bezverkhi (sbezverk)
Revision history for this message
Ryan Hallisey (rthall14) wrote :

may be time for kolla-k8s to move to galera

Changed in kolla:
importance: Undecided → Critical
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/370440

Changed in kolla:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/370440
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=cafdbb35d916fbc2ba59191f182ed946abd4cf2e
Submitter: Jenkins
Branch: master

commit cafdbb35d916fbc2ba59191f182ed946abd4cf2e
Author: Serguei Bezverkhi <email address hidden>
Date: Wed Sep 14 17:22:26 2016 -0400

    Mariadb bootstrap - no need to check for cluster ready in Kube

    The change c7c87909 has introduced an unconditional check
    for wsrep cluster during bootstrap process whether is ready or not
    and if it is not, bootstrap fails.
    In kubernetes environment wsrep driver is set to none, cluster will
    NEVER reach ready state hence causing kubernetes mariadb bootstrap job
    failure.

    Closes-Bug: #1623662

    Change-Id: I0e6fc098861b7eeab544229d0b04a78fa498ddb9

Changed in kolla:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.