Percona XtraDB Cluster - HA scalable solution for MySQL

Startup script on Debian shows "Failed" too soon

Reported by Miguel Angel Nieto on 2013-01-14
44
This bug affects 7 people
Affects Status Importance Assigned to Milestone
Percona XtraDB Cluster
Undecided
Raghavendra D Prabhu

Bug Description

When I start a node it needs to get a IST or SST from other nodes. This is usually a long process, but the script shows "Failed" without waiting to the process to finish:

root@PXC3:~# /etc/init.d/mysql start
Starting MySQL (Percona XtraDB Cluster) database server: mysqld . . . . . . . . . . . . . . failed!
root@PXC3:~# mysql
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
root@PXC3:~#

...wait... some time

root@PXC3:~# mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.5.28-55 Percona XtraDB Cluster (GPL), wsrep_23.7.r3821
[...]

It is not possible to accurately determine when the SST will be
complete since it depends on data size etc.

So, the solution here is to block the COM_PING till the server is
fully up, the blocking can be at unix socket level.

This situation is similar to how innodb blocking buffer pool
restore works.

Changed in percona-xtradb-cluster:
status: New → Confirmed
Alexey Bychko (abychko) on 2013-01-21
tags: added: pkg
MentalPower (mentalpower) wrote :

We do the following in our bootstrap and initialization scripts, in case anyone can use it:

/etc/init.d/mysql start
while [[ "$(pidof mysqld)" && `clustercheck | grep 503` ]]; do
    echo "Waiting for cluster to synchronize...";
    sleep 5;
done

It obviously requires a properly configured clustercheck user.

GRANT PROCESS on *.* TO 'clustercheckuser'@'localhost' IDENTIFIED BY 'somePassword';

tags: added: init-script
Changed in percona-xtradb-cluster:
milestone: none → 5.5.30-24.8

After discussion we decided to:

a) Have a --init-pid which passes the init pid from init-script
to mysqld.

b) Whenever there is a blocking operation, mysqld sends a signal
to this script which is trapped by it. Another such signal is
sent only when first is sent.
The timeout will be adjusted accordingly based on this.

dannie (daniz) wrote :

Looking forward to test and confirm from my end.

Changed in percona-xtradb-cluster:
status: Confirmed → In Progress
assignee: nobody → Raghavendra D Prabhu (raghavendra-prabhu)
Changed in percona-xtradb-cluster:
status: In Progress → Fix Committed
Changed in percona-xtradb-cluster:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers