Comment 7 for bug 1611728

Hi Danny,

I was able to reproduce something similar which hints toward the problem you are facing.

For sure you are using XB with backup locks disabled either by use of --no-backup-locks or FORCE_FTWRL=1 and I see we hit this scenario only when XB tries to do full FLUSH TABLE WITH READ LOCK.

(As per the updated FLUSH TABLE WITH READ LOCK desync+pause the node which is already done as part of SST flow and there probably is some race in that code path)

-------------------------------
From your log snippet:

WSREP_SST: [INFO] Evaluating innobackupex --defaults-file=/etc/my.cnf --defaults-group=mysqld --no-version-check ***********--no-backup-locks************ $tmpopts $INNOEXTRA --galera-info --stream=$sfmt $itmpdir 2>${DATA}/innobackup.backup.log | socat -u stdio TCP:192.168.1.24:4444; RC=( ${PIPESTATUS[@]} ) (20160810 05:18:28.695)
------------------------------

What is the quick workaround:
* Avoid suppressing backup-lock. As I understand backup locks were unstable before but they are good now. I am still running some local test to ensure that we can get rid of the limitation added to PXC documents.

"Backup locks used during SST or with Xtrabackup can crash, on donor, either use inno-backup-opts=’–no-backup-locks’ under [sst] in my.cnf or set FORCE_FTWRL=1 in /etc/sysconfig/mysql (or /etc/sysconfig/mysql.%i for corresponding unit/service) for CentOS/RHEL or /etc/default/mysql in debian/ubuntu. You can also use rsync as the alternate SST method if those don’t work."