Some output from the log file when this is set wrong, adding for bug history purposes
150727 15:18:48 [Note] WSREP: Flow-control interval: [23, 23] 150727 15:18:48 [Note] WSREP: New cluster view: global state: 02e60dd2-20fb-11e5-b5d1-b668c5066ff4:6805821, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 3 150727 15:18:48 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 150727 15:18:48 [Note] WSREP: REPL Protocols: 7 (3, 2) 150727 15:18:48 [Note] WSREP: Service thread queue flushed. 150727 15:18:48 [Note] WSREP: Assign initial position for certification: 6805821, protocol version: 3 150727 15:18:48 [Note] WSREP: Service thread queue flushed. 150727 15:18:49 [Note] WSREP: Member 1.0 (605021-infra02_galera_container-d43cfc6c) requested state transfer from '*any*'. Selected 0.0 (605023-infra03_galera_container-bd4b49ab)(SYNCED) as donor. 150727 15:18:49 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 6805826) 150727 15:18:49 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 150727 15:18:49 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'donor' --address '172.27.236.238:4444/xtrabackup_sst' --auth 'root:5cd96ef20bdc302a5c1d0e965bfe5b1a' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2-20fb-11e5-b5d1-b668c5066ff4:6805826'' 150727 15:18:49 [Note] WSREP: sst_donor_thread signaled with 0 WSREP_SST: [INFO] Streaming with tar (20150727 15:18:49.302) WSREP_SST: [INFO] Using socat as streamer (20150727 15:18:49.304) WSREP_SST: [INFO] Streaming the backup to joiner at 172.27.236.238 4444 (20150727 15:18:49.313) WSREP_SST: [INFO] Evaluating innobackupex --defaults-file=/etc/mysql/my.cnf $INNOEXTRA --galera-info --stream=$sfmt ${TMPDIR} 2>${DATA}/innobackup.backup.log | socat -u stdio TCP:172.27.236.238:4444; RC=( ${PIPESTATUS[@]} ) (20150727 15:18:49.316) WSREP_SST: [ERROR] innobackupex finished with error: 13. Check /var/lib/mysql//innobackup.backup.log (20150727 15:18:49.511) WSREP_SST: [ERROR] Cleanup after exit with status:22 (20150727 15:18:49.514) 150727 15:18:49 [ERROR] WSREP: Failed to read from: wsrep_sst_xtrabackup --role 'donor' --address '172.27.236.238:4444/xtrabackup_sst' --auth 'root:5cd96ef20bdc302a5c1d0e965bfe5b1a' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2-20fb-11e5-b5d1-b668c5066ff4:6805826' 150727 15:18:49 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'donor' --address '172.27.236.238:4444/xtrabackup_sst' --auth 'root:5cd96ef20bdc302a5c1d0e965bfe5b1a' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2-20fb-11e5-b5d1-b668c5066ff4:6805826': 22 (Invalid argument) 150727 15:18:49 [ERROR] WSREP: Command did not run: wsrep_sst_xtrabackup --role 'donor' --address '172.27.236.238:4444/xtrabackup_sst' --auth 'root:5cd96ef20bdc302a5c1d0e965bfe5b1a' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2-20fb-11e5-b5d1-b668c5066ff4:6805826' 150727 15:18:49 [Warning] WSREP: 0.0 (605023-infra03_galera_container-bd4b49ab): State transfer to 1.0 (605021-infra02_galera_container-d43cfc6c) failed: -22 (Invalid argument)
Some output from the log file when this is set wrong, adding for bug history purposes
150727 15:18:48 [Note] WSREP: Flow-control interval: [23, 23] 20fb-11e5- b5d1-b668c5066f f4:6805821, view# 2: Primary, number of nodes: 2, my index: 0, protocol version 3 infra02_ galera_ container- d43cfc6c) requested state transfer from '*any*'. Selected 0.0 (605023- infra03_ galera_ container- bd4b49ab) (SYNCED) as donor. sst_xtrabackup --role 'donor' --address '172.27. 236.238: 4444/xtrabackup _sst' --auth 'root:5cd96ef20 bdc302a5c1d0e96 5bfe5b1a' --socket '/var/run/ mysqld/ mysqld. sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2- 20fb-11e5- b5d1-b668c5066f f4:6805826' ' file=/etc/ mysql/my. cnf $INNOEXTRA --galera-info --stream=$sfmt ${TMPDIR} 2>${DATA} /innobackup. backup. log | socat -u stdio TCP:172. 27.236. 238:4444; RC=( ${PIPESTATUS[@]} ) (20150727 15:18:49.316) mysql// innobackup. backup. log (20150727 15:18:49.511) xtrabackup --role 'donor' --address '172.27. 236.238: 4444/xtrabackup _sst' --auth 'root:5cd96ef20 bdc302a5c1d0e96 5bfe5b1a' --socket '/var/run/ mysqld/ mysqld. sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2- 20fb-11e5- b5d1-b668c5066f f4:6805826' xtrabackup --role 'donor' --address '172.27. 236.238: 4444/xtrabackup _sst' --auth 'root:5cd96ef20 bdc302a5c1d0e96 5bfe5b1a' --socket '/var/run/ mysqld/ mysqld. sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2- 20fb-11e5- b5d1-b668c5066f f4:6805826' : 22 (Invalid argument) xtrabackup --role 'donor' --address '172.27. 236.238: 4444/xtrabackup _sst' --auth 'root:5cd96ef20 bdc302a5c1d0e96 5bfe5b1a' --socket '/var/run/ mysqld/ mysqld. sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --gtid '02e60dd2- 20fb-11e5- b5d1-b668c5066f f4:6805826' infra03_ galera_ container- bd4b49ab) : State transfer to 1.0 (605021- infra02_ galera_ container- d43cfc6c) failed: -22 (Invalid argument)
150727 15:18:48 [Note] WSREP: New cluster view: global state: 02e60dd2-
150727 15:18:48 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
150727 15:18:48 [Note] WSREP: REPL Protocols: 7 (3, 2)
150727 15:18:48 [Note] WSREP: Service thread queue flushed.
150727 15:18:48 [Note] WSREP: Assign initial position for certification: 6805821, protocol version: 3
150727 15:18:48 [Note] WSREP: Service thread queue flushed.
150727 15:18:49 [Note] WSREP: Member 1.0 (605021-
150727 15:18:49 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 6805826)
150727 15:18:49 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
150727 15:18:49 [Note] WSREP: Running: 'wsrep_
150727 15:18:49 [Note] WSREP: sst_donor_thread signaled with 0
WSREP_SST: [INFO] Streaming with tar (20150727 15:18:49.302)
WSREP_SST: [INFO] Using socat as streamer (20150727 15:18:49.304)
WSREP_SST: [INFO] Streaming the backup to joiner at 172.27.236.238 4444 (20150727 15:18:49.313)
WSREP_SST: [INFO] Evaluating innobackupex --defaults-
WSREP_SST: [ERROR] innobackupex finished with error: 13. Check /var/lib/
WSREP_SST: [ERROR] Cleanup after exit with status:22 (20150727 15:18:49.514)
150727 15:18:49 [ERROR] WSREP: Failed to read from: wsrep_sst_
150727 15:18:49 [ERROR] WSREP: Process completed with error: wsrep_sst_
150727 15:18:49 [ERROR] WSREP: Command did not run: wsrep_sst_
150727 15:18:49 [Warning] WSREP: 0.0 (605023-