cluster SST fail - /usr/bin/du: Argument list too long
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
MySQL patches by Codership | Status tracked in 5.6 | |||||
5.5 |
Fix Committed
|
Medium
|
Alex Yurchenko | |||
5.6 |
Fix Released
|
Medium
|
Alex Yurchenko | |||
Percona XtraDB Cluster moved to https://jira.percona.com/projects/PXC | Status tracked in 5.6 | |||||
5.5 |
Fix Released
|
Undecided
|
Unassigned | |||
5.6 |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
Hi Guys,
we have quite a lot of tables and databases (1000DB, each with 60tables).
When I try to add a new node to the cluster I receive an error:
WSREP_SST: [INFO] Evaluating xbstream -c ${INFO_FILE} | socat -u stdio TCP:10.
/usr//bin/
WSREP_SST: [INFO] Sleeping before data transfer for SST (20140319 02:22:25.456)
WSREP_SST: [INFO] Streaming the backup to joiner at 10.100.2.3 4444 (20140319 02:22:35.463)
WSREP_SST: [INFO] Evaluating innobackupex --defaults-
WSREP_SST: [ERROR] innobackupex finished with error: 1. Check /var/lib/
WSREP_SST: [ERROR] Cleanup after exit with status:22 (20140319 02:22:35.702)
140319 2:22:35 [ERROR] WSREP: Failed to read from: wsrep_sst_
140319 2:22:35 [ERROR] WSREP: Process completed with error: wsrep_sst_
140319 2:22:35 [Warning] WSREP: 0 (sql1002): State transfer to 1 (sql1003) failed: -1 (Operation not permitted)
140319 2:22:35 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 1402150009)
The problematic code line is:
payload=$(du --block-size=1 -c **/*.ibd **/*.MYI **/*.MYI ibdata1 | awk 'END { print $1 }')
When I try to run it manualy at mysql data directory I receive the same error.
root@sql1002:
-bash: /usr/bin/du: Argument list too long
root@sql1002:
I believe it can be fixed by replacing the line with this command:
payload=$(find . -name "*.ibd" -name "*.MYI" -name "ibdata1" -type f -print0 | xargs -0 du --block-size=1 -c | awk 'END { print $1 }')
tags: | added: sst xtrabackup |
Yes, it can hit this bug when the number of files matched by " **/*.ibd **/*.MYI **/*.MYI ibdata1 " is really high.
Also, this bug is hit only when 'progress' sst option is used.
Using find should help I guess.