feature request: parallel rsync for sst

[Mrten]

If an SST is needed, the manual states that rsync is the fastest way to get all the data.

However, I noticed that there is only one rsync running for the whole SST, which easily gets CPU-bound on my hardware (IBM x3550 with hardware raid). When this happens, sysstat shows IO as only 10% busy.

Could the rsync be parallelized in some way that is useful for speeding up the SST?

Perhaps like this for three rsyncs at a time, for just the databases:

find /var/lib/mysql -type d -print0 | xargs -P 3 -0 rsync [rsync-options]

and a separate rsync for the files in /var/lib/mysql with --no-recurse

Should speed things up a bit...

[Mrten (bugzilla-ii)]

A first stab at a patch. This makes my SST at least four times faster, uses my disks at 80% instead of 10, but I'm not quite sure it is fully OK.

[Alex Yurchenko (ayurchen)]

That's some powerful find kung-fu! So do I get it right that you don't need any changes on the joiner side?

[Mrten (bugzilla-ii)]

The joiner side is an rsync daemon which automatically accepts >1 connections, so no, nothing to be done on the joiner side.

The patch is not working correctly BTW, needs a slash after the second rsync. I'll post an update later.

[Mrten (bugzilla-ii)]

This one's better; gets me a 202G db copied, from scratch, in 20-30 minutes. Fixes a dud error I get all the time with 'permission denied on /dev/stderr' too.

I haven't tested the error-handling of the 'parallel rsync' portion.

[Raghavendra D Prabhu (raghavendra-prabhu)]

Good to see parallel rsync. However, the other SST method
Xtrabackup - already supports parallel copying and parallel
compression among other things. However, currently I see the
options to it are hardcoded in the SST script. I would add (in a separate bug) an
option so that it sources a config file if present for the
options.

[Raghavendra D Prabhu (raghavendra-prabhu)]

Report #5 as lp:1168361

[Alex Yurchenko (ayurchen)]

Mrten, can you give Codership a non-exclusive copyright to this patch, so that we can incorporate and maintain it in our code?

[Mrten (bugzilla-ii)]

#5: Please modify as you see fit. I used rsync as I didn't get innobackupex to work (yet). Does innobackupex use multiple cores?

#7: I can release my patch as public domain, is that good enough for you? If so, consider it done.

[Alex Yurchenko (ayurchen)]

Thanks! We'll try to incorporate it by the next release.

[Raghavendra D Prabhu (raghavendra-prabhu)]

@Mrten, yes, xtrabackup which is called by innobackupex, uses multiple cores if --parallel is used.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-1331