--exclude=<database list> command line option

[Nick Pavey]

Hi Percona,

I've modified the innobackupex-1.5.1 tool to add the following features:

  * --exclude=<database list> command line option : Exclude certain databases from copying of .frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ and .opt files
  * Added code to tolerate race condition in copying of .frm etc files
  * Added additional output to display more information on backups which don't complete 100% properly
  * minor code restructuring for readability

Given that this is open source, I thought I'd submit my modifications back to you, in case you feel like including them in a subsequent release.

Regarding the race condition, here's the scenario I'm trying to avoid

  1. MySQL server running during backup
  2. innobackupex in --no-lock mode, because I don't want to quiesce the database during the copy of .frm etc files
  3. In subroutine backup_files, a READDIR generates a list of all files to be copied to the backup area
  4. Copy starts, but can take a long time
  5. At some point after the READDIR but before to copy the active MySQL server receives a DROP TABLE command
  6. The associated files are removed, by the time innobackupex gets to try to do the copy
  7. innobackupex throws a fatal error, shutting down the backup

Depending on the use model of the database, this may or may not be a major issue. For us it is not, because the tables which come and go like this do not hold critical information - it's scratchpad information, essentially.

In this modification, I trap the return code for the 'cp' command (I also cover the scp and tar features, but in a different way) and make a note of which files could not be copied.

At the end of the backup process, the code displays which databases / files were either skipped (with --exclude) or were only partially copied. The exit code of innobackupex is modified so that only a 100% complete backup will return a zero exit code.

I will _try_ to attach a patch to this bug. If I can't do that, please contact me directly and I'll forward it along.

Regards,

Nick Pavey

[Vadim Tkachenko (vadim-tk)]

Assigned to core-longbow to prepare test-case

[Valentine Gostev (longbow)]

Added test case to a linked branch.

1. Patch breaks streaming backup. (test xb_stream fails)
2. While using patched innobackupex with exclude option, backup is done, but innobackupex completes with exit code '1'

[Alexey Kopytov (akopytov)]

See also bug #1272329.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-98