Innobackupex hangs on fatal error in some cases

Bug #1294782 reported by Raghavendra D Prabhu on 2014-03-19
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Percona XtraBackup moved to
Fix Released
Alexey Kopytov
Fix Released
Alexey Kopytov
Fix Released
Alexey Kopytov

Bug Description has introduced a serious regression reported earlier here:

This happens when backing up an empty test database and a directory called /tmp/test already exists. Innobackupex fails here

This also hangs innobackupex and xtrabackup indefinitely.

Happens here:

 opendir(DBDIR, "$source_dir/$database");
 @list = grep(/\.(frm|isl|MYD|MYI|MAD|MAI|MRG|TRG|TRN|ARM|ARZ|CSM|CSV|opt|par)$/, readdir(DBDIR));
 closedir DBDIR;
        $file_c = @list;
        if ($file_c <= $backup_file_print_limit) {
            $print_each_file = 1;
        } else {
            print STDERR "$prefix Backing up files " .
                "'$source_dir/$database/$wildcard' ($file_c files)\n";

        if ($file_c == 0 && $option_stream) {
            # Stream/encrypt empty directories by backing up a fake empty
            # db.opt file, so that empty databases are created in the backup
            mkdir("$option_tmpdir/$database") ||
                die "Failed to create directory $option_tmpdir/$database: $!";

            open XTRABACKUP_FH, "> $option_tmpdir/$database/db.opt"
                || die "Cannot create file $option_tmpdir/db.opt: $!";
            close XTRABACKUP_FH;

            backup_file_via_stream("$option_tmpdir", "$database/db.opt");

            unlink("$option_tmpdir/$database/db.opt") ||
                die "Failed to remove file $database/db.opt: $!";
            rmdir("$option_tmpdir/$database") ||
                die "Failed to remove directory $database: $!";

 mkdir("$option_tmpdir/$database") is triggering it, instead a mkdtemp-like function can be used to create unique directory to avoid collisions (which is pretty common for a name like 'test').

Tags: pxc Edit Tag help

Related branches

Even providing a temporary directory on command line doesn't work (ie. the last option), only tmpdir argument works here, even though this is an innobackupex exception.

Currently, working around this, with tmpdir for both xtrabackup and innobackupex:

/usr/sbin/innobackupex --defaults-file=/pxc56/etc/my.cnf.local --no-version-check --tmpdir=/tmp/tmp.IhuFVZftFn --socket=/pxc56/datadir/pxc.sock --galera-info --stream=xbstream /tmp/tmp.mUBsJCxrld

However, two bugs remain in this:

a) option_tmpdir being used by innobackupex than option backup_dir (/tmp/tmp.mUBsJCxrld)

b) That innobackupex/xtrabackup hang when it fails.

Alexey Kopytov (akopytov) wrote :

The only wrong behavior that I see here is b) from comment #2, i.e. that innobackupex hangs on error.

Everything else is expected:

Conflicts with existing files in the temporary directory are always possible. XtraBackup uses --tmpdir to create a PID file, and (for streaming backups) temporary copies of xtrabackup_logfile, xtrabackup_checkpoints, backup-my.cnf, xtrabackup_binlog_info, xtrabackup_slave_info, etc, or (for --rsync) xtrabackup_rsync_pass1, xtrabackup_rsync_pass2. If naming conflicts are possible with any of those files, one has to ensure that --tmpdir points to a unique temporary directory.

The fact that --tmpdir specifies the location for temporary files is also documented.

I'm changing the title to reflect the actual problem.

summary: - Innobackupex hangs when backing up empty test database if /tmp/test
- already exists
+ Innobackupex hangs on fatal error in some cases
tags: added: pxc

Percona now uses JIRA for bug reports so this bug report is migrated to:

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers