Compressed/Uncompressed incremental backup fails on compressed full backup in Xtrabackup 2.0.0

Reported by Jaime Sicam on 2012-04-10
26
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Percona XtraBackup
Critical
Alexey Kopytov
2.0
Critical
Alexey Kopytov
2.1
Critical
Alexey Kopytov

Bug Description

After creating a full compressed backup, performing a compressed/uncompressed incremental backup does not work:

innobackupex --user=backup --password=backup --slave-info --compress /backups/

---

innobackupex --incremental /backups/incremental --user=backup --password=backup --slave-info --compress --incremental-basedir /backups/2012-04-10_00-37-00

InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
and Percona Inc 2009-2012. All Rights Reserved.

This software is published under
the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.

120410 00:44:47 innobackupex: Starting mysql with options: --password=xxxxxxxx --user='backup' --unbuffered --
120410 00:44:47 innobackupex: Connected to database with mysql child process (pid=2118)
120410 00:44:53 innobackupex: Connection to database server closed
IMPORTANT: Please check that the backup run completes successfully.
           At the end of a successful backup run innobackupex
           prints "completed OK!".

innobackupex: Using mysql Ver 14.12 Distrib 5.0.95, for redhat-linux-gnu (x86_64) using readline 5.1
innobackupex: Using mysql server version Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

innobackupex: Created backup directory /backups/incremental/2012-04-10_00-44-53
120410 00:44:53 innobackupex: Starting mysql with options: --password=xxxxxxxx --user='backup' --unbuffered --
120410 00:44:53 innobackupex: Connected to database with mysql child process (pid=2142)
120410 00:44:55 innobackupex: Connection to database server closed

120410 00:44:55 innobackupex: Starting ibbackup with command: xtrabackup_51 --backup --suspend-at-end --target-dir=/backups/incremental/2012-04-10_00-44-53 --compress --compress-threads=1 --incremental-basedir='/backups/2012-04-10_00-37-00'
innobackupex: Waiting for ibbackup (pid=2149) to suspend
innobackupex: Suspend file '/backups/incremental/2012-04-10_00-44-53/xtrabackup_suspended'

xtrabackup: Error: cannot open /backups/2012-04-10_00-37-00/xtrabackup_checkpoints
xtrabackup: error: failed to read metadata from /backups/2012-04-10_00-37-00/xtrabackup_checkpoints
innobackupex: Error: ibbackup child process has died at /usr/bin/innobackupex line 371.

Jaime Sicam (jssicam) wrote :

To add, xtrabackup_checkpoints.qp is stored in full backup directory and not xtrabackup_checkpoints which is what the incremental backup script is looking for.

Jaime Sicam (jssicam) on 2012-05-07
tags: added: i22753

ok, i tried an uncompress, and it worked fine.

i agree, that this is a documentation bug.

# litle help for unpacking files
for bf in `find . -iname "*\.qp"`; do qpress -d $bf $(dirname $bf) && rm $bf; done

Thanks, Michael

Eli Klein (elijah-aclue) wrote :

This is absolutely not just a documentation bug. This new process requires you to extract the xtrabackup_checkpoints.qp file BEFORE you can run an incremental backup based on the existing full backup. The checkpoints file should clearly NOT be compressed if the innobackupex script doesn't expect it to be compressed.

** the rest of this is not really relevant to this bug, but was addressed by michael above is another issue altogether **

Furthermore, expecting users to loop through and decompress all of the files in a single threaded fashion defeats the purpose of having innobackupex parallelized. If you can write the files to disk in a parallelized way, why not make it so you can restore from disk in a parallel way? It's almost as if the restore process was forgotten about in this case.

Alexey Kopytov (akopytov) wrote :

Eli,

I think Michael meant to comment on bug #1003783, not this one.

As to this bug, I do not consider it as a documentation issue. I'm going to fix it in 2.0.1 by omitting xtrabackup_checkpoints from compression, so that a full backup could be used for incremental backups without decompression. I agree, it doesn't make much sense to compress it.

As to the other comments, there are 2 parts to this problem:

1) qpress can actually do multi-threaded decompression (the -T switch), and it uses 2 threads by default.
2) the dependency on an external utility to decompress backups makes the restore process confusing. To add insult to injury, qpress is unfortunately not packaged by all popular Linux distributions. So in addition to 1 more step with decompressing backups before restore, one may need to download and compile qpress before using compressed backups. I'm currently considering multiple options to make it less painful, but that will probably be implemented after 2.0.1.

Simon Rycroft (sdrycroft) wrote :

I am using the 2.0.1-446.squeeze version of percona-xtrabackup and am still experiencing this issue. Is there a downloadable package that has this fix in it?

Alexey Kopytov (akopytov) wrote :

Simon,

Does 2.0.1 still create a compressed xtrabackup_checkpoints file for you? Or creating an incremental backup with a full backup created with a previous version does not work?

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers