xtrabackup: taking backup while tables are droped and created breaks backup
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| Percona XtraBackup moved to https://jira.percona.com/projects/PXB |
Fix Released
|
Critical
|
Alexey Kopytov | |
| 1.6 |
Fix Released
|
Critical
|
Alexey Kopytov | |
| Trunk |
Fix Released
|
Critical
|
Alexey Kopytov |
Bug Description
Hi,
taking a backup with innobackup/
with:
InnoDB: Error: tablespace id is 849127 in the data dictionary
InnoDB: but in file ./magento/
110221 12:17:52 InnoDB: Assertion failure in thread 140051784468208 in file fil/fil0fil.c line 726
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://
InnoDB: about forcing recovery.
innobackupex-1.5.1: Error: ibbackup child process has died at ./innobackupex-
The table catalog_
So innodbacup/
Regards
Erkan
Using latest stable 1.4 even if verson reports:
./innobackupex-
Version string '' contains invalid data; ignoring: '' at ./innobackupex-
InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy.
All Rights Reserved.
Related branches
- Stewart Smith (community): Approve on 2011-09-21
-
Diff: 49 lines (+22/-0)1 file modifiedxtrabackup.c (+22/-0)
- Stewart Smith (community): Approve on 2011-09-15
-
Diff: 2266 lines (+476/-254)17 files modifiedpatches/innodb51.patch (+42/-41)
patches/innodb51_builtin.patch (+113/-37)
patches/innodb55.patch (+39/-35)
patches/xtradb51.patch (+38/-36)
patches/xtradb55.patch (+38/-36)
test/inc/common.sh (+9/-0)
test/t/bug722638.sh (+119/-0)
test/t/bug723097.sh (+2/-2)
test/t/bug810269.sh (+2/-4)
test/t/ib_csm_csv.sh (+4/-4)
test/t/ib_incremental.sh (+2/-2)
test/t/xb_export.sh (+3/-3)
test/t/xb_incremental.sh (+2/-2)
test/t/xb_incremental_compressed.sh (+2/-4)
test/t/xb_part_range.sh (+2/-2)
test/t/xb_partial.sh (+2/-2)
xtrabackup.c (+57/-44)
Changed in percona-xtrabackup: | |
assignee: | nobody → Valentine Gostev (core-longbow) |
Changed in percona-xtrabackup: | |
importance: | Undecided → High |
David Corley (david-corley) wrote : | #1 |
Peter Zaitsev (pz-percona) wrote : | #2 |
Can you try with Xtrabackup 1.5
It is called beta but it has a lot of bugs fixed
David Corley (david-corley) wrote : | #3 |
Hey Peter,
I tried the latest beta but the issue is still present.
David Corley (david-corley) wrote : | #4 |
I tried the newly released 1.6 version, and the issue remains.
Changed in percona-xtrabackup: | |
status: | New → Confirmed |
Changed in percona-xtrabackup: | |
importance: | High → Critical |
status: | Confirmed → Triaged |
assignee: | Valentine Gostev (longbow) → nobody |
Changed in percona-xtrabackup: | |
milestone: | none → 1.7 |
Vadim Tkachenko (vadim-tk) wrote : | #5 |
To clarify - the problem appears only with innodb_
If innodb_
juliet draper (juliet-draper) wrote : | #6 |
Is there a patch for this? I can't change out file per table settings and I would love to use xtrabackup!
Charl (charlretief) wrote : | #7 |
I reported a new issue related to this bug #826604. It is however not a duplicate.
Alexey Kopytov (akopytov) wrote : | #8 |
The root cause of the problem is that xtrabackup iterates .ibd files
based on a data dictionary snapshot taken at the very start of a
backup process. If a tablespace id changes due to dropping and
recreating the table with the same name, or an ALTER TABLE that
requires a table copy, or TRUNCATE (if it is performed as DROP +
CREATE TABLE), InnoDB complains about id mismatch, and crashes with an
assertion failure.
The idea of the fix is to ignore tablespaces with wrong ids at the
backup stage. They will be (re)created when doing recovery at the prepare
stage, since the corresponding delete/create log records will be
replayed on recovery.
All updates to the "old" version(s) of the tablespace (i.e. any log
records before the table was recreated) will be ignored on recovery,
because buf_read_
the tablespace with a specified id.
Originally I thought that table renaming needs special handling,
because we may end up with multiple tablespace with the same id, if
renaming happens after the source tablespace has been
copied. However, that can never be the case with xtrabackup because
it considers only tablespaces that were present at start.
So what needs to be done is to modify fil_node_
XtraBackup patches to InnoDB/XtraDB source to signal id mismatches
back to the caller (possibly with a warning) rather than crash with
an error, similarly to how OS_FILE_NOT_FOUND is ignored in the fix for
bug #713799. The warning message in xtrabackup.c has to be adjusted
to take this new case into account.
The fix has to be tested manually, because the xtrabackup test suite
does not have a DEBUG_SYNC analog which is required in this case to
synchronize file copying by xtrabackup and DDL operations in the
server.
Stewart Smith (stewart) wrote : Re: [Bug 722638] Re: xtrabackup: taking backup while tables are droped and created breaks backup | #9 |
On Wed, 07 Sep 2011 17:15:41 -0000, Alexey Kopytov <email address hidden> wrote:
> The fix has to be tested manually, because the xtrabackup test suite
> does not have a DEBUG_SYNC analog which is required in this case to
> synchronize file copying by xtrabackup and DDL operations in the
> server.
What about some perl and gdb?
--
Stewart Smith
Changed in percona-xtrabackup: | |
assignee: | nobody → Alexey Kopytov (akopytov) |
status: | Triaged → In Progress |
Changed in percona-xtrabackup: | |
status: | In Progress → Fix Committed |
Changed in percona-xtrabackup: | |
status: | Fix Committed → Fix Released |
Stewart Smith (stewart) wrote : | #10 |
As mentioned in https:/
"We should also document that the fix for bug #722638 may not work for old MySQL or Percona Server releases. If this limitation proves to be problematic, we may invest more time into fixing it."
Vojtech Kurka (vojtech-kurka) wrote : | #11 |
We are facing similar problem when renaming InnoDB tables in cycle:
-- versions -------
mysql --version
mysql Ver 14.14 Distrib 5.5.23, for Linux (x86_64) using readline 5.1
xtrabackup_55 -v
xtrabackup_55 version 2.0.3 for Percona Server 5.5.16 Linux (x86_64) (revision id: 470)
-- The problem occurs when applying log ------------
innobackupex --apply-log --use-memory=48GB /data/db_restore
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Error: trying to add tablespace 2788 of name './db2/
InnoDB: to the tablespace memory cache, but tablespace
InnoDB: 2788 of name './db2/
InnoDB: memory cache!
innobackupex: Error:
innobackupex: ibbackup failed at /usr/bin/
-- -------
The problem occurs always when this RENAME TABLE is done while the backup is being taken:
RENAME TABLE `ShopOfferAvgPo
Percona now uses JIRA for bug reports so this bug report is migrated to: https:/
We're getting hit by this one too. 5.1.47 with XtraBackup 1.4 rev 193