new InnoDB table is not included to full prepared backup from incremental

[Valentine Gostev]

After taking a full backup, we create an InnoDB table in its own table space. If we then take incremental backup, finally prepared backup will not contain table created after full and before incremental backup.

[Vadim Tkachenko (vadim-tk)]

Valentine,

Do you have repeatable case ?

As I understand XtraBackup should handle such case.

[Valentine Gostev (longbow)]

Vadim, linked branch contains test test/t/xb_inc_tbl_own.sh which indicates that feature works fine, but if we run the steps from test case manually, we do not get newly created table in full backup dir after applying incremental changes.

Then we discussed and after further investigation it was found that while running test case xtrabackup calls "open("./inctest/t2.ibd", O_RDWR|O_CREAT|O_EXCL, 0664) = 11", but when the same manually "open("./inctest/t2.ibd", O_RDWR|O_CREAT|O_EXCL, 0664) = 11" is not called.

[Alexey Kopytov (akopytov)]

Attaching the script which reproduces the bug. This is what I would actually like to have as a test case.

[Alexey Kopytov (akopytov)]

The problem is that when a new table is added after taking a full backup *and* log files have been flushed before an incremental one, the new table data goes to a .delta file. When trying to apply that .delta to a full backup later, xtrabackup_apply_delta() fails because the corresponding data file cannot be found.

The difference with the situation when log files have not been flushed before taking an incremental backup is that in this case a file creation event is in the log file, so .ibd is then created when doing crash recovery and replaying the log. The corresponding .delta file in this case will be empty.

I think the fix is to explicitly create a tablespace before applying the .delta file when the corresponding .ibd is missing.

[Sachin Kale (sachin-zmanda)]

I followed the steps for restoring as mentioned in link http://www.percona.com/docs/wiki/percona-xtrabackup:xtrabackup:incremental#preparing_the_backups with xtrabackup 1.6.2 ubuntu platform.

The final directory which was created did not have "frm" file for the new table although the ibd file was present, so in all, the new table created after full backup was not restored completely.
Is this expected?

the backup directory had "<table>.frm", "<table>.ibd.delta", "<table>ibd.meta" files.

[Emil Goicovici (emy-cs)]

I have the same problem with xtrabackup 1.6.2: the .ibd file is created (thanks to the fix made for this bug) from the ".delta" file which is present in the incremental backup, but the .frm file is missing. I had to manually copy it from the incremental backup to the MySQL data folder in order to make these tables visible again after restore.

In order to reproduce it, the scenario is the following: a full backup is performed; after this, a new InnoDB table is created in the MySQL database, followed by an incremental backup. The .delta and .meta files appear in the incremental backup as expected, but then, when performing the --apply-log step to unify the base backup with the incremental one, the .frm file is NOT copied from the incremental backup folder to the base backup folder.

[Alexey Kopytov (akopytov)]

The problem with .frm files is bug #759701.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-217