Failed to prepare full backup (size == space->size_in_header)

[Shahriyar Rzayev]

Steps are:

mysql> show create table sbtest1\G
*************************** 1. row ***************************
       Table: sbtest1
Create Table: CREATE TABLE `sbtest1` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `k` int(10) unsigned NOT NULL DEFAULT '0',
  `c` char(120) NOT NULL DEFAULT '',
  `pad` char(60) NOT NULL DEFAULT '',
  PRIMARY KEY (`id`),
  KEY `k_1` (`k`)
) ENGINE=InnoDB AUTO_INCREMENT=1464179 DEFAULT CHARSET=latin1 COMPRESSION='lz4'
1 row in set (0,00 sec)

Running with:

sysbench --test=/usr/share/doc/sysbench/tests/db/insert.lua --oltp-test-mode=complex --num-threads=1000 --mysql-db=dbtest --mysql-user=root --mysql-password=Baku12345# --db-driver=mysql --max-requests=0 run

and:

sysbench --test=/usr/share/doc/sysbench/tests/db/oltp.lua --oltp-test-mode=complex --num-threads=1000 --mysql-db=dbtest --mysql-user=root --mysql-password=Baku12345# --db-driver=mysql --max-requests=0 run

Take backup:

./xtrabackup --defaults-file=/etc/my.cnf --backup --datadir=/var/lib/mysql/ --target-dir=/home/backup_dir/full/ --user=root --password=Baku12345# --no-version-check

Prepare:

./xtrabackup --defaults-file=/home/backup_dir/full/backup-my.cnf --prepare --target-dir=/home/backup_dir/full
.
.
.
InnoDB: 96 redo rollback segment(s) found. 1 redo rollback segment(s) are active.
InnoDB: 32 non-redo rollback segment(s) are active.
InnoDB: Waiting for purge to start
2016-01-19 06:19:15 0x7f01897f4700 InnoDB: Assertion failure in thread 139644578514688 in file fsp0fsp.cc line 2986
InnoDB: Failing assertion: size == space->size_in_header
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
11:19:15 UTC - xtrabackup got signal 6 ;
This could be because you hit a bug or data is corrupted.
This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
InnoDB: 5.7.10 started; log sequence number 11766603797
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
InnoDB: not started
InnoDB: FTS optimize thread exiting.
InnoDB: Starting shutdown...
stack_bottom = 0 thread_stack 0x10000
./xtrabackup(my_print_stacktrace+0x35)[0x192777d]
./xtrabackup(handle_fatal_signal+0x2ab)[0x11a3371]
/lib64/libpthread.so.0(+0xf100)[0x7f0199bc4100]
/lib64/libc.so.6(gsignal+0x37)[0x7f01981285f7]
/lib64/libc.so.6(abort+0x148)[0x7f0198129ce8]
./xtrabackup[0x111e83f]
./xtrabackup(_Z24fsp_reserve_free_extentsPmmm13fsp_reserve_tP5mtr_tm+0xfa)[0xebd850]
./xtrabackup(_Z26btr_cur_pessimistic_deleteP7dberr_tmP9btr_cur_tmbP5mtr_t+0x29d)[0xe1094f]
./xtrabackup[0x1072b39]
./xtrabackup[0x1073257]
./xtrabackup[0x10733a6]
./xtrabackup[0x1073e79]
./xtrabackup[0x10740aa]
./xtrabackup(_Z14row_purge_stepP9que_thr_t+0x17f)[0x107436a]
./xtrabackup[0x10038b9]
./xtrabackup[0x1003ab8]
./xtrabackup(_Z15que_run_threadsP9que_thr_t+0x92)[0x1003c6e]
./xtrabackup(_Z9trx_purgemmb+0x2de)[0x10ebaff]
./xtrabackup[0x10bdc79]
./xtrabackup(srv_purge_coordinator_thread+0x211)[0x10be333]
/lib64/libpthread.so.0(+0x7dc5)[0x7f0199bbcdc5]
/lib64/libc.so.6(clone+0x6d)[0x7f01981e921d]

[Shahriyar Rzayev (rzayev-sehriyar)]

This assertion does not apply to incremental backups.
I am able to take full backup + 1 incremental backup and prepare successfully.

[Sergei Glushchenko (sergei.glushchenko)]

https://github.com/percona/percona-xtrabackup/pull/152

[Sergei Glushchenko (sergei.glushchenko)]

Prepare fails the same way without compression:

InnoDB: 96 redo rollback segment(s) found. 1 redo rollback segment(s) are active.
InnoDB: 32 non-redo rollback segment(s) are active.
InnoDB: Waiting for purge to start
InnoDB: 5.7.10 started; log sequence number 452498965
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
InnoDB: not started
InnoDB: FTS optimize thread exiting.
InnoDB: Starting shutdown...
2016-01-21 05:22:29 0x7efdd8bf7700 InnoDB: Assertion failure in thread 139628728252160 in file fsp0fsp.cc line 2986
InnoDB: Failing assertion: size == space->size_in_header
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
05:22:29 UTC - xtrabackup got signal 6 ;
This could be because you hit a bug or data is corrupted.
This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x10000
../src/xtrabackup(my_print_stacktrace+0x35)[0x14a5e1d]
../src/xtrabackup(handle_fatal_signal+0x2ab)[0x135db2d]
/lib64/libpthread.so.0(+0xf130)[0x7efde8fbf130]
/lib64/libc.so.6(gsignal+0x37)[0x7efde75245d7]
/lib64/libc.so.6(abort+0x148)[0x7efde7525cc8]
../src/xtrabackup[0xf2edef]
../src/xtrabackup(_Z24fsp_reserve_free_extentsPmmm13fsp_reserve_tP5mtr_tm+0xfa)[0xe22968]
../src/xtrabackup(_Z26btr_cur_pessimistic_deleteP7dberr_tmP9btr_cur_tmbP5mtr_t+0x29d)[0xe6e10b]
../src/xtrabackup[0x151823d]
../src/xtrabackup[0x151895b]
../src/xtrabackup[0x1518ccc]
../src/xtrabackup[0x151965e]
../src/xtrabackup[0x15197ae]
../src/xtrabackup(_Z14row_purge_stepP9que_thr_t+0x17f)[0x1519a6e]
../src/xtrabackup[0xf161a5]
../src/xtrabackup[0xf163a4]
../src/xtrabackup(_Z15que_run_threadsP9que_thr_t+0x92)[0xf1655a]
../src/xtrabackup(_Z9trx_purgemmb+0x2de)[0x10b53bb]
../src/xtrabackup(srv_purge_coordinator_thread+0x2d0)[0xfcc49a]
/lib64/libpthread.so.0(+0x7df5)[0x7efde8fb7df5]
/lib64/libc.so.6(clone+0x6d)[0x7efde75e51ad]

Please report a bug at https://bugs.launchpad.net/percona-xtrabackup

Assertion is DEBUG only

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-458