xb_incremental_bitmap_misc.sh fails in debug xtradb56 builds | missing bitmap data at the interval start fails " bitmap_files->files[0].start_lsn == first_file_start_lsn" assertion

Reported by Alexey Kopytov on 2013-07-23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona XtraBackup
Low
Laurynas Biveinis
2.1
Low
Laurynas Biveinis

Bug Description

After updating PS version used by the test suite from PS 5.6.10 to PS 5.6.11 (i.e. the one where page tracking is available), xb_incremental_bitmap_misc.sh fails in debug xtradb56 builds as follows:

incremental backup from 9000 is enabled.
xtrabackup: uses posix_fadvise().
xtrabackup: cd to /mnt/workspace/percona-xtrabackup-2.1-param/BUILD_TYPE/debug/Host/centos5-64/xtrabackuptarget/xtradb56/test/var/w1/var1/data
xtrabackup: using the following InnoDB configuration:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 50331648
>> log scanned up to (1634133)
InnoDB: Allocated tablespace 6, old maximum was 0
2013-07-22 12:10:05 2acfb2ea2580 InnoDB: Assertion failure in thread 47071548286336 in file changed_page_bitmap.cc line 404
InnoDB: Failing assertion: bitmap_files->files[0].start_lsn == first_file_start_lsn
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
inc/common.sh: line 70: 6377 Aborted (core dumped) "$@"
2013-07-22 12:10:05: run.sh: ===> xtrabackup_56 failed with exit code 134
------------

http://jenkins.percona.com/view/XtraBackup/job/percona-xtrabackup-2.1-param/386/BUILD_TYPE=debug,Host=centos5-64,xtrabackuptarget=xtradb56/testReport/junit/%28root%29/t_xb_incremental_bitmap_misc/sh/

The test fails in the debug code of changed_page_bitmap.cc, at the last test step, where it attempts to perform a bitmap backup (LSNs 9000..infinity)) with bitmap data missing for the start of interval.

With PS 5.6.11 the bitmap files are as follows:
     /home/laurynas/percona/src/bug1204075/test/var/w1/var1/data/ib_modified_log_4_1633805.xdb
     /home/laurynas/percona/src/bug1204075/test/var/w1/var1/data/ib_modified_log_5_1634123.xdb

With PS 5.5 the bitmap files are as follows:
     /home/laurynas/percona/src/bug1204075/test/var/w1/var1/data/ib_modified_log_4_1602083.xdb
     /home/laurynas/percona/src/bug1204075/test/var/w1/var1/data/ib_modified_log_5_1602404.xdb

I.e. identical for the purposes of the test. Given that xtradb51 and xtradb55 debug configurations are disabled in Jenkins and at least the xtradb55 one does not even build, I believe this bug has been from day one and surfaced on the first actually enabled debug xtradb configuration.

The release builds diagnose the missing LSN data at the interval start at xb_page_bitmap_init(), after
log_online_setup_bitmap_file_range() completes:

xtrabackup: warning: changed page data missing for LSNs between 9000 and 1633805
xtrabackup: using the full scan for incremental backup

But the debug build will attempt to assert that such situation is impossible:

ut_ad(bitmap_files->files[0].start_lsn == first_file_start_lsn);

If no bitmap file with LSN <= range start LSN, first_file_start_lsn will be LSN_MAX, and files[0].start_lsn will be the lowest LSN of the actual files found (1633805 in 5.6 example). Given that the missing data is diagnosed later, and log_online_setup_bitmap_file_range() is not the correct place to report missing data errors (xb_page_bitmap_init() is), the fix is to remove this assert as overzealous.

This bug is shared with PS I_S query code, where a corresponding testcase needs to be written and the fix ported too.

summary: - xb_incremental_bitmap_misc.sh fails in debug xtradb56 builds
+ xb_incremental_bitmap_misc.sh fails in debug xtradb56 builds | missing
+ bitmap data at the interval start fails "
+ bitmap_files->files[0].start_lsn == first_file_start_lsn" assertion
tags: added: bitmap xtradb
tags: added: 56qual
Roel Van de Paar (roel11) wrote :

Agreed w/ Laurynas on 56qual, so proposing high for re-triage on 5.6 only

The PS fix is proposed at bug 1191580. I failed to remember this bug while fixing that bug. See comment #24 there:

"Thus now there are two different fixes for this issue in the source shared between XB and PS, one merged to XB in 1204075, and one proposed for PS here. I am not sure how to best handle this situation. The current fix seems to be better and suitable for XB eventually as well. Thus if nobody objects I will remove XB from this bug and PS from bug 1204075."

no longer affects: percona-server
no longer affects: percona-server/5.1
no longer affects: percona-server/5.5
no longer affects: percona-server/5.6
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers