PURGE CHANGED_PAGE_BITMAPS BEFORE purges too many bitmap files

[Andrew Gaul]

log_online_purge_changed_page_bitmaps only considers the start LSN of a given bitmap when purging a given bitmap. If it is less than the provided LSN, XtraDB removes the bitmap. Consider this example:

$ ls -1 /var/lib/mysql/*.xdb
ib_modified_log_1_0.xdb
ib_modified_log_2_66075588.xdb
ib_modified_log_3_133835964.xdb
$ echo PURGE CHANGED_PAGE_BITMAPS BEFORE 70000000 | mysql -u root
$ ls -1 /var/lib/mysql/*.xdb
/var/lib/mysql/ib_modified_log_3_133835964.xdb

Instead, XtraDB should consider the start LSN of the subsequent bitmap to infer the end LSN of the current bitmap. With my patch:

$ ls -1 *.xdb
ib_modified_log_1_0.xdb
ib_modified_log_2_63575115.xdb
ib_modified_log_3_131771008.xdb
$ echo PURGE CHANGED_PAGE_BITMAPS BEFORE 70000000 | mysql -u root
$ ls -1 *.xdb
ib_modified_log_2_63575115.xdb
ib_modified_log_3_131771008.xdb

Without my patch, XtraBackup must perform a full table scan in incremental mode since the bitmap files do not represent the entire range of LSNs.

[Laurynas Biveinis (laurynas-biveinis)]

Andrew -

Thanks for the bug report and the patch. I am testing your patch and will apply it, with modifications:

- log_online_purge_changed_page_bitmaps() called with LSN 0 or LSN_MAX is used by RESET BITMAPS that should delete everything. Since the fix means that the last bitmap file in the range will never be deleted if it's the last bitmap file in the directory (as it may contain LSN data up to infinity), it had to be special-cased or RESET BITMAPS would leave that last file.
- There is an off-by-one error in start_lsn >= lsn comparison. I.e. if you have ib_modified_log_1_0.xdb, ib_modified_log_2_100.xdb and issue PURGE BEFORE 100, it would leave the 1st file although it may be safely deleted.
- Added an MTR testcase.

[Andrew Gaul (gaul)]

Thanks Laurynas, your amended commit with the off-by-one change makes sense. Can you help me understand this test:

PURGE CHANGED_PAGE_BITMAPS BEFORE 200;
ib_modified_log_2_100.xdb
ib_modified_log_3_200.xdb
ib_modified_log_4_300.xdb

I expect this to remove ib_modified_log_2_100.xdb.

[Laurynas Biveinis (laurynas-biveinis)]

Andrew -

Indeed it should, but currently does not. The problem here is that log_online_setup_bitmap_file_range() in log_online_purge_changed_page_bitmaps() does not include 3_200.xdb in the range as its start LSN falls outside, thus the range contains only one file 2_100.xdb, which is then not removed because it's the last in the range.

I am modifying the fix so that log_online_purge_changed_page_bitmaps() requests the range for 0..LSN_MAX instead of 0..lsn.That's suboptimal if there are many bitmap files, but the alternative of making log_online_setup_bitmap_file_range() to include one file past the range seems to be too complicated for a little benefit.

[Andrew Gaul (gaul)]

Looks good, thanks Laurynas!

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1455