Comment 18 for bug 1250762

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

The relevant threads are

...
#4 0x00000031df8340c5 in abort () from /lib64/libc.so.6
#5 0x00000000008953b7 in buf_flush_page_and_try_neighbors (bpage=0x7f410f2c3380, flush_type=BUF_FLUSH_LIST, n_to_flush=300, count=0x7f2138844568) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0flu.c:1608
#6 0x0000000000895b8e in buf_flush_flush_list_batch (buf_pool=0x3a1a178, flush_type=<value optimized out>, min_n=300, lsn_limit=18446744073709551615) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0flu.c:1762
...

and

...
#5 0x00000000008935aa in pfs_mutex_enter_func (bpage=0x7f410f2c3380, dpage=0x7f1b14035f00) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/include/sync0sync.ic:222
#6 buf_flush_relocate_on_flush_list (bpage=0x7f410f2c3380, dpage=0x7f1b14035f00) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0flu.c:613
#7 0x000000000089a1b0 in buf_LRU_free_block (bpage=0x7f410f2c3380, zip=0, have_LRU_mutex=1) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0lru.c:2034
...

Both threads operate on the same buffer block. The former thread holds the flush list mutex and asserts that the block is mapped to a data file page. The latter attempts to relocate the same block and for that has removed it from the page hash by buf_LRU_block_remove_hashed_page(), which sets the page state to BUF_BLOCK_REMOVE_HASH, which is what causes the assert in the first thread.

This root cause is the same as for bug 1227581, although the fix probably will have to be made anew and not backported, by auditing BUF_BLOCK_REMOVE_HASH code paths and buf_page_in_file() checks due to non-trivial 5.5 and 5.6 differences in flushing. Closing as a duplicate of bug 1227581 and continuing work there.