...
#4 0x00000031df8340c5 in abort () from /lib64/libc.so.6
#5 0x00000000008953b7 in buf_flush_page_and_try_neighbors (bpage=0x7f410f2c3380, flush_type=BUF_FLUSH_LIST, n_to_flush=300, count=0x7f2138844568) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0flu.c:1608
#6 0x0000000000895b8e in buf_flush_flush_list_batch (buf_pool=0x3a1a178, flush_type=<value optimized out>, min_n=300, lsn_limit=18446744073709551615) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0flu.c:1762
...
and
...
#5 0x00000000008935aa in pfs_mutex_enter_func (bpage=0x7f410f2c3380, dpage=0x7f1b14035f00) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/include/sync0sync.ic:222
#6 buf_flush_relocate_on_flush_list (bpage=0x7f410f2c3380, dpage=0x7f1b14035f00) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0flu.c:613
#7 0x000000000089a1b0 in buf_LRU_free_block (bpage=0x7f410f2c3380, zip=0, have_LRU_mutex=1) at /mnt/workspace/percona-server-5.5-binaries/label_exp/centos5-64/Percona-Server-5.5.34-rel32.0/storage/innobase/buf/buf0lru.c:2034
...
Both threads operate on the same buffer block. The former thread holds the flush list mutex and asserts that the block is mapped to a data file page. The latter attempts to relocate the same block and for that has removed it from the page hash by buf_LRU_block_remove_hashed_page(), which sets the page state to BUF_BLOCK_REMOVE_HASH, which is what causes the assert in the first thread.
This root cause is the same as for bug 1227581, although the fix probably will have to be made anew and not backported, by auditing BUF_BLOCK_REMOVE_HASH code paths and buf_page_in_file() checks due to non-trivial 5.5 and 5.6 differences in flushing. Closing as a duplicate of bug 1227581 and continuing work there.
The relevant threads are
... page_and_ try_neighbors (bpage= 0x7f410f2c3380, flush_type= BUF_FLUSH_ LIST, n_to_flush=300, count=0x7f21388 44568) at /mnt/workspace/ percona- server- 5.5-binaries/ label_exp/ centos5- 64/Percona- Server- 5.5.34- rel32.0/ storage/ innobase/ buf/buf0flu. c:1608 flush_list_ batch (buf_pool= 0x3a1a178, flush_type=<value optimized out>, min_n=300, lsn_limit= 184467440737095 51615) at /mnt/workspace/ percona- server- 5.5-binaries/ label_exp/ centos5- 64/Percona- Server- 5.5.34- rel32.0/ storage/ innobase/ buf/buf0flu. c:1762
#4 0x00000031df8340c5 in abort () from /lib64/libc.so.6
#5 0x00000000008953b7 in buf_flush_
#6 0x0000000000895b8e in buf_flush_
...
and
... enter_func (bpage= 0x7f410f2c3380, dpage=0x7f1b140 35f00) at /mnt/workspace/ percona- server- 5.5-binaries/ label_exp/ centos5- 64/Percona- Server- 5.5.34- rel32.0/ storage/ innobase/ include/ sync0sync. ic:222 relocate_ on_flush_ list (bpage= 0x7f410f2c3380, dpage=0x7f1b140 35f00) at /mnt/workspace/ percona- server- 5.5-binaries/ label_exp/ centos5- 64/Percona- Server- 5.5.34- rel32.0/ storage/ innobase/ buf/buf0flu. c:613 0x7f410f2c3380, zip=0, have_LRU_mutex=1) at /mnt/workspace/ percona- server- 5.5-binaries/ label_exp/ centos5- 64/Percona- Server- 5.5.34- rel32.0/ storage/ innobase/ buf/buf0lru. c:2034
#5 0x00000000008935aa in pfs_mutex_
#6 buf_flush_
#7 0x000000000089a1b0 in buf_LRU_free_block (bpage=
...
Both threads operate on the same buffer block. The former thread holds the flush list mutex and asserts that the block is mapped to a data file page. The latter attempts to relocate the same block and for that has removed it from the page hash by buf_LRU_ block_remove_ hashed_ page(), which sets the page state to BUF_BLOCK_ REMOVE_ HASH, which is what causes the assert in the first thread.
This root cause is the same as for bug 1227581, although the fix probably will have to be made anew and not backported, by auditing BUF_BLOCK_ REMOVE_ HASH code paths and buf_page_in_file() checks due to non-trivial 5.5 and 5.6 differences in flushing. Closing as a duplicate of bug 1227581 and continuing work there.