Comment 13 for bug 1368846

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

Interesting part from stack trace at the moment of crash:

Thread 8 (Thread 0x41e02940 (LWP 3699)):
#0 0x00000036aac0e438 in pread64 () from /lib64/libpthread.so.0
#1 0x0000000000636f15 in os_file_io(int, void*, unsigned long, long, unsigned long) ()
#2 0x00000000006373cc in os_file_pread(int, void*, unsigned long, unsigned long) ()
#3 0x0000000000637570 in os_file_read_func(int, void*, unsigned long, unsigned long) ()
#4 0x0000000000638000 in os_aio_func(unsigned long, unsigned long, char const*, int, void*, unsigned long, unsigned long, fil_node_t*, void*) ()
#5 0x00000000006e4ae9 in fil_io(unsigned long, bool, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, void*, void*) ()
#6 0x00000000007699bd in buf_read_page_low(dberr_t*, bool, unsigned long, unsigned long, unsigned long, unsigned long, long, unsigned long) ()
#7 0x0000000000769d08 in buf_read_page(unsigned long, unsigned long, unsigned long) ()
#8 0x000000000065f722 in buf_page_get_gen(unsigned long, unsigned long, unsigned long, unsigned long, buf_block_t*, unsigned long, char const*, unsigned long, mtr_t*) ()
#9 0x00000000007791e5 in btr_block_get_func ()
#10 0x00000000007794b4 in btr_cur_latch_leaves(unsigned char*, unsigned long, unsigned long, unsigned long, unsigned long, btr_cur_t*, mtr_t*) ()
#11 0x0000000000781186 in btr_cur_search_to_nth_level(dict_index_t*, unsigned long, dtuple_t const*, unsigned long, unsigned long, btr_cur_t*, unsigned long, char const*, unsigned long, mtr_t*) ()
#12 0x0000000000714858 in btr_pcur_open_with_no_init_func(dict_index_t*, dtuple_t const*, unsigned long, unsigned long, btr_pcur_t*, unsigned long, char const*, unsigned long, mtr_t*) ()
#13 0x0000000000715960 in btr_pcur_restore_position_func(unsigned long, btr_pcur_t*, char const*, unsigned long, mtr_t*) ()
#14 0x000000000072ff5a in ibuf_restore_pos(unsigned long, unsigned long, dtuple_t const*, unsigned long, btr_pcur_t*, mtr_t*) ()
#15 0x0000000000730558 in ibuf_delete_rec(unsigned long, unsigned long, btr_pcur_t*, dtuple_t const*, mtr_t*) ()
#16 0x00000000007358f4 in ibuf_merge_or_delete_for_page(buf_block_t*, unsigned long, unsigned long, unsigned long, unsigned long) ()
#17 0x00000000006618ed in buf_page_io_complete(buf_page_t*) ()
#18 0x00000000006e2e14 in fil_aio_wait(unsigned long) ()
#19 0x000000000079703a in io_handler_thread ()
#20 0x00000036aac0673d in start_thread () from /lib64/libpthread.so.0
#21 0x00000036aa0d44bd in clone () from /lib64/libc.so.6

Here we see that there is an ibuf merge is in progress caused by pending IO operation. I believe that page_cleaner thread should not exit until number of pending IOs comes down to zero.