MySQL crashes with Assertion failure in file btr0pcur.c line 452

[Andrew Sammut]

We are running Percona XtraDB Mysql 5.5.30-30.2 on CentOS 6.2 running kernel 2.6.32-279.19.1.el6.x86_64.

The following is the error that we are experiencing:

131231 18:29:47 InnoDB: Assertion failure in thread 139352866322176 in file btr0pcur.c line 452
InnoDB: Failing assertion: btr_page_get_prev(next_page, mtr) == buf_block_get_page_no(btr_pcur_get_block(cursor))
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
18:29:47 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=67108864
read_buffer_size=131072
max_used_connections=1216
max_threads=2002
thread_count=768
connection_count=768
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4446431 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x2429c1090
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7ebd9e186d98 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x35)[0x7b2d25]
/usr/sbin/mysqld(handle_fatal_signal+0x4b4)[0x68e5d4]
/lib64/libpthread.so.0[0x368ac0f500]
/lib64/libc.so.6(gsignal+0x35)[0x368a4328a5]
/lib64/libc.so.6(abort+0x175)[0x368a434085]
/usr/sbin/mysqld[0x8a52d8]
/usr/sbin/mysqld[0x853bb7]
/usr/sbin/mysqld[0x854609]
/usr/sbin/mysqld[0x827af4]
/usr/sbin/mysqld(_Z13rr_sequentialP11READ_RECORD+0x1f)[0x75643f]
/usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x79)[0x5b4b49]
/usr/sbin/mysqld[0x5b5517]
/usr/sbin/mysqld(_ZN4JOIN4execEv+0xc62)[0x5ca942]
/usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x12c)[0x5cc0cc]
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x1cd)[0x5ccb7d]
/usr/sbin/mysqld[0x589152]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1553)[0x58dad3]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x33b)[0x5912db]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x15cd)[0x59294d]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x13f)[0x62d47f]
/usr/sbin/mysqld(handle_one_connection+0x51)[0x62d561]
/lib64/libpthread.so.0[0x368ac07851]
/lib64/libc.so.6(clone+0x6d)[0x368a4e811d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7ebb6a32dc90): is an invalid pointer
Connection ID (thread ID): 130829116
Status: NOT_KILLED

You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.

The following is the code that is being run at the time:

INSERT INTO #####
(`id`, `userid`, `targetid`, `gifttype`, `stage`, `sendtime`, `accepttime`, `collecttime`)
SELECT
`id`, `userid`, `targetid`, `gifttype`, `stage`, `sendtime`, `accepttime`, `collecttime`
FROM #####
WHERE sendtime > 1384500000;

[Valerii Kravchuk (valerii-kravchuk)]

We had similar report: https://bugs.launchpad.net/percona-server/+bug/1252792

You case looks very similar., you also have innodb_lazy_drop_table=1. So, please, check if this kind of error ever happens again with this feature disabled?

Please, upload also the entire error log (compressed), or at least a part of since last startup before this assertion failure.

[Andrew Sammut (asammut)]

We have set innodb_lazy_drop_table to 0 and then ran the attached SQL again, and it still crashes.

Here is the latest crash information:

140228 1:22:25 InnoDB: Assertion failure in thread 139984215332608 in file btr0pcur.c line 452
InnoDB: Failing assertion: btr_page_get_prev(next_page, mtr) == buf_block_get_page_no(btr_pcur_get_block(cursor))
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
01:22:25 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=67108864
read_buffer_size=131072
max_used_connections=694
max_threads=2002
thread_count=358
connection_count=358
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4446431 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x2414aa3e0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f509d6dcd98 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x35)[0x7b2d25]
/usr/sbin/mysqld(handle_fatal_signal+0x4b4)[0x68e5d4]
/lib64/libpthread.so.0[0x368ac0f500]
/lib64/libc.so.6(gsignal+0x35)[0x368a4328a5]
/lib64/libc.so.6(abort+0x175)[0x368a434085]
/usr/sbin/mysqld[0x8a52d8]
/usr/sbin/mysqld[0x853bb7]
/usr/sbin/mysqld[0x854609]
/usr/sbin/mysqld[0x827af4]
/usr/sbin/mysqld[0x5b1f6d]
/usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x79)[0x5b4b49]
/usr/sbin/mysqld[0x5b5517]
/usr/sbin/mysqld(_ZN4JOIN4execEv+0xc62)[0x5ca942]
/usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x12c)[0x5cc0cc]
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x1cd)[0x5ccb7d]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x4086)[0x590606]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x33b)[0x5912db]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x15cd)[0x59294d]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x13f)[0x62d47f]
/usr/sbin/mysqld(handle_one_connection+0x51)[0x62d561]
/lib64/libpthread.so.0[0x368ac07851]
/lib64/libc.so.6(clone+0x6d)[0x368a4e811d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f4fec014c40): is an invalid pointer
Connection ID (thread ...

[Valerii Kravchuk (valerii-kravchuk)]

Please, send the output of:

explain SELECT
`id`, `userid`, `targetid`, `gifttype`, `stage`, `sendtime`, `accepttime`, `collecttime`
FROM #####
WHERE sendtime > 1384500000;

show create table #####\G
show table status like '#####'\G

check table #####;

for the table ##### used in SELECT part.

[Launchpad Janitor (janitor)]

[Expired for Percona Server because there has been no activity for 60 days.]

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-3098