abort (sig=6) in Diagnostics_area::set_error_status from inline_mysql_file_delete_with_symlink on I_S query | sql_error.cc:422: void Diagnostics_area::set_error_status(THD*, uint, const char*, const char*): Assertion `! is_set() || can_overwrite_status' failed.

[Roel Van de Paar]

mysqld: /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_error.cc:422: void Diagnostics_area::set_error_status(THD*, uint, const char*, const char*): Assertion `! is_set() || can_overwrite_status' failed.
12:00:55 UTC - mysqld got signal 6 ;
[...]
Query (7fa608004d80): SELECT CASE WHEN table_schema = 'information_schema' THEN 'INFORMATION_SCHEMA' ELSE table_schema END, table_name, CASE WHEN table_type = 'BASE TABLE' THEN''WHEN table_type = 'VIEW' THEN''WHEN table_type = 'SYSTEM VIEW' then''ELSE''END, column_name, CASE WHEN column_key = 'PRI' THEN''WHEN column_key = 'MUL' THEN''WHEN column_key = 'UNI' THEN''ELSE''END FROM information_schema.tables INNER JOIN information_schema.columns USING(table_schema, table_name) WHERE table_name <> 'DUMMY'
Connection ID (thread ID): 6
Status: NOT_KILLED

+bt
#0 0x000000340cc0c69c in pthread_kill () from /lib64/libpthread.so.0
#1 0x0000000000808e19 in my_write_core (sig=6) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/mysys/stacktrace.c:433
#2 0x00000000006bd895 in handle_fatal_signal (sig=6) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/signal_handler.cc:250
#3 <signal handler called>
#4 0x000000340c4328a5 in raise () from /lib64/libc.so.6
#5 0x000000340c434085 in abort () from /lib64/libc.so.6
#6 0x000000340c42ba1e in __assert_fail_base () from /lib64/libc.so.6
#7 0x000000340c42bae0 in __assert_fail () from /lib64/libc.so.6
#8 0x0000000000587590 in Diagnostics_area::set_error_status (this=0x32629a0, thd=<optimized out>, sql_errno_arg=6, message_arg=0x7fa62c189c00 "Error on delete of '/tmp/#sql_69f9_1.MYI' (Errcode: 2)", sqlstate=0xa836f9 "HY000") at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_error.cc:422
#9 0x0000000000578314 in THD::raise_condition (this=0x325fb00, sql_errno=6, sqlstate=0xa836f9 "HY000", level=<optimized out>, msg=0x7fa62c189c00 "Error on delete of '/tmp/#sql_69f9_1.MYI' (Errcode: 2)") at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_class.cc:1219
#10 0x0000000000519d6c in my_message_sql (error=6, str=<optimized out>, MyFlags=36) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/mysqld.cc:2828
#11 0x00000000007ff72c in my_error (nr=6, MyFlags=36) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/mysys/my_error.c:96
#12 0x00000000007ff0d5 in my_delete (name=0x7fa62c18a220 "/tmp/#sql_69f9_1.MYI", MyFlags=<optimized out>) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/mysys/my_delete.c:30
#13 0x00000000008060ae in my_delete_with_symlink (name=0x7fa62c18a220 "/tmp/#sql_69f9_1.MYI", MyFlags=16) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/mysys/my_symlink2.c:106
#14 0x00000000009fb022 in inline_mysql_file_delete_with_symlink (flags=16, name=0x7fa62c18a220 "/tmp/#sql_69f9_1.MYI", src_line=43, src_file=0xb750d8 "/bzr/5.5_val/Percona-Server-5.5.32-rel31.0/storage/myisam/mi_delete_table.c", key=<optimized out>) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/include/mysql/psi/mysql_file.h:1370
#15 mi_delete_table (name=0x7fa608022720 "/tmp/#sql_69f9_1") at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/storage/myisam/mi_delete_table.c:43
#16 0x00000000009e41b0 in ha_myisam::delete_table (this=<optimized out>, name=<optimized out>) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/storage/myisam/ha_myisam.cc:1944
#17 0x00000000006bddc2 in handler::drop_table (this=0x7fa608026e30, name=0x7fa608022720 "/tmp/#sql_69f9_1") at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/handler.cc:3396
#18 0x00000000006c6ae8 in handler::ha_drop_table (this=0x7fa608026e30, name=0x7fa608022720 "/tmp/#sql_69f9_1") at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/handler.cc:3703
#19 0x00000000005c735e in free_tmp_table (thd=0x325fb00, entry=0x7fa608021080) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_select.cc:11313
#20 0x0000000000561375 in close_thread_tables (thd=0x325fb00) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_base.cc:1499
#21 0x00000000005a7459 in mysql_execute_command (thd=0x325fb00) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_parse.cc:4722
#22 0x00000000005a7b70 in mysql_parse (thd=0x325fb00, rawbuf=0x7fa608004d80 "SELECT CASE WHEN table_schema = 'information_schema' THEN 'INFORMATION_SCHEMA' ELSE table_schema END, table_name, CASE WHEN table_type = 'BASE TABLE' THEN''WHEN table_type = 'VIEW' THEN''WHEN table_type = 'SYSTEM VIEW' then''ELSE''END, column_name, CASE WHEN column_key = 'PRI' THEN''WHEN column_key = 'MUL' THEN''WHEN column_key = 'UNI' THEN''ELSE''END FROM information_schema.tables INNER JOIN information_schema.columns USING(table_schema, table_name) WHERE table_name <> 'DUMMY'", length=<optimized out>, parser_state=0x7fa62c18bb10) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_parse.cc:5889
#23 0x00000000005a8ee2 in dispatch_command (command=COM_QUERY, thd=0x325fb00, packet=<optimized out>, packet_length=482) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_parse.cc:1079
#24 0x00000000005aa5c3 in do_command (thd=0x325fb00) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_parse.cc:789
#25 0x000000000065482f in do_handle_one_connection (thd_arg=<optimized out>) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_connect.cc:1487
#26 0x000000000065492e in handle_one_connection (arg=0x325fb00) at /bzr/5.5_val/Percona-Server-5.5.32-rel31.0/sql/sql_connect.cc:1402
#27 0x000000340cc07851 in start_thread () from /lib64/libpthread.so.0
#28 0x000000340c4e890d in clone () from /lib64/libc.so.6
+quit

[Roel Van de Paar (roel11)]

Testcase (sporadic, needs to be run in high concurrency, pasted here to get an idea of queries involved)

DROP DATABASE transforms;CREATE DATABASE transforms;DROP DATABASE test;CREATE DATABASE test;USE test;
SELECT CONNECTION_ID();
SELECT CASE WHEN table_schema = 'information_schema' THEN 'INFORMATION_SCHEMA' ELSE table_schema END, table_name, CASE WHEN table_type = 'BASE TABLE' THEN''WHEN table_type = 'VIEW' THEN''WHEN table_type = 'SYSTEM VIEW' then''ELSE''END, column_name, CASE WHEN column_key = 'PRI' THEN''WHEN column_key = 'MUL' THEN''WHEN column_key = 'UNI' THEN''ELSE''END FROM information_schema.tables INNER JOIN information_schema.columns USING(table_schema, table_name) WHERE table_name <> 'DUMMY';
SELECT collation_name,character_set_name FROM information_schema.collations;

And, server was running with these parameters:

--loose-new --sql-mode=no_engine_substitution --relay-log=slave-relay-bin --loose-innodb --secure-file-priv= --max-allowed-packet=16Mb --loose-innodb-status-file=1 --master-retry-count=65535 --loose-debug-assert-if-crashed-table --loose-debug-assert-on-error --skip-name-resolve --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --innodb_file_per_table=1 --innodb_adaptive_hash_index_partitions=1 --core

[Raghavendra D Prabhu (raghavendra-prabhu)]

@Roel,

Is this always on I_S queries? We have been hitting the crash here - lp:1208937 - but on different queries.

[Laurynas Biveinis (laurynas-biveinis)]

Is thread apply bt full available? Is /tmp/#sql_69f9_1.MYI present on the test system (assuming no /tmp/* cleanups after the run?)

[Valerii Kravchuk (valerii-kravchuk)]

Roel,

Assuming this assertion still happens with recent code, do you have any reply for comments #3 and #4?

[Roel Van de Paar (roel11)]

If I see it again, I will update bug report

[Launchpad Janitor (janitor)]

[Expired for Percona Server because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for Percona Server 5.6 because there has been no activity for 60 days.]

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-3012