sql_cache.cc:1209: void Query_cache::end_of_result(THD*): Assertion `thd->get_stmt_da()->is_eof()' failed | abort (sig=6) at Query_cache::end_of_result

[Roel Van de Paar]

Thread 11 (LWP 2631):
+bt
#0 0x000000363720c69c in pthread_kill () from /lib64/libpthread.so.0
#1 0x00000000006f14bd in handle_fatal_signal (sig=6) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/signal_handler.cc:254
#2 <signal handler called>
#3 0x0000003636e328a5 in raise () from /lib64/libc.so.6
#4 0x0000003636e34085 in abort () from /lib64/libc.so.6
#5 0x0000003636e2ba1e in __assert_fail_base () from /lib64/libc.so.6
#6 0x0000003636e2bae0 in __assert_fail () from /lib64/libc.so.6
#7 0x000000000073a5cc in Query_cache::end_of_result (this=0x156ab80 <query_cache>, thd=0x1a8fb210) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/sql_cache.cc:1209
#8 0x0000000000781006 in dispatch_command (command=<optimized out>, thd=0x1a8fb210, packet=<optimized out>, packet_length=<optimized out>) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/sql_parse.cc:1852
#9 0x0000000000782605 in do_command (thd=0x1a8fb210) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/sql_parse.cc:1051
#10 0x00000000008206b7 in threadpool_process_request (thd=0x1a8fb210) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/threadpool_common.cc:313
#11 0x0000000000823040 in handle_event (connection=0x1b2586a0) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/threadpool_unix.cc:1466
#12 0x0000000000823108 in worker_main (param=0x158fe00 <all_groups+1024>) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/threadpool_unix.cc:1519
#13 0x00000000009ec57a in pfs_spawn_thread (arg=0x1acee7b0) at /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/storage/perfschema/pfs.cc:1858
#14 0x0000003637207851 in start_thread () from /lib64/libpthread.so.0
#15 0x0000003636ee890d in clone () from /lib64/libc.so.6

2013-10-03T06:57:44 [2607] Query: SET GLOBAL INNODB_MAX_CHANGED_PAGES = 0
mysqld: /bzr/5.6-QA_val/Percona-Server-5.6.13-rc60.5/sql/sql_cache.cc:1209: void Query_cache::end_of_result(THD*): Assertion `thd->get_stmt_da()->is_eof()' failed.
20:57:44 UTC - mysqld got signal 6 ;
[...]
Some pointers may be invalid and cause the dump to abort.
Query (24f37710): SET GLOBAL 2013-10-03T06:57:44 [2586] Query: FLUSH TABLE `table10_innodb_compressed`
INNODB_MAX_CHANGED_PAGES = 0
Connection ID (thread ID): 26

Note that the "SET GLOBAL 2013-10-03T06:57:44 [2586] Query: FLUSH TABLE `table10_innodb_compressed`" bit is "inserted" into the error log at the time of printing the query. I think it should be read like this:

This statement was executing: SET GLOBAL INNODB_MAX_CHANGED_PAGES = 0
This statement was about to execute: FLUSH TABLE `table10_innodb_compressed` (or it may have just started?)

[Roel Van de Paar (roel11)]

Seen this bug when trying to reproduce bug 1168265. Bugs seem completely unrelated.

[Roel Van de Paar (roel11)]

Adding 2nd occurence

[Roel Van de Paar (roel11)]

Laurynas reviewed: likely upstream, not 56qual

[Ramesh Sivaraman (rameshvs02)]

This bug is also reported in 5.5

[Ramesh Sivaraman (rameshvs02)]

PS 5.5 gdb out file

[Ramesh Sivaraman (rameshvs02)]

PS 5.5 gdb FULL out file

[Roel Van de Paar (roel11)]

Seen in TokuDB RC as well.

[Roel Van de Paar (roel11)]

Seen in upstream 5.6.20 - logging bug there

[Roel Van de Paar (roel11)]

http://bugs.mysql.com/bug.php?id=73868

[SK (soumya-kandula)]

Any update on this? We are running Percona 5.6.24 and hit this bug today.

mysqld--dbname: /pathtodir/ndn-percona56-5.6.24/sql/sql_cache.cc:1209: void Query_cache::end_of_result(THD*): Assertion `thd->get_stmt_da()->is_eof()' failed.
07:00:01 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=33554432
read_buffer_size=1048576
max_used_connections=102
max_threads=1202
thread_count=31
connection_count=31
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2511099 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x5577b40
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 2bb05089e78 thread_stack 0x40000
/pathtomysql/dbname/mysqld--dbname(my_print_stacktrace+0x38)[0xaa2ac3]
/pathtomysql/dbname/mysqld--dbname(handle_fatal_signal+0x3bd)[0x7198bf]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x2bbb2e03340]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39)[0x2bbb2244cc9]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x2bbb22480d8]
/lib/x86_64-linux-gnu/libc.so.6(+0x2fb86)[0x2bbb223db86]
/lib/x86_64-linux-gnu/libc.so.6(+0x2fc32)[0x2bbb223dc32]
/pathtomysql/dbname/mysqld--dbname(_ZN11Query_cache13end_of_resultEP3THD+0xf6)[0x76f042]
/pathtomysql/dbname/mysqld--dbname(_Z16dispatch_command19enum_server_commandP3THDPcj+0x20c4)[0x7c84ae]
/pathtomysql/dbname/mysqld--dbname(_Z10do_commandP3THD+0x369)[0x7c5f9a]
/pathtomysql/dbname/mysqld--dbname(_Z24do_handle_one_connectionP3THD+0x1b5)[0x78fd32]
/pathtomysql/dbname/mysqld--dbname(handle_one_connection+0x33)[0x78f849]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)[0x2bbb2dfb182]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x2bbb230847d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0): is an invalid pointer
Connection ID (thread ID): 41577817
Status: NOT_KILLED
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
151208 23:00:01 mysqld_safe Number of processes running now: 0
151208 23:00:01 mysqld_safe mysqld restarted

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-730