handle_fatal_signal (sig=11) in set_null from set_field_to_null_with_conversions

[Roel Van de Paar]

+bt
#0 0x00007fbf2b19f741 in __pthread_kill (threadid=<optimized out>, signo=signo@entry=11) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:61
#1 0x00000000007f9078 in my_write_core (sig=11) at /git/percona-server_dbg/mysys/stacktrace.c:433
#2 0x00000000006b9296 in handle_fatal_signal (sig=11) at /git/percona-server_dbg/sql/signal_handler.cc:250
#3 <signal handler called>
#4 0x00000000006b479c in set_null (row_offset=0, this=0x7fbf0800aaf8) at /git/percona-server_dbg/sql/field.h:289
#5 set_field_to_null_with_conversions (field=field@entry=0x7fbf0800aaf8, no_conversions=no_conversions@entry=false) at /git/percona-server_dbg/sql/field_conv.cc:164
#6 0x00000000006cbd20 in Item::save_in_field (this=0x7fbf08005438, field=0x7fbf0800aaf8, no_conversions=<optimized out>) at /git/percona-server_dbg/sql/item.cc:5559
#7 0x0000000000552202 in fill_record (thd=thd@entry=0x34f9a30, ptr=0x7fbf0800aae8, values=..., ignore_errors=ignore_errors@entry=true) at /git/percona-server_dbg/sql/sql_base.cc:8812
#8 0x00000000005522a9 in fill_record_n_invoke_before_triggers (thd=0x34f9a30, ptr=<optimized out>, values=..., ignore_errors=ignore_errors@entry=true, triggers=0x0, event=event@entry=TRG_EVENT_INSERT) at /git/percona-server_dbg/sql/sql_base.cc:8853
#9 0x000000000057dc57 in select_create::store_values (this=<optimized out>, values=...) at /git/percona-server_dbg/sql/sql_insert.cc:4090
#10 0x000000000058047c in select_insert::send_data (this=0x7fbf080059b0, values=...) at /git/percona-server_dbg/sql/sql_insert.cc:3465
#11 0x00000000005df7c3 in JOIN::exec (this=this@entry=0x7fbf08007df0) at /git/percona-server_dbg/sql/sql_select.cc:1937
#12 0x00000000005db8f3 in mysql_select (thd=thd@entry=0x34f9a30, rref_pointer_array=rref_pointer_array@entry=0x34fbf68, tables=0x0, wild_num=0, fields=..., conds=<optimized out>, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=select_options@entry=2416184064, result=result@entry=0x7fbf080059b0, unit=unit@entry=0x34fb760, select_lex=select_lex@entry=0x34fbd80) at /git/percona-server_dbg/sql/sql_select.cc:2662
#13 0x00000000005dbb31 in handle_select (thd=thd@entry=0x34f9a30, lex=lex@entry=0x34fb6a0, result=result@entry=0x7fbf080059b0, setup_tables_done_option=setup_tables_done_option@entry=0) at /git/percona-server_dbg/sql/sql_select.cc:315
#14 0x000000000059716d in mysql_execute_command (thd=thd@entry=0x34f9a30) at /git/percona-server_dbg/sql/sql_parse.cc:2796
#15 0x000000000059dc13 in mysql_parse (thd=thd@entry=0x34f9a30, rawbuf=<optimized out>, length=84, parser_state=parser_state@entry=0x7fbf24125560) at /git/percona-server_dbg/sql/sql_parse.cc:6058
#16 0x000000000059f617 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x34f9a30, packet=packet@entry=0x3584cd1 "CREATE TABLE t2 ENGINE=MEMORY AS SELECT COALESCE(NULL,NULL,NULL),GREATEST(NULL,NULL);", packet_length=packet_length@entry=85) at /git/percona-server_dbg/sql/sql_parse.cc:1075
#17 0x00000000005a142f in do_command (thd=0x34f9a30) at /git/percona-server_dbg/sql/sql_parse.cc:789
#18 0x000000000064f906 in do_handle_one_connection (thd_arg=thd_arg@entry=0x34f9a30) at /git/percona-server_dbg/sql/sql_connect.cc:1418
#19 0x000000000064f9fc in handle_one_connection (arg=0x34f9a30) at /git/percona-server_dbg/sql/sql_connect.cc:1325
#20 0x00007fbf2b19adc5 in start_thread (arg=0x7fbf24126700) at pthread_create.c:308
#21 0x00007fbf29a1721d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

The attached tarball (1463364118_bug_bundle.tar.gz) gives the testcase as an exact match of our system, including some handy utilities

$ vi 1463364118_mybase # STEP1: Update the base path in this file (usually the only change required!). If you use a non-binary distribution, please update SOURCE_DIR location also
$ ./1463364118_init # STEP2: Initializes the data dir
$ ./1463364118_start # STEP3: Starts mysqld
$ ./1463364118_cl # STEP4: To check mysqld is up
$ ./1463364118_run_pquery # STEP5: Run the testcase with the pquery binary
$ vi /dev/shm/1463364118/error.log.out # STEP6: Verify the error log
$ ./1463364118_gdb # OPTIONAL: Brings you to a gdb prompt with gdb attached to the used mysqld and attached to the generated core
$ ./1463364118_parse_core # OPTIONAL: Creates 1463364118_STD.gdb and 1463364118_FULL.gdb; standard and full variables gdb stack traces

Note ./1463364118_run (Run the testcase with the mysql CLI) will not reproduce the issue! It only reproduces using the pquery binary

[Roel Van de Paar (roel11)]

Note that the reference to TokuDB engine in 1463364118.sql in the tarball is coincidental; you can just change the engine name to "DOESNOTEXIST".

SQL testcase (needs to be executed through pquery binary (included in tarball) as per above):

DROP DATABASE test;CREATE DATABASE test;USE test;
CREATE TABLE t1(c1 INT KEY,c2 INT,INDEX x(c2)) ENGINE=DOESNOTEXIST ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=16;
INSERT INTO t VALUES(NULL),(NULL);
ALTER TABLE t1 PARTITION BY KEY(c1)PARTITIONS 5;
CREATE TABLE t2 ENGINE=MEMORY AS SELECT COALESCE(NULL,NULL,NULL),GREATEST(NULL,NULL);

[Roel Van de Paar (roel11)]

This issue does not reproduce on upstream. This issue is a screenfiller type bug. It was seen 388 times in a weekend run.

[Roel Van de Paar (roel11)]

Issue may be connected with/related to COALESCE or even AS SELECT COALESCE(NULL,NULL,NULL).

[Roel Van de Paar (roel11)]

Quite convinced now that issue is connected with AS SELECT COALESCE(NULL, NULL, NULL)

[Laurynas Biveinis (laurynas-biveinis)]

Reduced testcase, crashes intermittently

CREATE TABLE t1(c1 INT KEY,c2 INT,INDEX x(c2)) ENGINE=DOESNOTEXIST;
ALTER TABLE t1 PARTITION BY KEY(c1)PARTITIONS 5;
CREATE TABLE t2 ENGINE=MEMORY AS SELECT COALESCE(NULL,NULL,NULL),GREATEST(NULL,NULL);

[Laurynas Biveinis (laurynas-biveinis)]

Even though "DOESNOTEXIST" is replaced by MyISAM in runtime, I could not get it to crash if MyISAM is specified explicitly.

[Roel Van de Paar (roel11)]

Thanks Laurynas.

Likely related with MS bug 81446, just logged. Hard to debug issue.

[Roel Van de Paar (roel11)]

MS Bug #81446: handle_fatal_signal (sig=11) in create_schema_table & set_null & is_null

[Laurynas Biveinis (laurynas-biveinis)]

The upstream bug testcases look close enough to this one to mark as a duplicate for the time being. This can be revisited as needed.

[Roel Van de Paar (roel11)]

Triage high is perhaps warranted: one testcase was found in upstream without COALESCE as well as seeing this crash in many different locations.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-983