handle_fatal_signal in ha_partition::clone (sql/ha_partition.cc:3412)

Bug #1193264 reported by Alexey Bychko on 2013-06-21
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Fix Released
High
Laurynas Biveinis
5.1
Invalid
Low
Unassigned
5.5
Invalid
Low
Unassigned
5.6
Fix Released
High
Laurynas Biveinis

Bug Description

Thread 1 (Thread 0x7fe73c991700 (LWP 12882)):
+bt
#0 0x00007fe7588ff69c in pthread_kill () from /lib64/libpthread.so.0
#1 0x0000000000661f9e in handle_fatal_signal (sig=11) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/signal_handler.cc:251
#2 <signal handler called>
#3 0x0000000000b11238 in ha_partition::clone (this=0x7fe6c400aa30, name=0x7fe6c40116e8 "./query112928/#sql-3113_3f", mem_root=0x1c63fc8) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/ha_partition.cc:3412
#4 0x0000000000708f22 in store_temporary_table_record (thd=0x1c609f0, table=0x7fe6b4016b30, tmp_table=0x7fe6c4010ae0, db=<optimized out>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_show.cc:3950
#5 0x000000000071a1e6 in fill_global_temporary_tables (thd=0x1cf7470, tables=0x7fe6d00055f8, cond=<optimized out>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_show.cc:4035
#6 0x0000000000708a4c in do_fill_table (join_table=<optimized out>, table_list=0x7fe6d00055f8, thd=0x1cf7470) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_show.cc:7684
#7 get_schema_tables_result (join=0x7fe6b400aae0, executed_place=PROCESSED_BY_JOIN_EXEC) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_show.cc:7785
#8 0x00000000006fd0dd in JOIN::prepare_result (this=0x7fe6b400aae0, columns_list=<optimized out>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_select.cc:823
#9 0x00000000006bd72f in JOIN::exec (this=0x7fe6b400aae0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_executor.cc:116
#10 0x00000000006ff2f0 in mysql_execute_select (free_join=true, select_lex=0x1cf9f08, thd=0x1cf7470) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_select.cc:1101
#11 mysql_select (thd=0x1cf7470, tables=0x7fe6d00055f8, wild_num=0, fields=..., conds=0x0, order=<optimized out>, group=0x1cfa008, having=0x0, select_options=2953054976, result=0x7fe6d0006ad8, unit=0x1cf98c0, select_lex=0x1cf9f08) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_select.cc:1222
#12 0x00000000006ffb67 in handle_select (thd=0x1cf7470, result=0x7fe6d0006ad8, setup_tables_done_option=0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_select.cc:111
#13 0x00000000006e0996 in mysql_execute_command (thd=0x1cf7470) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_parse.cc:3232
#14 0x00000000006e1588 in mysql_parse (thd=0x1cf7470, rawbuf=<optimized out>, length=30382072, parser_state=<optimized out>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_parse.cc:6451
#15 0x00000000006e2441 in dispatch_command (command=COM_QUERY, thd=0x1cf7470, packet=0x1cfb3b1 "CREATE TABLE transforms.insert_select_12865 SELECT COUNT(*) FROM INFORMATION_SCHEMA.TEMPORARY_TABLES", packet_length=100) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/sql_parse.cc:1376
#16 0x000000000077ba2f in threadpool_process_request (thd=0x1cf7470) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/threadpool_common.cc:312
#17 0x000000000077ddd8 in handle_event (connection=0x1c301d0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/threadpool_unix.cc:1466
#18 worker_main (param=0x12df200 <all_groups+5632>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/threadpool_unix.cc:1519
#19 0x0000000000ab05d9 in pfs_spawn_thread (arg=0x7fe6d0015930) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/perfschema/pfs.cc:1853
#20 0x00007fe7588fa851 in start_thread () from /lib64/libpthread.so.0
#21 0x00007fe75706d90d in clone () from /lib64/libc.so.6

Related branches

Alexey Bychko (abychko) wrote :
tags: added: qa
Changed in percona-server:
importance: Undecided → High
status: New → Confirmed
Alexey Bychko (abychko) wrote :
Alexey Bychko (abychko) wrote :
Alexey Bychko (abychko) on 2013-06-21
Changed in percona-server:
assignee: nobody → Laurynas Biveinis (laurynas-biveinis)
Alexey Bychko (abychko) wrote :

command to reproduce:

bash -c "set -o pipefail; perl runall.pl --queries=100000000 --seed=3 --duration=300 --querytimeout=60 --short_column_names --reporter=Shutdown,Backtrace,QueryTimeout,ErrorLog,ErrorLogAlarm --mysqld=--log-output=none --mysqld=--sql_mode=ONLY_FULL_GROUP_BY --grammar=conf/percona_qa/5.6/5.6.yy --gendata=conf/percona_qa/5.6/5.6.zz --basedir=/sdd/Percona-Server-5.6.11-rc60.3-387.Linux.x86_64 --threads=25 --mask-level=1 --notnull --validator=Transformer --mysqld=--thread_handling=pool-of-threads --mysqld=--innodb_log_file_size=1048576 --mysqld=--innodb_log_files_in_group=2 --mysqld=--innodb_log_buffer_size=1048576 --mysqld=--innodb_log_block_size=512 --mysqld=--innodb_fast_shutdown=2 --mysqld=--innodb_log_group_home_dir=/sdd/223308/_epoch/1371735977789416680 --mysqld=--innodb_use_global_flush_log_at_trx_commit=0 --mysqld=--userstat --mysqld=--innodb_flush_method=O_DSYNC --mtr-build-thread=747 --mask=36130 --vardir1=/sdd/223308/cmdrun_124 > /sdd/223308/cmdrun124.log 2>&1"

Roel Van de Paar (roel11) wrote :
tags: added: i-s-temp-tables
Download full text (4.8 KiB)

Temp table in the crashing thread is belongs to Thread 77. This should be enough for a DEBUG_SYNC testcase and fix. Not a duplicate of bug 1193308, but indeed similar.

Thread 77 (Thread 0x7fe73f535700 (LWP 12875)):
+bt
#0 0x00007fe758901c0d in fsync () from /lib64/libpthread.so.0
#1 0x000000000093958d in os_file_fsync (file=131) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/os/os0file.cc:2285
#2 0x0000000000939639 in os_file_flush_func (file=<optimized out>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/os/os0file.cc:2386
#3 0x0000000000a383a4 in pfs_os_file_flush_func (file=131, src_line=3440, src_file=0xc013f0 "/bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/fil/fil0fil.cc") at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/include/os0file.ic:355
#4 0x0000000000a43cf0 in fil_create_new_single_table_tablespace (space_id=69, tablename=0x7fe6c40089c0 "query112928/#sql-ib70", dir_path=<optimized out>, flags=0, flags2=<optimized out>, size=4) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/fil/fil0fil.cc:3440
#5 0x0000000000a15ba3 in dict_build_table_def_step (thr=0x7fe6c40224e0, node=0x7fe6c403bc68) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/dict/dict0crea.cc:306
#6 0x0000000000a161db in dict_create_table_step (thr=0x7fe6c40224e0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/dict/dict0crea.cc:1030
#7 0x000000000095acdd in que_thr_step (thr=0x7fe6c40224e0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/que/que0que.cc:1123
#8 que_run_threads_low (thr=0x7fe6c40224e0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/que/que0que.cc:1177
#9 que_run_threads (thr=0x7fe6c40224e0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/que/que0que.cc:1218
#10 0x000000000098357c in row_create_table_for_mysql (table=0x7fe6c40107a8, trx=0x7fe6c4020bb0, commit=false) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/row/row0mysql.cc:2263
#11 0x00000000008fe413 in prepare_inplace_alter_table_dict (ha_alter_info=0x7fe73f532ca0, altered_table=0x7fe6c4010ae0, old_table=0x7fe62800e390, table_name=0x7fe62800834c "t1", flags=1, flags2=16, fts_doc_id_col=18446744073709551615, add_fts_doc_id=false, add_fts_doc_id_idx=<optimized out>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/handler/handler0alter.cc:2812
#12 0x000000000090031f in ha_innobase::prepare_inplace_alter_table (this=<optimized out>, altered_table=<optimized out>, ha_alter_info=<optimized out>) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/storage/innobase/handler/handler0alter.cc:3821
#13 0x0000000000b09e1a in ha_partition::prepare_inplace_alter_table (this=0x7fe62800ec80, altered_table=0x7fe6c4010ae0, ha_alter_info=0x7fe73f532ca0) at /bzr/5.6_opt/Percona-Server-5.6.11-rc60.3/sql/ha_partition.cc:7950
#14 0x000000000072eba5 in mysql_inplace_alter_table (thd=0x1c609f0, table_list=0x7fe628004da0, table=0x7fe62800e390, altered_table=0x7fe6c4010ae0, ha_alter_info=0x7fe73f532ca0, inplace_supported=HA_ALTER_INPLACE_NO_LOCK_AFTER_PREPARE, target_mdl_request=0x7fe73f532a10, alter_ctx=0x7fe73f5313f0) at /bzr/5.6_opt/Percona-Server-5.6.11-r...

Read more...

Roel Van de Paar (roel11) wrote :

Adding qablock; seeing this is a lot of runs

tags: added: qablock
Roel Van de Paar (roel11) wrote :
Roel Van de Paar (roel11) wrote :
Roel Van de Paar (roel11) wrote :
Roel Van de Paar (roel11) wrote :

Added one more occurrence in #8-#10. Run details:

[roel@qaserver 209184]$ cat cmd414
ps -ef | grep 'cmdrun_414' | grep -v grep | awk '{print $2}' | xargs sudo kill -9
rm -Rf /ssd/209184/cmdrun_414
mkdir /ssd/209184/cmdrun_414
cd /ssd/randgen
bash -c "set -o pipefail; perl /ssd/randgen/runall.pl --queries=100000000 --seed=7272 --duration=300 --querytimeout=60 --reporter=Shutdown,Backtrace,QueryTimeout,ErrorLog,ErrorLogAlarm --mysqld=--log-output=none --mysqld=--sql_mode=ONLY_FULL_GROUP_BY --mysqld=--slow_query_log --mysqld=--userstat --threads=25 --basedir=/ssd/Percona-Server-5.6.12-rc60.4-405-debug.Linux.x86_64 --grammar=/tmp/976291/75.yy --gendata=/ssd/randgen/conf/replication/replication_single_engine_pk.zz --mtr-build-thread=769 --mask=19835 --vardir1=/ssd/209184/cmdrun_414 > /ssd/209184/cmdrun414.log 2>&1"

Roel Van de Paar (roel11) wrote :

MTR testcase, almost identical to that of bug 1193308, but with partitioning specified:

# Test for bug 1193264 (handle_fatal_signal in ha_partition::clone (sql/ha_partition.cc:3412))

--source include/have_innodb.inc
--source include/have_debug_sync.inc

--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings

CREATE TABLE t1(a INT PRIMARY KEY, b INT, c INT) ENGINE=InnoDB PARTITION BY HASH(a) PARTITIONS 2;
INSERT INTO t1 VALUES (1, 1, 1);

SET DEBUG_SYNC= 'innodb_alter_commit_after_lock_table SIGNAL alter_ready WAIT_FOR i_s_completed';
send ALTER TABLE t1 DROP COLUMN c, ALGORITHM=INPLACE;

connect (conn2,localhost,root,,);
connection conn2;

SET DEBUG_SYNC= 'now WAIT_FOR alter_ready';
SELECT COUNT(*) FROM INFORMATION_SCHEMA.GLOBAL_TEMPORARY_TABLES;
SET DEBUG_SYNC= 'now SIGNAL i_s_completed';

disconnect conn2;
connection default;
reap;

DROP TABLE t1;

The cause is similar to bug 1193308. ha_partition::clone() dereferences NULL if this->table == NULL, which will happen for temp tables created for 5.6 ALTER.

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-681

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers