Percona Server moved to https://jira.percona.com/projects/PS

Failing assertion: !mutex_own(mutex) in sync0sync.ic line 307

  1. Series 5.6
  2. Bug #1435678
Bug #1435678 reported by Ramesh Sivaraman
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Status tracked in 5.7
5.1
Won't Fix
Undecided
Unassigned
5.5
New
Undecided
Unassigned
5.6
Triaged
High
Unassigned
5.7
Triaged
High
Unassigned

Bug Description

**GDB info

#0 0x00007faa6c746771 in pthread_kill () from /lib64/libpthread.so.0
#1 0x0000000000abb48e in my_write_core (sig=6) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-7
2.1/mysys/stacktrace.c:422
#2 0x00000000007312c3 in handle_fatal_signal (sig=6) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.
6.23-72.1/sql/signal_handler.cc:236
#3 <signal handler called>
#4 0x00007faa6b34a5c9 in raise () from /lib64/libc.so.6
#5 0x00007faa6b34bcd8 in abort () from /lib64/libc.so.6
#6 0x0000000000cf026e in mutex_enter_func (mutex=0x7faa65f925b8, file_name=0x10bbef8 "/mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_ex
p/centos6-64/percona-server-5.6.23-72.1/storage/innobase/dict/dict0dict.cc", line=5790, priority=DEFAULT_PRIO) at /mnt/workspace/percona-server-5.6-bi
naries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/include/sync0sync.ic:307
#7 0x0000000000cf0467 in pfs_mutex_enter_func (mutex=0x7faa65f925b8, file_name=0x10bbef8 "/mnt/workspace/percona-server-5.6-binaries-debug-yassl/labe
l_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/dict/dict0dict.cc", line=5790, priority=DEFAULT_PRIO) at /mnt/workspace/percona-server-5.
6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/include/sync0sync.ic:409
#8 0x0000000000d009e9 in dict_table_set_corrupt_by_space (space_id=1, need_mutex=1) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_e
xp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/dict/dict0dict.cc:5790
#9 0x0000000000cc3d1f in buf_page_io_complete (bpage=0x7faa5b720fa8) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/p
ercona-server-5.6.23-72.1/storage/innobase/buf/buf0buf.cc:4352
#10 0x0000000000ce1205 in buf_read_page_low (err=0x7faa6ccf785c, sync=true, mode=132, space=1, zip_size=0, unzip=0, tablespace_version=5, offset=0, tr
x=0x0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/buf/buf0rea.cc:258
#11 0x0000000000ce1656 in buf_read_page (space=1, zip_size=0, offset=0, trx=0x0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/c
entos6-64/percona-server-5.6.23-72.1/storage/innobase/buf/buf0rea.cc:462
#12 0x0000000000cbffb5 in buf_page_get_gen (space=1, zip_size=0, offset=0, rw_latch=2, guess=0x0, mode=10, file=0x10cb540 "/mnt/workspace/percona-serv
er-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/fsp/fsp0fsp.cc", line=200, mtr=0x7faa6ccf7bb0) at /mnt/wo
rkspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/buf/buf0buf.cc:2713
#13 0x0000000000d37e20 in fsp_get_space_header (id=1, zip_size=0, mtr=0x7faa6ccf7bb0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_
exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/fsp/fsp0fsp.cc:200
#14 0x0000000000d3d8e8 in fsp_reserve_free_extents (n_reserved=0x7faa6ccf7a80, space=1, n_ext=3, alloc_type=1000000, mtr=0x7faa6ccf7bb0) at /mnt/works
pace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/fsp/fsp0fsp.cc:2777
#15 0x0000000000c9dfb0 in btr_cur_pessimistic_insert (flags=0, cursor=0x7faa6ccf7b30, offsets=0x7faa6ccf80a0, heap=0x7faa6ccf8090, entry=0x7faa4c06cd7
8, rec=0x7faa6ccf8088, big_rec=0x7faa6ccf8098, n_ext=0, thr=0x7faa4c075658, mtr=0x7faa6ccf7bb0) at /mnt/workspace/percona-server-5.6-binaries-debug-ya
ssl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/btr/btr0cur.cc:1681
#16 0x0000000000bc49e6 in row_ins_clust_index_entry_low (flags=0, mode=33, index=0x7faa4c06c5f8, n_uniq=2, entry=0x7faa4c06cd78, n_ext=0, thr=0x7faa4c
075658) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/row/row0ins.cc:2512
#17 0x0000000000bc5936 in row_ins_clust_index_entry (index=0x7faa4c06c5f8, entry=0x7faa4c06cd78, thr=0x7faa4c075658, n_ext=0) at /mnt/workspace/percon
a-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/row/row0ins.cc:2918
#18 0x0000000000bc5b57 in row_ins_index_entry (index=0x7faa4c06c5f8, entry=0x7faa4c06cd78, thr=0x7faa4c075658) at /mnt/workspace/percona-server-5.6-bi
naries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/row/row0ins.cc:2995
#19 0x0000000000bc5e38 in row_ins_index_entry_step (node=0x7faa4c0753c8, thr=0x7faa4c075658) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl
/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/row/row0ins.cc:3072
#20 0x0000000000bc613c in row_ins (node=0x7faa4c0753c8, thr=0x7faa4c075658) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/row/row0ins.cc:3212
#21 0x0000000000bc64c1 in row_ins_step (thr=0x7faa4c075658) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/row/row0ins.cc:3337
#22 0x0000000000b99f4c in que_thr_step (thr=0x7faa4c075658) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/que/que0que.cc:1095
#23 0x0000000000b9a26f in que_run_threads_low (thr=0x7faa4c075658) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/que/que0que.cc:1177
#24 0x0000000000b9a43e in que_run_threads (thr=0x7faa4c075658) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/que/que0que.cc:1218
#25 0x0000000000b9a6e6 in que_eval_sql (info=0x7faa4c06c878, sql=0x10c34c8 "PROCEDURE TABLE_STATS_SAVE () IS\nBEGIN\nDELETE FROM \"mysql/innodb_table_
stats\"\nWHERE\ndatabase_name = :database_name AND\ntable_name = :table_name;\nINSERT INTO \"mysql/innodb_table_stats\"\nVALUES\n(\n:database_name,\n:
table_name,\n:last_update,\n:n_rows,\n:clustered_index_size,\n:sum_of_other_index_sizes\n);\nEND;", reserve_dict_mutex=0, trx=0x7faa4c0ca678) at /mnt/
workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/que/que0que.cc:1301
#26 0x0000000000d16c88 in dict_stats_exec_sql (pinfo=0x7faa4c06c878, sql=0x10c34c8 "PROCEDURE TABLE_STATS_SAVE () IS\nBEGIN\nDELETE FROM \"mysql/innod
b_table_stats\"\nWHERE\ndatabase_name = :database_name AND\ntable_name = :table_name;\nINSERT INTO \"mysql/innodb_table_stats\"\nVALUES\n(\n:database_
name,\n:table_name,\n:last_update,\n:n_rows,\n:clustered_index_size,\n:sum_of_other_index_sizes\n);\nEND;", trx=0x7faa4c0ca678) at /mnt/workspace/perc
ona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/dict/dict0stats.cc:317
#27 0x0000000000d1a5e0 in dict_stats_save (table_orig=0x7faa4c337478, only_for_index=0x0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/la
bel_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/dict/dict0stats.cc:2424
#28 0x0000000000d1bc87 in dict_stats_update (table=0x7faa4c337478, stats_upd_option=DICT_STATS_EMPTY_TABLE) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/dict/dict0stats.cc:3166
#29 0x0000000000ae0c3c in ha_innobase::create (this=0x7faa4c0a9510, name=0x7faa6ccfb930 "./test/bench_2769", form=0x7faa6ccf9970, create_info=0x7faa6ccfc240) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/innobase/handler/ha_innodb.cc:10487
#30 0x0000000000647dee in handler::ha_create (this=0x7faa4c0a9510, name=0x7faa6ccfb930 "./test/bench_2769", form=0x7faa6ccf9970, info=0x7faa6ccfc240) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/handler.cc:4722
#31 0x0000000000648c93 in ha_create_table (thd=0x7faa5ab71000, path=0x7faa6ccfb930 "./test/bench_2769", db=0x7faa4c01f670 "test", table_name=0x7faa4c01f0c8 "bench_2769", create_info=0x7faa6ccfc240, update_create_info=false, is_temp_table=false) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/handler.cc:5069
#32 0x00000000008c2622 in rea_create_table (thd=0x7faa5ab71000, path=0x7faa6ccfb930 "./test/bench_2769", db=0x7faa4c01f670 "test", table_name=0x7faa4c01f0c8 "bench_2769", create_info=0x7faa6ccfc240, create_fields=..., keys=0, key_info=0x7faa4c01fe70, file=0x7faa4c01f998, no_ha_table=false) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/unireg.cc:527
#33 0x0000000000855827 in create_table_impl (thd=0x7faa5ab71000, db=0x7faa4c01f670 "test", table_name=0x7faa4c01f0c8 "bench_2769", path=0x7faa6ccfb930 "./test/bench_2769", create_info=0x7faa6ccfc240, alter_info=0x7faa6ccfbce0, internal_tmp_table=false, select_field_count=0, no_ha_table=false, is_trans=0x7faa6ccfbbbe, key_info=0x7faa6ccfbb38, key_count=0x7faa6ccfbb34) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_table.cc:4977
#34 0x0000000000855cf2 in mysql_create_table_no_lock (thd=0x7faa5ab71000, db=0x7faa4c01f670 "test", table_name=0x7faa4c01f0c8 "bench_2769", create_info=0x7faa6ccfc240, alter_info=0x7faa6ccfbce0, select_field_count=0, is_trans=0x7faa6ccfbbbe) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_table.cc:5087
#35 0x0000000000855e35 in mysql_create_table (thd=0x7faa5ab71000, create_table=0x7faa4c01f110, create_info=0x7faa6ccfc240, alter_info=0x7faa6ccfbce0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_table.cc:5139
#36 0x00000000007e5c13 in mysql_execute_command (thd=0x7faa5ab71000) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_parse.cc:3500
#37 0x00000000007ef059 in mysql_parse (thd=0x7faa5ab71000, rawbuf=0x7faa4c01f010 "create table bench_2769(i int)", length=30, parser_state=0x7faa6ccfcdf0) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_parse.cc:6922
#38 0x00000000007e0ecf in dispatch_command (command=COM_QUERY, thd=0x7faa5ab71000, packet=0x7faa53e91001 "create table bench_2769(i int)", packet_length=30) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_parse.cc:1434
#39 0x00000000007dfded in do_command (thd=0x7faa5ab71000) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_parse.cc:1051
#40 0x00000000007a7d0d in do_handle_one_connection (thd_arg=0x7faa5ab71000) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_connect.cc:1532
#41 0x00000000007a7815 in handle_one_connection (arg=0x7faa5ab71000) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/sql/sql_connect.cc:1443
#42 0x0000000000dcc8c0 in pfs_spawn_thread (arg=0x7faa65f41240) at /mnt/workspace/percona-server-5.6-binaries-debug-yassl/label_exp/centos6-64/percona-server-5.6.23-72.1/storage/perfschema/pfs.cc:1860
#43 0x00007faa6c741df3 in start_thread () from /lib64/libpthread.so.0
#44 0x00007faa6b40b1ad in clone () from /lib64/libc.so.6

** Testcase info

Testcase have 250 sql statements. Could not reduce the testcase further even after trying manually. All sql statements are "CREATE TABLE" except last 4 lines.

DROP DATABASE test;CREATE DATABASE test;USE test;
create table bench_1784(i int);
[..]
create table bench_2765(i int);
SET GLOBAL innodb_buffer_pool_dump_at_shutdown=ON;
SET GLOBAL innodb_corrupt_table_action='warn';
SET @orig=@@global.innodb_buffer_pool_dump_at_shutdown;
SET GLOBAL innodb_checksum_algorithm=@orig;
create table bench_2769(i int);

No extra mysqld start-up variable is required.

When we execute last 5 lines from testcase getting assertion similar to https://bugs.launchpad.net/percona-server/+bug/1385030

The attached tarball gives the testcase as an exact match of our system, including some handy utilities

$ vi {epoch}_mybase # Update base path in this file (the only change
required!). For non-binary distribution please update SOURCE_DIR location also.
$ ./{epoch}_init # Initializes the data dir
$ ./{epoch}_start # Starts mysqld
$ ./{epoch}_cl # To check mysqld is up
$ ./{epoch}_run # Run the testcase with pquery binary(produces
output)
$ vi /dev/shm/{epoch}/error.log.out # Verify the error log
$ ./{epoch}_gdb # Brings you to a gdb prompt attached to correct
mysqld
& generated core
$ ./{epoch}_parse_core # Create {epoch}_STD.gdb and {epoch}_FULL.gdb;
standard and full var gdb stack traces
etc.

Revision history for this message
Ramesh Sivaraman (rameshvs02) wrote :
Laurynas Biveinis (laurynas-biveinis)
tags: added: corrupt-table-action
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-884

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.