InnoDB: Assertion failure in thread 139864880994048 in file trx0sys.cc line 181 | UNIV_DEBUG

[Raghavendra D Prabhu]

Hit this crash with UNIV_DEBUG build:

2014-01-20 08:35:32 7f34d48c5700 InnoDB: Assertion failure in thread 139864880994048 in file trx0sys.cc line 181
InnoDB: Failing assertion: mutex_own(&trx_sys->mutex)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
13:35:32 UTC - mysqld got signal 6 ;

Backtrace:

# 2014-01-20T08:35:53 [32197] #0 0x00007f34ee5cf8ac in pthread_kill () from /lib64/libpthread.so.0
# 2014-01-20T08:35:53 [32197] #1 0x00000000006a3d6e in handle_fatal_signal (sig=6) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/signal_handler.cc:254
# 2014-01-20T08:35:53 [32197] #2 <signal handler called>
# 2014-01-20T08:35:53 [32197] #3 0x00007f34ecc32925 in raise () from /lib64/libc.so.6
# 2014-01-20T08:35:53 [32197] #4 0x00007f34ecc34105 in abort () from /lib64/libc.so.6
# 2014-01-20T08:35:53 [32197] #5 0x0000000000b4c13d in trx_sys_flush_max_trx_id () at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/storage/innobase/trx/trx0sys.cc:181
# 2014-01-20T08:35:53 [32197] #6 0x0000000000a09e21 in trx_sys_get_new_trx_id (hton=<value optimized out>, thd=<value optimized out>) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/storage/innobase/include/trx0sys.ic:491
# 2014-01-20T08:35:53 [32197] #7 wsrep_fake_trx_id (hton=<value optimized out>, thd=<value optimized out>) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/storage/innobase/handler/ha_innodb.cc:17772
# 2014-01-20T08:35:53 [32197] #8 0x00000000005cd75d in ha_wsrep_fake_trx_id (thd=0x3648aa0) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/handler.cc:7783
# 2014-01-20T08:35:53 [32197] #9 0x0000000000724487 in select_create::binlog_show_create_table (this=0x7f34700018c8, tables=<value optimized out>, count=<value optimized out>) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_insert.cc:4352
# 2014-01-20T08:35:53 [32197] #10 0x0000000000724766 in postlock (this=0x7f34700018c8) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/handler.h:1437
# 2014-01-20T08:35:53 [32197] #11 select_create::prepare2 (this=0x7f34700018c8) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_insert.cc:4242
# 2014-01-20T08:35:53 [32197] #12 0x000000000076b1fe in JOIN::prepare_result (this=0x7f3470001a48, columns_list=<value optimized out>) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_select.cc:820
# 2014-01-20T08:35:53 [32197] #13 0x000000000071d401 in JOIN::exec (this=0x7f3470001a48) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_executor.cc:116
# 2014-01-20T08:35:53 [32197] #14 0x000000000076e1b0 in mysql_execute_select (thd=0x3648aa0, tables=0x7f34700011a8, wild_num=1, fields=..., conds=0x0, order=<value optimized out>, group=0x364b900, having=0x0, select_options=2416184064, result=0x7f34700018c8, unit=0x364b1b8, select_lex=0x364b800) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_select.cc:1101
# 2014-01-20T08:35:53 [32197] #15 mysql_select (thd=0x3648aa0, tables=0x7f34700011a8, wild_num=1, fields=..., conds=0x0, order=<value optimized out>, group=0x364b900, having=0x0, select_options=2416184064, result=0x7f34700018c8, unit=0x364b1b8, select_lex=0x364b800) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_select.cc:1222
# 2014-01-20T08:35:53 [32197] #16 0x000000000076ecab in handle_select (thd=0x3648aa0, result=0x7f34700018c8, setup_tables_done_option=0) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_select.cc:111
# 2014-01-20T08:35:53 [32197] #17 0x00000000007490d2 in mysql_execute_command (thd=0x3648b90) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_parse.cc:3667
# 2014-01-20T08:35:53 [32197] #18 0x0000000000749e08 in mysql_parse (thd=0x3648aa0, rawbuf=0x364b0e8 "P&>\001", length=<value optimized out>, parser_state=<value optimized out>) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_parse.cc:7188
# 2014-01-20T08:35:53 [32197] #19 0x0000000000749fff in wsrep_mysql_parse (thd=0x3648aa0, rawbuf=0x7f34700008d0 "CREATE TABLE s AS SELECT * FROM `table100_innodb_key_pk_parts_2_int_autoinc`", length=77, parser_state=0x7f34d48c4920) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_parse.cc:6940
# 2014-01-20T08:35:53 [32197] #20 0x000000000074bf6d in dispatch_command (command=COM_QUERY, thd=0x3648aa0, packet=0x36366e1 " CREATE TABLE s AS SELECT * FROM `table100_innodb_key_pk_parts_2_int_autoinc`", packet_length=78) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_parse.cc:1630
# 2014-01-20T08:35:53 [32197] #21 0x000000000074d6bf in do_command (thd=0x3648aa0) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_parse.cc:1133
# 2014-01-20T08:35:53 [32197] #22 0x000000000070db3f in do_handle_one_connection (thd_arg=<value optimized out>) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_connect.cc:1557
# 2014-01-20T08:35:53 [32197] #23 0x000000000070dd17 in handle_one_connection (arg=0x3648aa0) at /mnt/workspace/build-xtradb-cluster-binaries-56/BUILD_TYPE/debug/label_exp/centos6-64/Percona-XtraDB-Cluster-5.6.15/sql/sql_connect.cc:1461
# 2014-01-20T08:35:53 [32197] #24 0x00007f34ee5ca9d1 in start_thread () from /lib64/libpthread.so.0
# 2014-01-20T08:35:53 [32197] #25 0x00007f34ecce8b6d in clone () from /lib64/libc.so.6

Now, this is because

Allocates a new transaction id.
@return new, allocated trx id */
UNIV_INLINE
trx_id_t
trx_sys_get_new_trx_id(void)
/*========================*/
{
#ifndef WITH_WSREP
 /* wsrep_fake_trx_id violates this assert */
 ut_ad(mutex_own(&trx_sys->mutex));
#endif /* WITH_WSREP */

 /* VERY important: after the database is started, max_trx_id value is
 divisible by TRX_SYS_TRX_ID_WRITE_MARGIN, and the following if
 will evaluate to TRUE when this function is first time called,
 and the value for trx id will be written to disk-based header!
 Thus trx id values will not overlap when the database is
 repeatedly started! */

 if (!(trx_sys->max_trx_id % (trx_id_t) TRX_SYS_TRX_ID_WRITE_MARGIN)) {

  trx_sys_flush_max_trx_id();
 }

 return(trx_sys->max_trx_id++);
}

trx_sys_flush_max_trx_id again has the same assert:

Writes the value of max_trx_id to the file based trx system header. */
UNIV_INTERN
void
trx_sys_flush_max_trx_id(void)
/*==========================*/
{
 mtr_t mtr;
 trx_sysf_t* sys_header;

 ut_ad(mutex_own(&trx_sys->mutex));
 ---------------------------------> crashes here

 if (!srv_read_only_mode) {
  mtr_start(&mtr);

  sys_header = trx_sysf_get(&mtr);

  mlog_write_ull(
   sys_header + TRX_SYS_TRX_ID_STORE,
   trx_sys->max_trx_id, &mtr);

  mtr_commit(&mtr);
 }
}

So, need to add ifdef around it. (btw, why is the assertion
violated in first place?)

[Seppo Jaakola (seppo-jaakola)]

Merged the fix from PXC branch into wsrep-5.6, in revision:
http://bazaar.launchpad.net/~codership/codership-mysql/5.6/revision/4035

[Raghavendra D Prabhu (raghavendra-prabhu)]

Not required for 5.5.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-1588