A crash that leaves behind an InnoDB temporary table with temporary indexes results in an unbootable server

[Laurynas Biveinis]

No concrete test case yet, but it needs a temporary table that has at least one index and a server crash that leaves such table behind.

Subsequent server reboots will try to remove such table and fail:

120514 6:43:57 InnoDB: Assertion failure in thread 47327229830816 in file dict0dict.c line 868
InnoDB: Failing assertion: table2 == NULL
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

[Laurynas Biveinis (laurynas-biveinis)]

To reproduce, add a new crash injection site at the end of non-error path in ha_innobase::add_index()
 DBUG_EXECUTE_IF("crash_innodb_add_index_after", DBUG_SUICIDE(););

and then run the following test case:

-- source include/not_embedded.inc
-- source include/not_valgrind.inc
-- source include/not_crashrep.inc
-- source include/have_debug.inc
-- source include/have_innodb.inc

--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings

SET SESSION expand_fast_index_creation=ON;

CREATE TEMPORARY TABLE t1 (a INT, b INT, INDEX(a));

--exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect

SET debug="+d,crash_innodb_add_index_after";
--error 2013
ALTER TABLE t1 ADD INDEX (b);

# Turn on reconnect
--enable_reconnect

# Call script that will poll the server waiting for it to be back online again
--source include/wait_until_connected_again.inc

SHOW TABLES;

The only place in InnoDB that uses temporary indexes is the regular table index creation and drop. This code however is not used for the temporary tables, as they are created anew with the final index definitions and then data is copied.

However the Percona Server expanded index creation might use the regular index creation and drop code path in a corner case with expanded fast index creation enabled: when there is an already-existing secondary index on the table, it will be dropped from the new temp table and recreated after copying the data.

[Laurynas Biveinis (laurynas-biveinis)]

This does not explain how does the upstream server end up in a similar situation, but it's possible that MySQL Cluster somehpw triggers the fast index creation path for InnoDB temp tables too.

[Sergei Glushchenko (sergei.glushchenko)]

5.6 is not affected by this bug. Bug 1133926 has a different root cause.

[Laurynas Biveinis (laurynas-biveinis)]

5.6 seems to be affected through the online alter code path, see the linked upstream bug.

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-557