Failing assertion: page_get_n_recs(page) > 1

[Laurent Bigonville]

Hi,

When upgrading from mysql (vanilla) 5.1.54 to mysql vanilla or perconna 5.5 we get this assertion error:

110509 17:50:44 InnoDB: Assertion failure in thread 139847062464272 in file ibuf0ibuf.c
line 4196
InnoDB: Failing assertion: page_get_n_recs(page) > 1
InnoDB: We intentionally generate a memory trap.

After this error, the database is crashing in loop and never comes back online.

Reverting to 5.1.54 doesn't help, the DB is still crashing

More information on http://bugs.mysql.com/bug.php?id=61104

[Laurent Bigonville (bigon)]

0x7ca7c9 my_print_stacktrace + 57
0x4ff7e9 handle_segfault + 889
0x7fed662505d0 _end + 1696332848
0x7fed65475945 _end + 1681805221
0x7fed65476f21 _end + 1681810817
0x90b591 ibuf_merge_or_delete_for_page + 5777
0x8b2c55 buf_page_io_complete + 1157
0x8e90c1 fil_aio_wait + 545
0x8649e0 io_handler_thread + 80
0x7fed662485f0 _end + 1696300112
0x7fed6551784d _end + 1682468525

[yinfeng (yinfeng-zwx)]

changelog for mysql5.5.21 :

InnoDB Storage Engine: The MySQL server could halt with an assertion error:

InnoDB: Failing assertion: page_get_n_recs(page) > 1
Subsequent restarts could fail with the same error. The error occurred during a purge operation involving the InnoDB change buffer. The workaround was to set the configuration option innodb_change_buffering=inserts. (Bug #13413535, Bug #61104)

[yinfeng (yinfeng-zwx)]

while it's not safe to use this new future , i think it's better to change defaults value of innodb_change_buffering=inserts

a simple fix

[Alexey Kopytov (akopytov)]

See also http://bugs.mysql.com/bug.php?id=66819

[Alexey Kopytov (akopytov)]

http://bugs.mysql.com/bug.php?id=66819 will be fixed upstream in 5.1.67, 5.5.29, 5.6.8, 5.7.0.

[yinfeng (yinfeng-zwx)]

Is there any separate patch to fix this bug?

[Alexey Kopytov (akopytov)]

To my knowledge, the fix is not publicly available yet.

[yinfeng (yinfeng-zwx)]

The upstream bug#66819 was marked as closed.

@Alexey Kopytov , I had read the related code of MySQL5.5.29. It seems MySQL5.5.29 didn't fix all potential bug you had pointed out.

IBUF_OP_DELETE operation in ibuf_merge_or_delete_for_page() still commit mtr before calling ibuf_delete_rec().

Related：
http://bazaar.launchpad.net/~mysql/mysql-server/5.5/revision/3979

[Raghavendra D Prabhu (raghavendra-prabhu)]

@yinfeng,

The latest comment in 66819 reiterates that.

So, it looks like purge buffering is still not crash safe.

[Bart Verwilst (verwilst)]

I have added several comments in the mysql bugtracker as a followup.. Using 5.5.30, i'm still seeing a lot of corruption issues.

[Alexey Kopytov (akopytov)]

Yes, the problem is still there for innodb_change_buffering values 'all' or 'purges'.

[Raghavendra D Prabhu (raghavendra-prabhu)]

5.5.31 seems to have fixed for purge as well

"
InnoDB: If the MySQL server halted at a precise moment when a purge operation was being applied from the change buffer, the operation could be incorrectly performed again during the next restart. A workaround was to set the configuration option innodb_change_buffering=changes, to turn off change buffering for purge operations. (Bug #16183892, Bug #14636528)
"

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-501