Crash on page 6

[Daniël van Eeden]

[FATAL] InnoDB: Apparent corruption of an index page [page id: space=0, page number=6] to be written to data file. We intentionally crash the server to prevent corrupt data from ending up in data files.
2017-03-13 13:01:11 0x7f11067fc700 InnoDB: Assertion failure in thread 139711100208896 in file ut0ut.cc line 916
InnoDB: We intentionally generate a memory trap.

This might be related to
https://github.com/percona/percona-xtrabackup/commit/a96a71037320a48645b93df4b02d399bb98f3c3e
https://bugs.launchpad.net/percona-xtrabackup/+bug/1641426
but I'm not sure at all about that.

This is identical to what happens on this question:
https://answers.launchpad.net/percona-xtrabackup/+question/550345

This happens for me with:
xtrabackup version 2.4.5 based on MySQL server 5.7.13 Linux (x86_64) (revision id: e41c0be)
MySQL 5.6.35

This instance was probably created with 5.5 or before and upgraded to 5.6.

When I stop writes during the backup (stop slave sql_thread) then the apply-log works

When the apply log works:
InnoDB: Resetting invalid page [page id: space=0, page number=3] type 0 to 6.
InnoDB: Resetting invalid page [page id: space=0, page number=5] type 0 to 7.
InnoDB: Resetting invalid page [page id: space=0, page number=6] type 0 to 6.
InnoDB: Resetting invalid page [page id: space=0, page number=7] type 0 to 6.

When it fails:
InnoDB: Resetting invalid page [page id: space=0, page number=16384] type 17855 to 9 when flushing.
InnoDB: Resetting invalid page [page id: space=0, page number=0] type 0 to 8 when flushing.
InnoDB: Resetting invalid page [page id: space=0, page number=5] type 0 to 7 when flushing.
InnoDB: Resetting invalid page [page id: space=0, page number=32768] type 17855 to 9 when flushing.

Note that the '...when flushing' messages come from the recently patched buf_flush_init_for_writing() in buf0flu.cc and the other message comes from fil_page_reset_type() from fil0fil.cc

[Daniël van Eeden (dveeden)]

buf_dblwr_assert_on_corrupt_block() is what is called on page 6.

So it looks like we *might* have to reset page=6 to FIL_PAGE_TYPE_SYS.

[Daniël van Eeden (dveeden)]

Now testing this:
https://github.com/dveeden/percona-xtrabackup/tree/bug1672374

[Daniël van Eeden (dveeden)]

I made two copies of a backup. And the apply-log fails w/o the patch and succeeds with the patch.

[Daniël van Eeden (dveeden)]

https://github.com/percona/percona-xtrabackup/pull/330

[Shahriyar Rzayev (rzayev-sehriyar)]

Seems to be same as -> https://bugs.launchpad.net/percona-xtrabackup/+bug/1671722

[Daniël van Eeden (dveeden)]

@rzayev-sehriyar that url returns a page not found for me

[Shahriyar Rzayev (rzayev-sehriyar)]

Oh, sorry, for that, but for some reason, original reporter marked it as "Private Security" report.

[Sergei Glushchenko (sergei.glushchenko)]

fixed by PR https://github.com/percona/percona-xtrabackup/pull/379