: Page directory corruption: infimum not pointed to

Bug #1643780 reported by mejsin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona XtraBackup moved to https://jira.percona.com/projects/PXB
New
Undecided
Unassigned

Bug Description

Hi
When using xtrabackup for restore we get the error and is only able to start db with inndo_:
InnoDB: Doing recovery: scanned up to log sequence number 9704698992 (0%)
InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: 128 rollback segment(s) are active.
InnoDB: Waiting for purge to start
InnoDB: Page directory corruption: infimum not pointed to
2016-11-18 19:05:47 7f31659ab700 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex deadbeef00000000000000000000000000000000000000000000000000000000000000000000434f4d504143545000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
..
0000000000000000000000000000000000000000000deadbeef00000000; asc COMPACTP
InnoDB: End of page dump
2016-11-18 19:05:47 7f31659ab700 InnoDB: uncompressed page, stored checksum in field1 3735928559, calculated checksums for field1: crc32 177807968, innodb 3475755373, none 3735928559, stored checksum in field2 3735928559, calculated checksums for field2: crc32 177807968, innodb 1624572976, none 3735928559, page LSN 0 0, low 4 bytes of LSN at page end 0, page number (if stored to page already) 0, space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
InnoDB: Page directory corruption: supremum not pointed to
2016-11-18 19:05:47 7f31659ab700 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex deadbeef00000000000000000000000000000000000000000000000000000000000000000000434f4d5041435450000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

InnoDB: End of page dump
2016-11-18 19:05:47 7f31659ab700 InnoDB: uncompressed page, stored checksum in field1 3735928559, calculated checksums for field1: crc32 177807968, innodb 3475755373, none 3735928559, stored checksum in field2 3735928559, calculated checksums for field2: crc32 177807968, innodb 1624572976, none 3735928559, page LSN 0 0, low 4 bytes of LSN at page end 0, page number (if stored to page already) 0, space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
..
InnoDB: End of page dump
2016-11-18 19:05:47 7f31659ab700 InnoDB: uncompressed page, stored checksum in field1 3735928559, calculated checksums for field1: crc32 177807968, innodb 3475755373, none 3735928559, stored checksum in field2 3735928559, calculated checksums for field2: crc32 177807968, innodb 1624572976, none 3735928559, page LSN 0 0, low 4 bytes of LSN at page end 0, page number (if stored to page already) 0, space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a freshly allocated page
2016-11-18 19:05:47 7f31659ab700 InnoDB: Assertion failure in thread 139850134763264 in file page0page.ic line 775
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
19:05:47 UTC - xtrabackup got signal 6 ;
This could be because you hit a bug or data is corrupted.
This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x10000
xtrabackup(my_print_stacktrace+0x2e) [0x8f139e]
xtrabackup(handle_fatal_signal+0x262) [0x7d1c82]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340) [0x7f317984c340]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7f3177fb0cc9]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x7f3177fb40d8]
xtrabackup(page_cur_search_with_match(buf_block_t const*, dict_index_t const*, dtuple_t const*, unsigned long, unsigned long*, unsigned long*, unsigned long*, unsigned long*, page_cur_t*)+0x45d) [0x6d87dd]
xtrabackup(btr_cur_search_to_nth_level(dict_index_t*, unsigned long, dtuple_t const*, unsigned long, unsigned long, btr_cur_t*, unsigned long, char const*, unsigned long, mtr_t*)+0x3a2) [0x5e9b52]
xtrabackup(row_search_index_entry(dict_index_t*, dtuple_t const*, unsigned long, btr_pcur_t*, mtr_t*)+0x86) [0x6a1616]
xtrabackup() [0x6c3aa0]
xtrabackup(row_purge_step(que_thr_t*)+0x4b7) [0x6c4cc7]
xtrabackup(que_run_threads(que_thr_t*)+0x498) [0x6178c8]
xtrabackup(trx_purge(unsigned long, unsigned long, bool)+0x776) [0x7021a6]
xtrabackup(srv_purge_coordinator_thread+0x27f) [0x68034f]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182) [0x7f3179844182]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f317807447d]

Version is:
 xtrabackup --version
xtrabackup version 2.3.3 based on MySQL server 5.6.24 Linux (x86_64) (revision id: 525ca7d)

server:
 3.13.0-36-generic #63-Ubuntu SMP Wed Sep 3 21:30:07 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Revision history for this message
mejsin (mejsin) wrote :

running:
xtrabackup --prepare --target-dir=/tmp/restore/ --rebuild-indexes --rebuild-threads=8

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Hi,
I think there is already one reported here -> https://bugs.launchpad.net/percona-xtrabackup/+bug/1192834

Revision history for this message
mejsin (mejsin) wrote :

Hi,

Thanks for link, but unsure about status of the bug (https://bugs.launchpad.net/percona-xtrabackup/+bug/1192834) is it fixed or ??

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Nope.
https://blog.launchpad.net/general/of-bugs-and-statuses
According to this "Triaged" means:

"Triaged (Keep This Bug On The Radar)

This is an intermediate status that can be used to distinguish community-confirmed bugs from bugs that the official QA team has acknowledged and plans on working on. Many projects don’t need a separate stepping stone and developers can pick bugs directly from Confirmed into In Progress. At this point, we are sure that a) the bug has been reproduced and established b) developers have acknowledged."

So it is not fixed yet.

Thank you for report, marking this as duplicate of #1192834

Revision history for this message
mejsin (mejsin) wrote :

Hi,

Is there any workaround for this bug ? as it's critical for us in our prod system

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

Only known workaround is don't use compact backups

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.