InnoDB table has corrupted after unique index of that table is corrupted

Bug #1616336 reported by Mars
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
New
Undecided
Unassigned

Bug Description

Hi,

There is "maybe hit a mysql bug" in my error log.
One table is corrupted after the unique index of this table was corrupted, and there is no any error info before the error info of "unique index is corrupted."

OS: Ubuntu 14.04.1 LTS
MySQL server: Percona 5.6.26

the schema of table likes below:
create table tbl(
    id int not null auto_increment primary key,
    testid varchar(64) not null,
    col1 int not null,
    unique key uq (testid)
) engine=InnoDB default charset utf8mb4;

The table contains about 200 columns.

error info is below:

2016-08-24 11:31:41 8047 [ERROR] /usr/local/percona/bin/mysqld: Sort aborted: Query execution was interrupted
2016-08-24 11:32:12 8047 [ERROR] /usr/local/percona/bin/mysqld: Sort aborted: Query execution was interrupted
2016-08-24 11:32:16 8047 [ERROR] /usr/local/percona/bin/mysqld: Sort aborted: Query execution was interrupted
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 31144.
InnoDB: You may have to recover from a backup.
2016-08-24 11:43:18 7f24ae5a7700 InnoDB: Page dump in ascii and hex (16384 bytes):
 len 16384; hex ....
InnoDB: End of page dump
2016-08-24 11:43:18 7f24ae5a7700 InnoDB: uncompressed page, stored checksum in field1 338203751, calculated checksums for field1: crc32 4194159487, innodb 4269339206, none 3735928559, stored checksum in field2 3385392786, calculated checksums for field2: crc32 4194159487, innodb 3385392786, none 3735928559, page LSN 369 474393477, low 4 bytes of LSN at page end 474393477, page number (if stored to page already) 31144, space id (if created with >= MySQL-4.1.1 and stored already) 139
InnoDB: Page may be an index page where index id is 1020
InnoDB: (index "PRIMARY" of table "dbname"."tablename")
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 31144.
InnoDB: You may have to recover from a backup.
InnoDB: It is also possible that your operating
InnoDB: system has corrupted its own file cache
InnoDB: and rebooting your computer removes the
InnoDB: error.
InnoDB: If the corrupt page is an index page
InnoDB: you can also try to fix the corruption
InnoDB: by dumping, dropping, and reimporting
InnoDB: the corrupt table. You can use CHECK
InnoDB: TABLE to scan your table for corruption.
InnoDB: See also http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
2016-08-24 11:43:18 7f24ae5a7700 InnoDB: Assertion failure in thread 139795520714496 in file buf0lru.cc line 2321
InnoDB: Failing assertion: bpage->buf_fix_count == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
03:43:18 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=66
max_threads=502
thread_count=44
connection_count=44
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 216087 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
2016-08-24 11:43:18 8047 [ERROR] Got error 180 when reading table './dbname/tablename'
stack_bottom = 0 thread_stack 0x30000
/usr/local/percona/bin/mysqld(my_print_stacktrace+0x2e)[0x8de2ae]
/usr/local/percona/bin/mysqld(handle_fatal_signal+0x491)[0x6a66b1]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7f25e3f95340]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39)[0x7f25e2554bb9]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7f25e2557fc8]
/usr/local/percona/bin/mysqld[0xa39273]
/usr/local/percona/bin/mysqld[0xa3ee1c]
/usr/local/percona/bin/mysqld[0xa2aa79]
/usr/local/percona/bin/mysqld[0xa79a58]
/usr/local/percona/bin/mysqld[0x9c74b0]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)[0x7f25e3f8d182]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f25e2618efd]
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
160824 11:43:19 mysqld_safe Number of processes running now: 0
160824 11:43:19 mysqld_safe mysqld restarted
2016-08-24 11:43:19 0 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
2016-08-24 11:43:19 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2016-08-24 11:43:19 0 [Note] /usr/local/percona/bin/mysqld (mysqld 5.6.26-74.0-log) starting as process 4208 ...
2016-08-24 11:43:19 4208 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
2016-08-24 11:43:19 4208 [Note] Plugin 'FEDERATED' is disabled.
2016-08-24 11:43:19 4208 [Note] InnoDB: Using atomics to ref count buffer pool pages
2016-08-24 11:43:19 4208 [Note] InnoDB: The InnoDB memory heap is disabled
2016-08-24 11:43:19 4208 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2016-08-24 11:43:19 4208 [Note] InnoDB: Memory barrier is not used
2016-08-24 11:43:19 4208 [Note] InnoDB: Compressed tables use zlib 1.2.3
2016-08-24 11:43:19 4208 [Note] InnoDB: Using Linux native AIO
2016-08-24 11:43:19 4208 [Note] InnoDB: Not using CPU crc32 instructions
2016-08-24 11:43:19 4208 [Note] InnoDB: Initializing buffer pool, size = 4.0G
2016-08-24 11:43:19 4208 [Note] InnoDB: Completed initialization of buffer pool
2016-08-24 11:43:19 4208 [Note] InnoDB: Highest supported file format is Barracuda.
2016-08-24 11:43:19 4208 [Note] InnoDB: Log scan progressed past the checkpoint lsn 1591363417614
2016-08-24 11:43:19 4208 [Note] InnoDB: Database was not shutdown normally!
2016-08-24 11:43:19 4208 [Note] InnoDB: Starting crash recovery.
2016-08-24 11:43:19 4208 [Note] InnoDB: Reading tablespace information from the .ibd files...
2016-08-24 11:43:20 4208 [Note] InnoDB: Restoring possible half-written data pages
2016-08-24 11:43:20 4208 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1591363614788
2016-08-24 11:43:20 4208 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 211974841, file name mysqlbin3306.000670
2016-08-24 11:43:21 4208 [Note] InnoDB: 128 rollback segment(s) are active.
2016-08-24 11:43:21 4208 [Note] InnoDB: Waiting for purge to start
2016-08-24 11:43:21 4208 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.26-rel74.0 started; log sequence number 1591363614788
2016-08-24 11:43:21 4208 [Note] Recovering after a crash using $datadir/mysqlbin3306
2016-08-24 11:43:22 4208 [Note] Starting crash recovery...
2016-08-24 11:43:22 4208 [Note] Crash recovery finished.
2016-08-24 11:43:22 4208 [Note] RSA private key file not found: $datadir/private_key.pem. Some authentication plugins will not work.
2016-08-24 11:43:22 4208 [Note] RSA public key file not found: $datadir/public_key.pem. Some authentication plugins will not work.
2016-08-24 11:43:22 4208 [Note] Server hostname (bind-address): '*'; port: 3306
2016-08-24 11:43:22 4208 [Note] IPv6 is available.
2016-08-24 11:43:22 4208 [Note] - '::' resolves to '::';
2016-08-24 11:43:22 4208 [Note] Server socket created on IP: '::'.
2016-08-24 11:43:22 4208 [Warning] 'user' entry 'root@hostname' ignored in --skip-name-resolve mode.
2016-08-24 11:43:22 4208 [Warning] 'proxies_priv' entry '@ root@hostname' ignored in --skip-name-resolve mode.
2016-08-24 11:43:22 4208 [Note] Event Scheduler: Loaded 0 events
2016-08-24 11:43:22 4208 [Note] /usr/local/percona/bin/mysqld: ready for connections.
Version: '5.6.26-74.0-log' socket: '$socketdir/mysql.sock' port: 3306 Percona Server (GPL), Release 74.0, Revision 32f8dfd
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 31144.
InnoDB: You may have to recover from a backup.
2016-08-24 11:54:14 7f7e6166d700 InnoDB: Page dump in ascii and hex (16384 bytes):
 len 16384;
InnoDB: End of page dump
2016-08-24 11:54:14 7f7e6166d700 InnoDB: uncompressed page, stored checksum in field1 338203751, calculated checksums for field1: crc32 4194159487, innodb 4269339206, none 3735928559, stored checksum in field2 3385392786, calculated checksums for field2: crc32 4194159487, innodb 3385392786, none 3735928559, page LSN 369 474393477, low 4 bytes of LSN at page end 474393477, page number (if stored to page already) 31144, space id (if created with >= MySQL-4.1.1 and stored already) 139
InnoDB: Page may be an index page where index id is 1020
InnoDB: (index "PRIMARY" of table "dbname"."tablename")
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 31144.
InnoDB: You may have to recover from a backup.
InnoDB: It is also possible that your operating
InnoDB: system has corrupted its own file cache
InnoDB: and rebooting your computer removes the
InnoDB: error.
InnoDB: If the corrupt page is an index page
InnoDB: you can also try to fix the corruption
InnoDB: by dumping, dropping, and reimporting
InnoDB: the corrupt table. You can use CHECK
InnoDB: TABLE to scan your table for corruption.
InnoDB: See also http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
2016-08-24 11:54:14 7f7e6166d700 InnoDB: Assertion failure in thread 140180776736512 in file buf0lru.cc line 2321
InnoDB: Failing assertion: bpage->buf_fix_count == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
03:54:14 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=40
max_threads=502
thread_count=40
connection_count=40
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 216087 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/local/percona/bin/mysqld(my_print_stacktrace+0x2e)[0x8de2ae]
/usr/local/percona/bin/mysqld(handle_fatal_signal+0x491)[0x6a66b1]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7f7f96c12340]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39)[0x7f7f951d1bb9]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7f7f951d4fc8]
/usr/local/percona/bin/mysqld[0xa39273]
/usr/local/percona/bin/mysqld[0xa3ee1c]
/usr/local/percona/bin/mysqld[0xa2aa79]
/usr/local/percona/bin/mysqld[0xa79a58]
/usr/local/percona/bin/mysqld[0x9c74b0]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)[0x7f7f96c0a182]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f7f95295efd]
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
160824 11:54:14 mysqld_safe Number of processes running now: 0
160824 11:54:14 mysqld_safe mysqld restarted
2016-08-24 11:54:14 0 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
2016-08-24 11:54:14 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2016-08-24 11:54:14 0 [Note] /usr/local/percona/bin/mysqld (mysqld 5.6.26-74.0-log) starting as process 8471 ...
2016-08-24 11:54:14 8471 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
2016-08-24 11:54:14 8471 [Note] Plugin 'FEDERATED' is disabled.
2016-08-24 11:54:14 8471 [Note] InnoDB: Using atomics to ref count buffer pool pages
2016-08-24 11:54:14 8471 [Note] InnoDB: The InnoDB memory heap is disabled
2016-08-24 11:54:14 8471 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2016-08-24 11:54:14 8471 [Note] InnoDB: Memory barrier is not used
2016-08-24 11:54:14 8471 [Note] InnoDB: Compressed tables use zlib 1.2.3
2016-08-24 11:54:14 8471 [Note] InnoDB: Using Linux native AIO
2016-08-24 11:54:14 8471 [Note] InnoDB: Not using CPU crc32 instructions
2016-08-24 11:54:14 8471 [Note] InnoDB: Initializing buffer pool, size = 4.0G
2016-08-24 11:54:15 8471 [Note] InnoDB: Completed initialization of buffer pool
2016-08-24 11:54:15 8471 [Note] InnoDB: Highest supported file format is Barracuda.
2016-08-24 11:54:15 8471 [Note] InnoDB: Log scan progressed past the checkpoint lsn 1591373062723
2016-08-24 11:54:15 8471 [Note] InnoDB: Database was not shutdown normally!
2016-08-24 11:54:15 8471 [Note] InnoDB: Starting crash recovery.
2016-08-24 11:54:15 8471 [Note] InnoDB: Reading tablespace information from the .ibd files...
2016-08-24 11:54:15 8471 [Note] InnoDB: Restoring possible half-written data pages
2016-08-24 11:54:15 8471 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 1591373107978
2016-08-24 11:54:15 8471 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 4383910, file name mysqlbin3306.000671
2016-08-24 11:54:16 8471 [Note] InnoDB: 128 rollback segment(s) are active.
2016-08-24 11:54:16 8471 [Note] InnoDB: Waiting for purge to start
2016-08-24 11:54:16 8471 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.26-rel74.0 started; log sequence number 1591373107978
2016-08-24 11:54:16 8471 [Note] Recovering after a crash using $datadir/mysqlbin/mysqlbin3306
2016-08-24 11:54:16 8471 [Note] Starting crash recovery...
2016-08-24 11:54:16 8471 [Note] Crash recovery finished.
2016-08-24 11:54:16 8471 [Note] RSA private key file not found: $datadir/private_key.pem. Some authentication plugins will not work.
2016-08-24 11:54:16 8471 [Note] RSA public key file not found: $datadir/public_key.pem. Some authentication plugins will not work.
2016-08-24 11:54:16 8471 [Note] Server hostname (bind-address): '*'; port: 3306
2016-08-24 11:54:16 8471 [Note] IPv6 is available.
2016-08-24 11:54:16 8471 [Note] - '::' resolves to '::';
2016-08-24 11:54:16 8471 [Note] Server socket created on IP: '::'.
2016-08-24 11:54:16 8471 [Warning] 'user' entry 'root@hostname' ignored in --skip-name-resolve mode.
2016-08-24 11:54:16 8471 [Warning] 'proxies_priv' entry '@ root@hostname' ignored in --skip-name-resolve mode.
2016-08-24 11:54:16 8471 [Note] Event Scheduler: Loaded 0 events
2016-08-24 11:54:16 8471 [Note] /usr/local/percona/bin/mysqld: ready for connections.
Version: '5.6.26-74.0-log' socket: '$socketdir/mysql.sock' port: 3306 Percona Server (GPL), Release 74.0, Revision 32f8dfd

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-3524

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.