InnoDB: Assertion failure in thread

[chongguang lin]

171204 01:00:02 innobackupex: Starting the backup operation

IMPORTANT: Please check that the backup run completes successfully.
           At the end of a successful backup run innobackupex
           prints "completed OK!".

171204 01:00:03 version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup;
port=3306;mysql_socket=/data/mysql/data_back/mysql.sock' as 'root' (using password: NO).
171204 01:00:03 version_check Connected to MySQL server
171204 01:00:03 version_check Executing a version check against the server...
171204 01:00:03 version_check Done
Using server version 5.6.17-log
innobackupex version 2.4.4 based on MySQL server 5.7.13 Linux (x86_64) (revision id: df58cf2)

171204 01:41:22 >> log scanned up to (8141434107856)
171204 01:41:23 >> log scanned up to (8141434275278)
171204 01:41:24 >> log scanned up to (8141434364129)
171204 01:41:25 >> log scanned up to (8141434478794)
171204 01:41:26 >> log scanned up to (8141434503832)
171204 01:41:27 >> log scanned up to (8141434533847)
2017-12-04 01:41:28 0x7f6db9d9f700 InnoDB: Assertion failure in thread 140109246232320 in file mtr0log.cc line 604
InnoDB: Failing assertion: DATA_TRX_ID_LEN == dict_index_get_nth_col(ind, DATA_TRX_ID - 1 + n_uniq)->len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
17:41:28 UTC - xtrabackup got signal 6 ;
This could be because you hit a bug or data is corrupted.
This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x10000
innobackupex(my_print_stacktrace+0x35)[0x1037575]
innobackupex(handle_fatal_signal+0x273)[0xa268c3]
/lib64/libpthread.so.0[0x3d5ba0f710]
/lib64/libc.so.6(gsignal+0x35)[0x3d5b632925]
/lib64/libc.so.6(abort+0x175)[0x3d5b634105]
innobackupex(_Z22trx_undo_free_preparedP5trx_t+0x0)[0x75a761]
innobackupex[0x8f4387]
innobackupex[0x90963c]
innobackupex(_Z19recv_parse_log_recsm7store_tb+0x4df)[0x90bbdf]
innobackupex[0x765a48]
innobackupex[0x765bd6]
innobackupex[0x765f83]
/lib64/libpthread.so.0[0x3d5ba079d1]
/lib64/libc.so.6(clone+0x6d)[0x3d5b6e8b6d]

[Muhammad Irfan (muhammad-irfan)]

Can you post us exact backup command you used along with configuration options (my.cnf) in place to check further. Thanks.

[chongguang lin (lcg3141592654)]

This is the mysql server configuration file information:
----------------------------------------------------------------------------------------------
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
# *** DO NOT EDIT THIS FILE. It's a template which will be copied to the
# *** default location during install, and will be replaced if you
# *** upgrade to a newer version of MySQL.

[client]
socket = /data/mysql/data_back/mysql.sock
default-character-set = utf8

[mysqld]

# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.

 innodb_buffer_pool_size = 100G

# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
 log_bin = /data/mysql/data_back/mysql-bin.log
# These are commonly set, remove the # and set as required.
 basedir = /data/mysql
 datadir = /data/mysql/data_back
 port = 3306
 server_id = 100
 socket = /data/mysql/data_back/mysql.sock
 max_connections = 5000
 log-error=/var/log/mysqld_back.log
 slow_query_log=ON
 slow_launch_time=1
 slow_query_log_file=/data/mysql/slow-log-back
 pid-file=/data/mysql/data_back/mysqld.pid
 character-set-server=utf8
 collation-server=utf8_general_ci
 skip-character-set-client-handshake
 skip-host-cache
 skip-name-resolve
#binlog_format="STATEMENT"
binlog_format="ROW"
# binlog_format="MIXED"
lower_case_table_names=1
#read_only=1
expire_logs_days=7
innodb_read_io_threads=32
innodb_write_io_threads=32
relay-log=mysqld-relay-bin
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M

sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

[mysqld_safe]
log-error=/var/log/mysqld_back.log
default-character-set = utf8
----------------------------------------------------------------------------------------------
this is my backup mysql command:

innobackupex --defaults-file=/etc/my.cnf --user=root --stream=tar /backup/bak 2>/backup/bak/bak-`date +%Y%m%d`.log 1>/backup/bak/`date +%Y%m%d`-backup.tar

<email address hidden>

From: Muhammad Irfan
Date: 2017-12-05 16:36
To: doublelightone
Subject: [Bug 1736289] Re: InnoDB: Assertion failure in thread
Can you post us exact backup command you used along with configuration
options (my.cnf) in place to check further. Thanks.

** Changed in: percona-xtrabackup
       Status: New => Incomplete

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1736289

Title:
  InnoDB: Assertion failure in thread

Status in Percona XtraBackup:
  Incomplete

Bug description:
  171204 01:00:02 innobackupex: Starting the backup operation

  IMPORTANT: Please check that the backup run completes successfully.
             At the end of a successful backup run in...

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-1487