intermittent assertion failure in compact backup prepare
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Percona XtraBackup moved to https://jira.percona.com/projects/PXB |
Triaged
|
Medium
|
Alexey Kopytov | ||
2.1 |
Triaged
|
Medium
|
Alexey Kopytov | ||
2.2 |
Triaged
|
Medium
|
Alexey Kopytov | ||
2.3 |
Triaged
|
Medium
|
Alexey Kopytov |
Bug Description
While performing SST MTR testing with production builds, during a PXC SST transfer using XtraBackup, the XtraBackup prepare stage will intermittently fail with a SIGABRT while rebuilding an index causing the SST transfer to fail.
==== Platform: Centos 5 x66_64
==== Binaries (from Jenkins production builds):
Percona-
percona-
(PXB compiled with -DCMAKE_
==== MTR test: t/xb_galera_
==== Configuration
# parallel+
[xtrabackup]
parallel=4
compact
[sst]
time=1
streamfmt=
progress=
==== Pertinent MTR test output (xb_galera_
...
2014-09-21 22:06:54: run.sh: Made 29 attempts to connect to server
2014-09-21 22:06:55: run.sh: Made 30 attempts to connect to server
2014-09-21 22:06:56: run.sh: Server process PID=2749 died.
2014-09-21 22:06:56: run.sh: Can't start the server. Server log (if exists):
2014-09-21 22:06:56: run.sh: ----------------
2014-09-21 22:06:56: run.sh: Error log for server with id: 1
...
WSREP_SST: [INFO] Evaluating innobackupex --defaults-
0-25.7.
ennett/
WSREP_SST: [ERROR] Cleanup after exit with status:1 (20140921 22:06:46.433)
WSREP_SST: [INFO] Removing the sst_in_progress file (20140921 22:06:46.435)
2014-09-21 22:06:46 2749 [ERROR] WSREP: Process completed with error: wsrep_sst_
2014-09-21 22:06:46 2749 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2014-09-21 22:06:46 2749 [ERROR] WSREP: SST failed: 1 (Operation not permitted)
2014-09-21 22:06:46 2749 [ERROR] Aborting
...
===== pertinent xtrabackup stderr log output
...
[01] Checking if there are indexes to rebuild in table sbtest/sbtest1 (space id: 6)
[01] Found index k_1
[01] Rebuilding 1 index(es).
2014-09-21 22:06:46 7ff4f37fe940 InnoDB: Assertion failure in thread 140690033994048 in file ut0byte.ic line 109
InnoDB: Failing assertion: ptr
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://
InnoDB: about forcing recovery.
02:06:46 UTC - xtrabackup got signal 6 ;
This could be because you hit a bug or data is corrupted.
This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x10000
xtrabackup(
xtrabackup(
/lib64/
/lib64/
/lib64/
xtrabackup [0x7708ed]
xtrabackup [0x77091f]
xtrabackup [0x771add]
xtrabackup [0x77c7d0]
xtrabackup(
xtrabackup [0x5f2a46]
xtrabackup [0x5f2d86]
/lib64/
/lib64/
Changed in percona-xtrabackup: | |
status: | Incomplete → Confirmed |
assignee: | nobody → Alexey Kopytov (akopytov) |
summary: |
- intermittent SST failure in PXC causing XtraBackup assertion failure + intermittent assertion failure in compact backup prepare |
MTR test output