@Shahriyar I don't think you'll be able to reproduce easily without the actual database
Out of >800 recent restores for this particular version combination (different databases, with similar my.cnf), I have 3 confirmed "bad" backups that can't be restored with that problem.
The only thing I know is that:
1. I have seen it on mysql 5.5.36 only
2. It doesn't appear to occur on 5.6 (>1600 restores with no issue)
3. I have seen it with xtrabackup 2.2.10 and 2.2.12
4. It only occurs on differential backups
5. The 3 cases I've seen are on large databases (>2T database size)
6. In the most recent case, two successive differentials of the same DB (same base LSN) exhibit the same problem
7. A new full backup, then differential on the same DB can be restored just fine (confirming in the next couple days)
@Shahriyar I don't think you'll be able to reproduce easily without the actual database
Out of >800 recent restores for this particular version combination (different databases, with similar my.cnf), I have 3 confirmed "bad" backups that can't be restored with that problem.
The only thing I know is that:
1. I have seen it on mysql 5.5.36 only
2. It doesn't appear to occur on 5.6 (>1600 restores with no issue)
3. I have seen it with xtrabackup 2.2.10 and 2.2.12
4. It only occurs on differential backups
5. The 3 cases I've seen are on large databases (>2T database size)
6. In the most recent case, two successive differentials of the same DB (same base LSN) exhibit the same problem
7. A new full backup, then differential on the same DB can be restored just fine (confirming in the next couple days)
The command that fails is:
innobackupex --apply-log --incremental-dir /glide/ mysqld/ aztemptesting_ 3400_giga/ temp/restore_ 2016-02- 01_1336089/ azprod_ 3403_db160098_ 37-2_2016- mysqld/ aztemptesting_ 3400_giga/ temp/restore_ 2016-02- 01_1336089/ azprod_ 3403_db160098_ 37_2016- 01-29_1844126 2> /glide/ backup/ log/restore. 2 01_1336089. azprod_ 3403_db160098_ 37-2_2016- 01-31_1844124. log
01-31_1844124 --use-memory=32G /glide/
016-02-