Decrypting backup with the wrong key make the backup unusable and unrecoverable

[Jaime Sicam]

Tested on Xtrabackup 2.1.9

Perform backup with encryption:
innobackupex --defaults-file=/home/user/sandboxes/rsandbox_mysql-5_5_39/master/my.sandbox.cnf --encrypt=AES256 --encrypt-key="A1EDC73815467C083B0869508406637E" --host=127.0.0.1 --port=24394 --user=root --password=msandbox /backups/

Decrypt backup with wrong key:
innobackupex --decrypt=AES256 --encrypt-key="A1EDC73815467C083B0869508406637L" /backups/2015-01-21_10-27-04/

Apply logs:
150121 10:30:39 innobackupex: Starting ibbackup with command: xtrabackup_55 --defaults-file="/backups/2015-01-21_10-27-04/backup-my.cnf" --defaults-group="mysqld" --prepare --target-dir=/backups/2015-01-21_10-27-04 --tmpdir=/tmp

xtrabackup_55 version 2.1.9 for Percona Server 5.5.35 Linux (x86_64) (revision id: 746)
xtrabackup: cd to /backups/2015-01-21_10-27-04
xtrabackup: error: xtrabackup_read_metadata()
xtrabackup: This target seems not to have correct metadata...
xtrabackup: notice: xtrabackup_logfile was already used to '--prepare'.
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 268435456
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 268435456
xtrabackup: Starting InnoDB instance for recovery.
xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
150121 10:30:39 InnoDB: The InnoDB memory heap is disabled
150121 10:30:39 InnoDB: Mutexes and rw_locks use GCC atomic builtins
150121 10:30:39 InnoDB: Compressed tables use zlib 1.2.3
150121 10:30:39 InnoDB: Initializing buffer pool, size = 100.0M
150121 10:30:39 InnoDB: Completed initialization of buffer pool
InnoDB: Error: checksum mismatch in data file ./ibdata1
150121 10:30:39 InnoDB: Could not open or create data files.
150121 10:30:39 InnoDB: If you tried to add new data files, and it failed here,
150121 10:30:39 InnoDB: you should now edit innodb_data_file_path in my.cnf back
150121 10:30:39 InnoDB: to what it was, and remove the new ibdata files InnoDB created
150121 10:30:39 InnoDB: in this failed attempt. InnoDB only wrote those files full of
150121 10:30:39 InnoDB: zeros, but did not yet use them in any way. But be careful: do not
150121 10:30:39 InnoDB: remove old data files which contain your precious data!
xtrabackup: innodb_init(): Error occured.
innobackupex: Error:
innobackupex: ibbackup failed at /usr/bin/innobackupex line 2560.

Since *.xbcrypt get deleted after decryption, you cannot recover the backup with the correct key.

[George Ormond Lorch III (gl-az)]

Ouch, that's pretty bad. I suppose there could be two parts to fixing this:
1-provide some kind of option to allow the user to keep the original .xbcrypt files on decrypt but that will also mean making --copyback aware so it doesn't copy the .xbcrypt files back on restore.
2-bump the xbcrypt chunk header version and add some kind of marker at the beginning of the encrypted chunk that can be validated on decryption, but this would effectively weaken the encryption and would not solve the issue of older/existing backups.

[Jericho Rivera (jericho-rivera)]

The same happens for Xtrabackup 2.2.8, using wrong key on decrypt. This deletes the *.xbcrypt file in the backup dir.

Apply log:
150121 12:25:10 innobackupex: Starting ibbackup with command: xtrabackup --defaults-file="/tmp/backups/2015-01-21_12-23-41/backup-my.cnf" --defaults-group="mysqld" --prepare --target-dir=/tmp/backups/2015-01-21_12-23-41

xtrabackup version 2.2.8 based on MySQL server 5.6.22 Linux (x86_64) (revision id: )
xtrabackup: cd to /tmp/backups/2015-01-21_12-23-41
xtrabackup: error: xtrabackup_read_metadata()
xtrabackup: This target seems not to have correct metadata...
xtrabackup: notice: xtrabackup_logfile was already used to '--prepare'.
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 50331648
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 50331648
xtrabackup: Starting InnoDB instance for recovery.
xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
InnoDB: Using atomics to ref count buffer pool pages
InnoDB: The InnoDB memory heap is disabled
InnoDB: Mutexes and rw_locks use GCC atomic builtins
InnoDB: Memory barrier is not used
InnoDB: Compressed tables use zlib 1.2.7
InnoDB: Using CPU crc32 instructions
InnoDB: Initializing buffer pool, size = 100.0M
InnoDB: Completed initialization of buffer pool
InnoDB: Space id in fsp header 811381095,but in the page header 3084749289
InnoDB: Restoring page 0 of tablespace 0
InnoDB: Doublewrite does not have page_no=0 of space: 0
InnoDB: checksum mismatch in data file ./ibdata1
InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data!
xtrabackup: innodb_init(): Error occured.
innobackupex: got a fatal error with the following stacktrace: at /usr/bin/innobackupex line 2633
 main::apply_log() called at /usr/bin/innobackupex line 1561
innobackupex: Error:
innobackupex: ibbackup failed at /usr/bin/innobackupex line 2633.

[Nilnandan Joshi (nilnandan-joshi)]

Set to confirmed as Jericho already verified it.

[Sergei Glushchenko (sergei.glushchenko)]

Merged pull requests https://github.com/percona/percona-xtrabackup/pull/12, https://github.com/percona/percona-xtrabackup/pull/13

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-419