Encrypted backups using --stream cannot be decrypted

[Jeremy Tinley]

I've completed an ecrypted backup using the following:

innobackupex --no-lock --parallel=8 --stream=xbstream --user=$USER --password=$PASS
    --tmpdir=/var/tmp/mysql/3306 --host=127.0.0.1 \
    -encrypt=AES256 --encrypt-key-file=$VALIDPATH --encrypt-threads=4 \
    --port=3306 \
    /var/lib/dbbackups/ > backup.xbstream

The process completes successfully.

Immediately after:
cat backup.xbstream | xbstream -xv

Produces the following:
xb_stream_read_chunk(): wrong chunk magic at offset 0x0.

Removing --encrypt options generates a valid xbstream file.

[Jervin R (revin)]

Hi,

You have to pass it through xbcrypt as the stream is encrypted.

cat backup.xbstream | xbcrypt --encrypt=AES256 --encryp-key-file=$VALID_PATH --decrypt | xbstream -x -C /path/to/destination

[Jeremy Tinley (techwolf359)]

Thanks Jervin, that worked. This needs documentation update then:
https://www.percona.com/doc/percona-xtrabackup/2.2/innobackupex/encrypted_backups_innobackupex.html

There's a gap between following the encrypt options and the decrypt then.

It reads out as if you can deserialize the data with xbstream first, then pass decrypt options to decrypt each file individually.

I would suggest adding a section for encrypting and decrypting with --stream=xbstream.

[Nilnandan Joshi (nilnandan-joshi)]

Confirmed it as a feature request to add specific section for "encrypting and decrypting with --stream=xbstream."

[brad svee (brad-svee)]

this change is a shift away from the original behavior, and it affects the ease of use requiring a custom loop to decrypt the files:
Example compress, xbstream, and encrypt script:

innobackupex --defaults-file=/etc/mysql/backup.cnf --stream=xbstream --encrypt=AES256 --encrypt-key-file=/etc/mysql/.secret --compress ./ > backuparchive.xbstream

in version 2.2 the -encrypt flag on innobackupex would encrypt the final resulting archive that came out of xbstream. so to decrypt you would pipe the backuparchive file through xbcrypt first, and then pipe that through to xbstream.

In version 2.3 the encrypt happens on each individual file (after the optional compression step), so you have to pipe the backuparchive file through xbstream first, then loop through the resulting directories/files to find each *.xbcrypt file and individually pipe it through xbcrypt -d into the file of the same name minus the .xbcrypt and then delete the original *.xbcrypt file

I imagine it's much more useful to have individually encrypted files, and faster to encrypt/decrypt them, but what this feature request is asking for maybe is a utility to do clunky part at the end: find+decrypt+delete for each file.

[Sergei Glushchenko (sergei.glushchenko)]

We need to add --decrypt option to xbstream and support it in SST script

[Sergei Glushchenko (sergei.glushchenko)]

This can be done after we implement https://blueprints.launchpad.net/percona-xtrabackup/+spec/checksum-unencrypted-chunk

[Sergei Glushchenko (sergei.glushchenko)]

There is an xbstream --decrypt option now. Closing this bug. https://blueprints.launchpad.net/percona-xtrabackup/+spec/xbstream-decrypt

https://www.percona.com/doc/percona-xtrabackup/2.4/xbstream/xbstream.html

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-1843

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-435