Backup chain corrupted due to scp failure

Bug #1776733 reported by Bouke
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Duplicity
New
Undecided
Unassigned

Bug Description

Today I've been trying to salvage some lost data using our duplicity backups. However during the restoration I've run into the following error a few times:

    Invalid data - SHA1 hash mismatch for file:
    duplicity-inc.20180601T033002Z.to.20180601T073003Z.vol16.difftar.gpg

Digging through the backup volumes, that specific increment caught my attention:

    200M Jun 1 09:38 duplicity-inc.20180601T033002Z.to.20180601T073003Z.vol15.difftar.gpg
     21M Jun 1 09:38 duplicity-inc.20180601T033002Z.to.20180601T073003Z.vol16.difftar.gpg
    201M Jun 1 10:27 duplicity-inc.20180601T033002Z.to.20180601T073003Z.vol19.difftar.gpg

Notably:

 - volume 16 is incomplete
 - volume 17 and 18 are missing

In the logs when creating that backup, the following relevant messages have been outputted:

    Giving up after 5 attempts. BackendException: Error running 'scp -oServerAliveInterval=15 -oServerAliveCountMax=2 /tmp/duplicity-VA2x4D-tempdir/mktemp-fqSlsK-18 offsite-backup.cb.local:/backup/dbs02r2.cb.local/duplicity-inc.20180601T033002Z.to.20180601T073003Z.vol16.difftar.gpg'
    Giving up after 5 attempts. BackendException: Error running 'scp -oServerAliveInterval=15 -oServerAliveCountMax=2 /tmp/duplicity-VA2x4D-tempdir/mktemp-RbVN9m-19 offsite-backup.cb.local:/backup/dbs02r2.cb.local/duplicity-inc.20180601T033002Z.to.20180601T073003Z.vol17.difftar.gpg'
    Giving up after 5 attempts. BackendException: Error running 'scp -oServerAliveInterval=15 -oServerAliveCountMax=2 /tmp/duplicity-VA2x4D-tempdir/mktemp-SEdTr8-20 offsite-backup.cb.local:/backup/dbs02r2.cb.local/duplicity-inc.20180601T033002Z.to.20180601T073003Z.vol18.difftar.gpg'

For reference, the command used to produce this backup is:

    /usr/bin/duplicity --full-if-older-than 1M --encrypt-key XXXXXXXX --max-blocksize 1048576 --gpg-options -z 1 --asynchronous-upload -v 8 --include /my/data --exclude ** /my pexpect+scp://offsite-backup//backup

Duplicity version: 0.7.15
Python version: 2.7.5
OS: CentOS 7.3 EL7 (centos-release-7-3.1611.el7.centos.x86_64)
Filesystem: XFS

Looking through the backup logs, I see quite a few of those "giving up" messages. Probably those increments are corrupted as well. I'm starting to lose my confidence in the backups produced by duplicity. I expect that when an increment hasn't fully completed, duplicity would not continue to use that increment for the backup chain.

Am I doing something horribly wrong, or is this expected behaviour from duplicity?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.