Possible data loss when restarting in the middle of a deleted file
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
Fix Released
|
Critical
|
Unassigned | ||
duplicity (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Lucid |
Fix Released
|
Undecided
|
Unassigned | ||
Precise |
Fix Released
|
Undecided
|
Unassigned | ||
Quantal |
Won't Fix
|
Undecided
|
Unassigned | ||
Saucy |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
This was recently fixed in duplicity trunk. But I'm filing a bug for paperwork purposes and for Ubuntu SRUs.
[Impact]
When restarting a backup, duplicity may accidentally skip the first 65k chunk of one of the source files. This means that when it is restored, it will be incomplete/
Note that symptoms of this bug are diverse and may look like:
assert len( patch_seq ) == 1, len( patch_seq )
or
assert first.difftype != "diff", patch_seq
or
librsync error 103 while in patch cycle
These are all different results from this root cause. In all cases, it is because chunks of files are missing in your backup. There is nothing we can do to get those chunks back. But a related patch in duplicity 0.6.23 will let you restore the rest of your files (instead of just crashing halfway through a restore like 0.6.22 does). So try duplicity 0.6.23 to get a little progress from your broken backups. And use duplicity 0.6.23 for all future backups.
[Test Case]
Download and install the 'test1.sh' file attached to this bug report, and run it like 'sh test1.sh'. If it prints the following line, the bug is present:
Binary files /tmp/source/newfile and /tmp/restore/
Or, follow these manual steps:
mkdir /tmp/source
dd if=/dev/urandom of=/tmp/
# This next command will intentionally fail after the second volume!
duplicity full /tmp/source file:///tmp/backup --vol 1 --fail 2 --no-encryption
mv /tmp/source/bigfile /tmp/source/newfile
duplicity full /tmp/source file:///tmp/backup --no-encryption
duplicity restore file:///tmp/backup /tmp/restore --no-encryption
# This next line will say the files differ if the bug is present
diff /tmp/source/newfile /tmp/restore/
[Regression Potential]
It's a relatively small patch, only affecting the specific case of restarting a backup when the file we were in the middle of is no longer there. I'd say minor regression potential.
Related branches
- duplicity-team: Pending requested
-
Diff: 103 lines (+42/-2)3 files modifiedbin/duplicity (+5/-2)
duplicity/diffdir.py (+13/-0)
testing/tests/restarttest.py (+24/-0)
description: | updated |
description: | updated |
Changed in duplicity: | |
milestone: | none → 0.6.23 |
Changed in duplicity: | |
status: | Fix Committed → Fix Released |
description: | updated |
no longer affects: | duplicity (Ubuntu Raring) |
This bug was fixed in the package duplicity - 0.6.22-1ubuntu3
---------------
duplicity (0.6.22-1ubuntu3) trusty; urgency=low
* debian/ patches/ 04-dont- skip-first- chunk-on- restart. patch:
- When restarting a backup, if the file we were in the middle of
backing up is now deleted, don't skip the first 65k chunk of the
next file. Patch backported from upstream trunk. LP: #1252484
-- Michael Terry <email address hidden> Mon, 18 Nov 2013 16:51:52 -0500