Data corruption when resuming
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
Fix Released
|
Medium
|
Unassigned | ||
duplicity (Fedora) |
Fix Released
|
Undecided
|
|||
duplicity (Ubuntu) |
Fix Released
|
Critical
|
Michael Terry | ||
Lucid |
Fix Released
|
Undecided
|
Unassigned | ||
Oneiric |
Won't Fix
|
Undecided
|
Unassigned | ||
Precise |
Fix Released
|
Undecided
|
Unassigned | ||
Quantal |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
[Impact]
It's very easy to accidentally corrupt the backup copy of a file when resuming an interrupted backup.
Specifically, there are several ways to end up backing up corrupted data:
A) If resuming after a volume that ended in a one-block file, we would skip the first block of the next file.
B) If resuming after a volume that ended in a multi-block file, we would skip the first block of the next file.
C) If resuming a non-encrypted backup after a volume that spanned a multi-block file, we would skip some data inside the file.
These are all very similar but have slightly different code fixes. Together they amount to "if you resume a backup, it's very likely you will have corrupted data somewhere". The only situation that doesn't is resuming in the middle of a file while using encryption.
[Test Case]
Download the script 'auto-ctrl-
sudo /bin/bash ./auto-
It should end with "***** Diff between /lib and /tmp/restore" and no lines following if the bug is fixed. If the bug is still there, there will be several lines explaining the difference between the original files and the restored ones.
[Regression Potential]
This patch changes (1) how much data we read from source files when backing up and (2) how we pick where to resume a backup.
If a regression did appear because of this branch, I would expect it to be a similar data corruption problem (not starting in the right place or reading too much/too little from source files).
[Original Report]
I have found a bug similar to #613244.
It seems that some files are corrupted when the backup is interrupted and resumed when using --no-encryption option.
The make-ctrl-c-test.sh test fails if I add the --no-encryption option to duplicity. Without this option it works fine.
I have added an option to make-ctrl-c-test.sh test to run it with or without encryption (patch attached).
I have seen this bug in the current duplicity quantal version (0.6.19-0ubuntu2) and in the last bazaar revision (897).
I also attached a patch that seems to fix this issue. I don't know anything about duplicity internals, so I don't know if this patch is correct or it will break something. Actually, I have ran the run-test script and it fails in test_GzipWriteFile test. But hopefully this can help you to fix the problem.
These are the final log lines of the make-ctrl-c-test.sh script, with and without encryption.
make-ctrl-c-test.sh with encryption:
...
Restoring backups...
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Dec 17 14:07:59 2012
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Dec 17 14:08:13 2012
Diff between /lib and /tmp/restore1
Diff between /tmp/restore1 and /tmp/restore2
make-ctrl-c-test.sh with --no-encryption and without fix applied:
...
Restoring backups...
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Dec 17 14:08:55 2012
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Dec 17 14:09:16 2012
Diff between /lib and /tmp/restore1
Diff between /tmp/restore1 and /tmp/restore2
Files /tmp/restore1/
make-ctrl-c-test.sh with --no-encryption and fix applied:
...
Restoring backups...
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Dec 17 14:19:43 2012
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Dec 17 14:20:14 2012
Diff between /lib and /tmp/restore1
Diff between /tmp/restore1 and /tmp/restore2
The test_GzipWriteFile fails with this error:
=======
FAIL: test_GzipWriteFile (__main__.GPGTest)
Test GzipWriteFile
-------
Traceback (most recent call last):
File "tests/gpgtest.py", line 135, in test_GzipWriteFile
assert size - 64 * 1024 <= os.stat(
AssertionError
-------
Ran 8 tests in 1.326s
FAILED (failures=1)
Test failed
Related branches
- duplicity-team: Pending requested
-
Diff: 498 lines (+188/-47)6 files modifiedbin/duplicity (+13/-5)
duplicity/diffdir.py (+21/-19)
duplicity/dup_temp.py (+4/-2)
duplicity/gpg.py (+4/-6)
testing/tests/gpgtest.py (+10/-5)
testing/tests/restarttest.py (+136/-10)
Changed in duplicity: | |
status: | New → Fix Committed |
importance: | Undecided → Medium |
summary: |
- Data corruption when resuming with --no-encryption option + Data corruption when resuming |
description: | updated |
Changed in duplicity: | |
status: | Fix Committed → Triaged |
Changed in duplicity: | |
status: | Triaged → Fix Committed |
Changed in duplicity: | |
milestone: | none → 0.6.21 |
status: | Fix Committed → Fix Released |
Changed in duplicity (Ubuntu Oneiric): | |
status: | Fix Committed → Won't Fix |
Changed in duplicity (Fedora): | |
importance: | Unknown → Undecided |
status: | Unknown → Fix Released |
The attachment "no-encryption- corruption- fix.patch" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.
[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]