remove-* commands don't remove signature-files
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
Fix Released
|
Medium
|
Unassigned | ||
Debian |
Fix Released
|
Unknown
|
Bug Description
Duplicity version: 0.6.06 (debian: 0.6.06-3)
Python version: 2.5.2 (debian: 2.5.2-15+lenny1
OS Distro and version: Debian Lenny + packages from Testing
When working with duplicity for some time, the archive-dir grows and grows. (One user on the mailinglist had a 19GiB archive-dir)
The problem seems to be, that duplicity does not cleanup signature-files
when removing old backups, so the signatures occupy more and more space.
To reproduce, I did some backups (locally), some with a forced full
backup and some incementals. Afterwards the backup looks like this:
$ PASSPHRASE="foo" duplicity collection-status file:///tmp/duptest
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Feb 10 16:51:46 2010
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /home/tim/
Found 2 secondary backup chains.
Secondary chain 1 of 2:
-------
Chain start time: Wed Feb 10 16:50:16 2010
Chain end time: Wed Feb 10 16:50:44 2010
Number of contained backup sets: 4
Total number of contained volumes: 4
Type of backup set: Time: Num volumes:
-------
Secondary chain 2 of 2:
-------
Chain start time: Wed Feb 10 16:51:01 2010
Chain end time: Wed Feb 10 16:51:39 2010
Number of contained backup sets: 4
Total number of contained volumes: 4
Type of backup set: Time: Num volumes:
-------
Found primary backup chain with matching signature chain:
-------
Chain start time: Wed Feb 10 16:51:46 2010
Chain end time: Wed Feb 10 16:52:04 2010
Number of contained backup sets: 2
Total number of contained volumes: 2
Type of backup set: Time: Num volumes:
-------
No orphaned or incomplete backup sets found.
$ ls /tmp/duptest/
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
Now I remove the old backups:
$ PASSPHRASE="foo" duplicity remove-
file:///tmp/duptest
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Wed Feb 10 16:51:46 2010
Deleting backup sets at times:
Wed Feb 10 16:50:16 2010
Wed Feb 10 16:50:36 2010
Wed Feb 10 16:50:41 2010
Wed Feb 10 16:50:44 2010
Wed Feb 10 16:51:01 2010
Wed Feb 10 16:51:36 2010
Wed Feb 10 16:51:38 2010
Wed Feb 10 16:51:39 2010
Warning, found the following local orphaned signature files:
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
Note the "locally orphaned files".
When I look at the status:
$ PASSPHRASE="foo" duplicity collection-status file:///tmp/duptest
Synchronizing remote metadata to local cache...
Copying duplicity-
Copying duplicity-
Last full backup date: Wed Feb 10 16:51:46 2010
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /home/tim/
Found 0 secondary backup chains.
Found primary backup chain with matching signature chain:
-------
Chain start time: Wed Feb 10 16:51:46 2010
Chain end time: Wed Feb 10 16:52:04 2010
Number of contained backup sets: 2
Total number of contained volumes: 2
Type of backup set: Time: Num volumes:
-------
No orphaned or incomplete backup sets found.
Note that duplicity did fetch the signatures of the 2 old (deleted!)
full-backups
When looking at the target, it becomes clear that "duplicity remove..."
did only remove the data (*vol*difftar-
$ ls /tmp/duptest/
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
These files are also present (in decrypted form) in the cache-dir. And
they stay there (or are refetched from the backup-target).
Only workaround: delete the unneeded signature-files on the target, then
duplicity also removes them from the cache (but be carefull to only
remove old and unneeded files):
$ cd /tmp/duptest/
:/tmp/duptest$ rm duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
:/tmp/duptest$ ls
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
duplicity-
:/tmp/duptest$ cd -
~$ PASSPHRASE="foo" duplicity collection-status file:///tmp/duptest
Synchronizing remote metadata to local cache...
Deleting local
/home/tim/
(not authoritative at backend).
Deleting local
/home/tim/
(not authoritative at backend).
Deleting local
/home/tim/
(not authoritative at backend).
Deleting local
/home/tim/
(not authoritative at backend).
Deleting local
/home/tim/
(not authoritative at backend).
Deleting local
/home/tim/
(not authoritative at backend).
Deleting local
/home/tim/
(not authoritative at backend).
Deleting local
/home/tim/
(not authoritative at backend).
Last full backup date: Wed Feb 10 16:51:46 2010
Collection Status
-----------------
Connecting with backend: LocalBackend
Archive dir: /home/tim/
Found 0 secondary backup chains.
Found primary backup chain with matching signature chain:
-------
Chain start time: Wed Feb 10 16:51:46 2010
Chain end time: Wed Feb 10 16:52:04 2010
Number of contained backup sets: 2
Total number of contained volumes: 2
Type of backup set: Time: Num volumes:
-------
No orphaned or incomplete backup sets found.
By doing this, I could shrink the archive-dir (from my real backup) from over 3 GiB back to 1 GiB
Changed in debian: | |
status: | Unknown → Fix Released |
Changed in duplicity: | |
status: | New → Fix Released |
Changed in duplicity: | |
status: | Fix Released → Confirmed |
importance: | Undecided → Medium |
Changed in duplicity: | |
milestone: | none → 0.6.20 |
status: | Confirmed → Fix Committed |
Changed in duplicity: | |
status: | Fix Committed → Fix Released |
this is probably related to / the cause for /bugs.launchpad .net/duplicity/ +bug/497243
https:/