AssertionError when one volume has been non-destructively decompressed

[Tim Waugh]

When one volume of a backup repository has been decompressed (perhaps by an over-eager file manager application when a curious user has double-clicked on a compressed volume), leaving the original compressed file in place, duplicity raises an AssertionError when asked to operate on that collection.

Example:

$ cd /tmp
$ mkdir source backup
$ touch source/foo
$ duplicity full --no-encryption /tmp/source file:///tmp/backup/
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
--------------[ Backup Statistics ]--------------
StartTime 1501707218.27 (Wed Aug 2 21:53:38 2017)
EndTime 1501707218.27 (Wed Aug 2 21:53:38 2017)
ElapsedTime 0.00 (0.00 seconds)
SourceFiles 2
SourceFileSize 60 (60 bytes)
NewFiles 2
NewFileSize 60 (60 bytes)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 2
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 127 (127 bytes)
Errors 0
-------------------------------------------------

$ FILE=$(echo backup/duplicity-full.*.vol1.difftar.gz)
$ gunzip -c "$FILE" > "${FILE%.gz}"
$ ls backup | cat
duplicity-full.20170802T205426Z.manifest
duplicity-full.20170802T205426Z.vol1.difftar
duplicity-full.20170802T205426Z.vol1.difftar.gz
duplicity-full-signatures.20170802T205426Z.sigtar.gz
$ duplicity collection-status file:///tmp/backup
Synchronising remote metadata to local cache...
Copying duplicity-full-signatures.20170802T205426Z.sigtar.gz to local cache.
Copying duplicity-full.20170802T205426Z.manifest to local cache.
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1540, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1534, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1385, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1410, in do_backup
    globals.archive_dir).set_values()
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 710, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 836, in get_backup_chains
    add_to_sets(f)
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 824, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 105, in add_filename
    (self.volume_name_dict, filename)
AssertionError: ({1: 'duplicity-full.20170802T205426Z.vol1.difftar'}, 'duplicity-full.20170802T205426Z.vol1.difftar.gz')

$ rpm -q duplicity
duplicity-0.7.13.1-1.fc26.x86_64
$ sed 1q /usr/bin/duplicity
#!/usr/bin/env python2
$ type python2
python2 is /usr/bin/python2
$ rpm -qf /usr/bin/python2
python2-2.7.13-11.fc26.x86_64

Full -v9 output:

Using archive dir: /home/twaugh/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75
Using backup name: ba8d32ccb88d13597b4784252744fc75
GPG binary is gpg, version 1.4.22
Import of duplicity.backends.acdclibackend Succeeded
Import of duplicity.backends.azurebackend Succeeded
Import of duplicity.backends.b2backend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.dpbxbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.giobackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.hubicbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.lftpbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.mediafirebackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.multibackend Succeeded
Import of duplicity.backends.ncftpbackend Succeeded
Import of duplicity.backends.onedrivebackend Succeeded
Import of duplicity.backends.par2backend Succeeded
Import of duplicity.backends.pydrivebackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.ssh_paramiko_backend Succeeded
Import of duplicity.backends.ssh_pexpect_backend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.sxbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Main action: collection-status
Acquiring lockfile /home/twaugh/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75/lockfile
================================================================================
duplicity 0.7.13.1 (June 18, 2017)
Args: /usr/bin/duplicity collection-status -v9 file:///tmp/backup
Linux river 4.11.11-300.fc26.x86_64 #1 SMP Mon Jul 17 16:32:11 UTC 2017 x86_64 x86_64
/usr/bin/python2 2.7.13 (default, Jun 26 2017, 10:20:05)
[GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
================================================================================
Local and Remote metadata are synchronized, no sync needed.
4 files exist on backend
3 files exist in cache
Extracting backup chains from list of files: [u'duplicity-full.20170802T205426Z.vol1.difftar', u'duplicity-full.20170802T205426Z.manifest', u'duplicity-full-signatures.20170802T205426Z.sigtar.gz', u'duplicity-full.20170802T205426Z.vol1.difftar.gz']
File duplicity-full.20170802T205426Z.vol1.difftar is not part of a known set; creating new set
Processing local manifest /home/twaugh/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75/duplicity-full.20170802T205426Z.manifest (181)
Found manifest volume 1
Found 1 volumes in manifest
File duplicity-full.20170802T205426Z.manifest is part of known set
File duplicity-full-signatures.20170802T205426Z.sigtar.gz is not part of a known set; creating new set
Ignoring file (rejected by backup set) 'duplicity-full-signatures.20170802T205426Z.sigtar.gz'
Releasing lockfile /home/twaugh/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75/lockfile
Using temporary directory /tmp/duplicity-57Q9j_-tempdir
Releasing lockfile /home/twaugh/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75/lockfile
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1540, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1534, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1385, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1410, in do_backup
    globals.archive_dir).set_values()
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 710, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 836, in get_backup_chains
    add_to_sets(f)
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 824, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib64/python2.7/site-packages/duplicity/collections.py", line 105, in add_filename
    (self.volume_name_dict, filename)
AssertionError: ({1: 'duplicity-full.20170802T205426Z.vol1.difftar'}, 'duplicity-full.20170802T205426Z.vol1.difftar.gz')

Releasing lockfile /home/twaugh/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75/lockfile