Backup failed with duplicity assertion error

[Oded Arbel]

I have configured deja-dup to backup to an S3 folder and recently all backups are failing with this error:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1500, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1494, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1327, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 1047, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 221, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 207, in get_backend_object
    return factory(pu)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 161, in __init__
    self.resetConnection()
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 183, in resetConnection
    self.conn = get_connection(self.scheme, self.parsed_url, self.storage_uri)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/_boto_single.py", line 97, in get_connection
    assert scheme == 's3'
AssertionError

ProblemType: Bug
DistroRelease: Ubuntu 15.04
Package: deja-dup 32.0-0ubuntu2
ProcVersionSignature: Ubuntu 3.18.0-13.14-generic 3.18.5
Uname: Linux 3.18.0-13-generic x86_64
ApportVersion: 2.16.1-0ubuntu2
Architecture: amd64
CurrentDesktop: Unity
Date: Mon Feb 16 10:48:38 2015
InstallationDate: Installed on 2015-01-09 (37 days ago)
InstallationMedia: Ubuntu 15.04 "Vivid Vervet" - Alpha amd64 (20150109)
SourcePackage: deja-dup
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Tim Fisken (tim2)]

I experienced this with deja-dup 32.0-0ubuntu5 and duplicity 0.7.01-1ubuntu1. I think it's caused by this bug in duplicity:

https://bugs.launchpad.net/duplicity/+bug/1411803

That bug has been fixed in duplicity 0.7.02. I installed duplicity 0.7.02 from source, and I am no longer experiencing the problem.

[soundgeek (hmyjr)]

Up until about a month ago, I was backing up successfully to my USB drive. Now Backups fails with an error box.

With Ubuntu 14.04, fully updated to today, I get a similar error as son as I click 'Back Up Now':

Backup Failed
Failed with unknown error
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1366, in do_backup
    sync_archive(decrypt)
  File "/usr/bin/duplicity", line 1100, in sync_archive
    remote_metafiles, ignored, rem_needpass = get_metafiles(remlist)
  File "/usr/bin/duplicity", line 992, in get_metafiles
    pr = file_naming.parse(fn)
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 389, in parse
    pr = check_full()
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 308, in check_full
    t = str2time((m1 or m2).group("time"), short)
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 281, in str2time
    t = dup_time.genstrtotime(timestr.upper())
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 278, in genstrtotime
    return override_curtime - intstringtoseconds(timestr)
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 190, in intstringtoseconds
    error()
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 181, in error
    raise TimeException(bad_interval_string % interval_string)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 15: ordinal not in range(128)

[soundgeek (hmyjr)]

I mounted my backup USB drive on Win7 and it told me the drive had errors. I reformatted & found the Ubuntu "Backups" application works again for recording a backup.

Perhaps "Backups" needs to check disk integrity before proceeding and report errors in a tidy way and offer recovery? I expect there are Linux utilities to do all this already.

[Sebastien Bacher (seb128)]

seems like the issue is fixed with the wily version of duplicity and should be closed?

[Cysioland (cysioland)]

It isn't fixed, it occurs also to me

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1380, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1401, in do_backup
    sync_archive(decrypt)
  File "/usr/bin/duplicity", line 1139, in sync_archive
    remote_metafiles, ignored, rem_needpass = get_metafiles(remlist)
  File "/usr/bin/duplicity", line 1029, in get_metafiles
    pr = file_naming.parse(fn)
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 398, in parse
    pr = check_full()
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 317, in check_full
    t = str2time((m1 or m2).group("time"), short)
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 290, in str2time
    t = dup_time.genstrtotime(timestr.upper())
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 295, in genstrtotime
    return override_curtime - intstringtoseconds(timestr)
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 203, in intstringtoseconds
    error()
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 194, in error
    raise TimeException(bad_interval_string % interval_string)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 15: ordinal not in range(128)

[Canale Grande (canalegrande)]

Same here:
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1380, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1508, in do_backup
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 653, in incremental_backup
    bytes_written = dummy_backup(tarblock_iter)
  File "/usr/bin/duplicity", line 232, in dummy_backup
    while tarblock_iter.next():
  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 523, in next
    result = self.process(self.input_iter.next())
  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 195, in get_delta_iter
    for new_path, sig_path in collated:
  File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 276, in collate2iters
    relem1 = riter1.next()
  File "/usr/lib/python2.7/dist-packages/duplicity/selection.py", line 184, in Iterate
    subpath, val = diryield_stack[-1].next()
  File "/usr/lib/python2.7/dist-packages/duplicity/selection.py", line 153, in diryield
    error_handler, Path.append, (path, filename))
  File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 38, in check_common_error
    return function(*args)
  File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 535, in append
    return self.__class__(self.base, self.index + (ext,))
  File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 515, in __init__
    self.setdata()
  File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 520, in setdata
    self.stat = os.lstat(self.name)
OSError: [Errno 116] Stale file handle: '/media/backup'

[Mx. Wolf (mx-wolf)]

Has there been any movement on this? I can confirm it's also happening on a fresh installation of Ubuntu-Gnome 16.10 LTS. :-(

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1380, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1496, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 567, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 425, in write_multivol
    at_end = gpg.GPGWriteFile(tarblock_iter, tdp.name, globals.gpg_profile, globals.volsize)
  File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 332, in GPGWriteFile
    file = GPGFile(True, path.Path(filename), profile)
  File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 110, in __init__
    self.logger_fp = tempfile.TemporaryFile(dir=tempdir.default().dir())
  File "/usr/lib/python2.7/tempfile.py", line 511, in TemporaryFile
    (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags)
  File "/usr/lib/python2.7/tempfile.py", line 244, in _mkstemp_inner
    fd = _os.open(file, flags, 0600)
OSError: [Errno 2] No such file or directory: '/home/<username>/.cache/deja-dup/tmp/duplicity-nLhRj1-tempdir/tmpFdltD5'

[Jelle De Loecker (skerit)]

It seems everyone has a slightly different tracestack.
The error I receive seems to come from the patchdir.py file.

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1380, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1457, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 722, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
    for ropath in rop_iter:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 523, in integrate_patch_iters
    for patch_seq in collated:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 389, in yield_tuples
    setrorps(overflow, elems)
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 378, in setrorps
    elems[i] = iter_list[i].next()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 107, in filter_path_iter
    for path in path_iter:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 121, in difftar2path_iter
    tarinfo_list = [tar_iter.next()]
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 339, in next
    self.set_tarfile()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 333, in set_tarfile
    self.current_fp = self.fileobj_iter.next()
  File "/usr/bin/duplicity", line 758, in get_fileobj_iter
    backup_set.volume_name_dict[vol_num],
KeyError: 1

[Vej (vej)]

This bug report traces very different bugs (which all seem to come from duplicity).

The original stacktrace should be fixed with 7.02.

The bug mentioned in comments #4 and #7 is fix released with bug #1377873.

For the other reports: Please check, if you still experience this issue. If so check for duplicates and indicate that they affect you as well. If you do not find a duplicate please open a new bug report.

[i2000s (i2000s)]

It still exists on Ubuntu 16.04.3:

raceback (most recent call last):
  File "/usr/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1380, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1457, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 722, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 560, in Write_ROPaths
    for ropath in rop_iter:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 523, in integrate_patch_iters
    for patch_seq in collated:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 389, in yield_tuples
    setrorps(overflow, elems)
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 378, in setrorps
    elems[i] = iter_list[i].next()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 107, in filter_path_iter
    for path in path_iter:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 121, in difftar2path_iter
    tarinfo_list = [tar_iter.next()]
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 339, in next
    self.set_tarfile()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 333, in set_tarfile
    self.current_fp = self.fileobj_iter.next()
  File "/usr/bin/duplicity", line 758, in get_fileobj_iter
    backup_set.volume_name_dict[vol_num],
KeyError: 4