duplicity won't restore backup due to unknown error

[mariana xavier]

Hello,

Last week I did a fresh install on my Ubuntu but now I can't restore the backup I had made. I get the following error message:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1560, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1546, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1398, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1477, in do_backup
    restore(col_stats)
  File "/usr/bin/duplicity", line 733, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 558, in Write_ROPaths
    for ropath in rop_iter:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 521, in integrate_patch_iters
    for patch_seq in collated:
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 389, in yield_tuples
    setrorps(overflow, elems)
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 378, in setrorps
    elems[i] = iter_list[i].next()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 121, in difftar2path_iter
    tarinfo_list = [tar_iter.next()]
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 339, in next
    self.set_tarfile()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 333, in set_tarfile
    self.current_fp = self.fileobj_iter.next()
  File "/usr/bin/duplicity", line 769, in get_fileobj_iter
    backup_set.volume_name_dict[vol_num],
 KeyError: 1

 lsb_release -d
Description: Ubuntu 18.04.1 LTS

 dpkg-query -W deja-dup duplicity
deja-dup 37.1-2fakesync1
duplicity 0.7.18.2+bzr1367-0ubuntu1~ubuntu18.04.1

[Michael Terry (mterry)]

Switching to a duplicity bug - I think this is on their side.

[Kenneth Loafman (kenneth-loafman)]

It appears that one of the backup volumes is missing, namely #1. Could you list the contents of the backup location and see if that is the case?

[mariana xavier (marianaxrp)]

Hi, thanks for the reply. Here's the content of the backup folder:

/home/mariana/bckp/duplicity-full.20181126T133412Z.vol4.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol37.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol40.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol50.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol51.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol56.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol75.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol76.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol96.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol99.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol104.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol105.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol131.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol133.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol138.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol141.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol142.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol149.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol151.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol160.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol165.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol173.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol189.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol193.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol202.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol237.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol265.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol266.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol283.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol284.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol286.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol291.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol295.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol307.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol310.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol314.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol315.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol317.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol325.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol326.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol341.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol354.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol355.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol358.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol359.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol362.difftar
/home/mariana/bckp/duplicity-full.20181126T133412Z.vol365.difftar
/home/mariana/bckp/duplicity-full.2018112...

[Kenneth Loafman (kenneth-loafman)]

It's obvious a lot of difftar files are missing, it starts with vol4 then skips to vol37. There are a lot of other gaps like that, all fatal.

What is your commandline for the backup?

[mariana xavier (marianaxrp)]

I'm sorry, I'm not much of an expert in this. I did the backup using the backup tool in Ubuntu. I saved the files on Google Drive and it seems that it doesn't have gaps there, maybe it was an error when downloading the files to my computer. The problem is I can't restore the files through Google Drive either even though it seems to have no gaps. This error message appears:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1560, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1546, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1398, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1424, in do_backup
    action).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 710, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 836, in get_backup_chains
    add_to_sets(f)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 824, in add_to_sets
    if set.add_filename(filename):
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 105, in add_filename
    (self.volume_name_dict, filename)
 AssertionError: ({1: 'duplicity-full.20181126T133412Z.vol1.difftar.gpg', 2: 'duplicity-full.20181126T133412Z.vol2.difftar.gpg', 3: 'duplicity-full.20181126T133412Z.vol3.difftar.gpg', 4: 'duplicity-full.20181126T133412Z.vol4.difftar.gpg', 5: 'duplicity-full.20181126T133412Z.vol5.difftar.gpg', 6: 'duplicity-full.20181126T133412Z.vol6.difftar.gpg', 7: 'duplicity-full.20181126T133412Z.vol7.difftar.gpg', 8: 'duplicity-full.20181126T133412Z.vol8.difftar.gpg', 9: 'duplicity-full.20181126T133412Z.vol9.difftar.gpg', 10: 'duplicity-full.20181126T133412Z.vol10.difftar.gpg', 12: 'duplicity-full.20181126T133412Z.vol12.difftar.gpg', 14: 'duplicity-full.20181126T133412Z.vol14.difftar.gpg', 16: 'duplicity-full.20181126T133412Z.vol16.difftar.gpg', 17: 'duplicity-full.20181126T133412Z.vol17.difftar.gpg', 18: 'duplicity-full.20181126T133412Z.vol18.difftar.gpg', 19: 'duplicity-full.20181126T133412Z.vol19.difftar.gpg', 20: 'duplicity-full.20181126T133412Z.vol20.difftar.gpg', 21: 'duplicity-full.20181126T133412Z.vol21.difftar.gpg', 22: 'duplicity-full.20181126T133412Z.vol22.difftar.gpg', 23: 'duplicity-full.20181126T133412Z.vol23.difftar.gpg', 24: 'duplicity-full.20181126T133412Z.vol24.difftar.gpg', 25: 'duplicity-full.20181126T133412Z.vol25.difftar.gpg', 26: 'duplicity-full.20181126T133412Z.vol26.difftar.gpg', 27: 'duplicity-full.20181126T133412Z.vol27.difftar.gpg', 29: 'duplicity-full.20181126T133412Z.vol29.difftar.gpg', 30: 'duplicity-full.20181126T133412Z.vol30.difftar.gpg', 31: 'duplicity-full.20181126T133412Z.vol31.difftar.gpg', 32: 'duplicity-full.20181126T133412Z.vol32.difftar.gpg', 33: 'duplicity-full.20181126T133412Z.vol33.difftar.gpg', 34: 'duplicity-full.20181126T133412Z.vol34.difftar.gpg', 35: 'duplicity-full.20181126T133412Z.vol35.difftar.gpg', 36: 'duplicity-full.20181126T133412Z.vol36.difftar.gpg', 37: 'duplicity-full.20181126T133412Z.vol37.difftar.gpg', 39: 'duplicity-full.20181126T133412Z.vol39.difftar.gpg', 41: 'duplicity-full.20181126T133412Z....

[Kenneth Loafman (kenneth-loafman)]

Please check Google Drive to see if there are two entries for vol46.difftar.gpg. This has happened before.

[mariana xavier (marianaxrp)]

Yes, there are two vol46 files. Should I delete one of them?

[Kenneth Loafman (kenneth-loafman)]

No, move one out of the way and try again.

On Wed, Jan 23, 2019 at 7:55 AM mariana xavier <email address hidden>
wrote:

> Yes, there are two vol46 files. Should I delete one of them?
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1806466
>
> Title:
> duplicity won't restore backup due to unknown error
>
> Status in Duplicity:
> New
>
> Bug description:
> Hello,
>
> Last week I did a fresh install on my Ubuntu but now I can't restore
> the backup I had made. I get the following error message:
>
>
> Traceback (innermost last):
> File "/usr/bin/duplicity", line 1560, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1546, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1398, in main
> do_backup(action)
> File "/usr/bin/duplicity", line 1477, in do_backup
> restore(col_stats)
> File "/usr/bin/duplicity", line 733, in restore
> restore_get_patched_rop_iter(col_stats)):
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 558, in Write_ROPaths
> for ropath in rop_iter:
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 521, in integrate_patch_iters
> for patch_seq in collated:
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 389, in yield_tuples
> setrorps(overflow, elems)
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 378, in setrorps
> elems[i] = iter_list[i].next()
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 121, in difftar2path_iter
> tarinfo_list = [tar_iter.next()]
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 339, in next
> self.set_tarfile()
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 333, in set_tarfile
> self.current_fp = self.fileobj_iter.next()
> File "/usr/bin/duplicity", line 769, in get_fileobj_iter
> backup_set.volume_name_dict[vol_num],
> KeyError: 1
>
>
> lsb_release -d
> Description: Ubuntu 18.04.1 LTS
>
> dpkg-query -W deja-dup duplicity
> deja-dup 37.1-2fakesync1
> duplicity 0.7.18.2+bzr1367-0ubuntu1~ubuntu18.04.1
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/duplicity/+bug/1806466/+subscriptions
>

[mariana xavier (marianaxrp)]

I did that and I thought it was going to work, it was restoring files normally but then this error message appeared:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1560, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1546, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1398, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1419, in do_backup
    sync_archive()
  File "/usr/bin/duplicity", line 1162, in sync_archive
    remlist = globals.backend.list()
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 395, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 79, in uexc
    return ufn(unicode(e).encode('utf-8'))
 UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 25: ordinal not in range(128)

[Kenneth Loafman (kenneth-loafman)]

Try swapping the files.

On Wed, Jan 23, 2019 at 8:51 AM mariana xavier <email address hidden>
wrote:

> I did that and I thought it was going to work, it was restoring files
> normally but then this error message appeared:
>
> Traceback (innermost last):
> File "/usr/bin/duplicity", line 1560, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1546, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1398, in main
> do_backup(action)
> File "/usr/bin/duplicity", line 1419, in do_backup
> sync_archive()
> File "/usr/bin/duplicity", line 1162, in sync_archive
> remlist = globals.backend.list()
> File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 395,
> in inner_retry
> % (n, e.__class__.__name__, util.uexc(e)))
> File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 79, in
> uexc
> return ufn(unicode(e).encode('utf-8'))
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 25:
> ordinal not in range(128)
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1806466
>
> Title:
> duplicity won't restore backup due to unknown error
>
> Status in Duplicity:
> New
>
> Bug description:
> Hello,
>
> Last week I did a fresh install on my Ubuntu but now I can't restore
> the backup I had made. I get the following error message:
>
>
> Traceback (innermost last):
> File "/usr/bin/duplicity", line 1560, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1546, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1398, in main
> do_backup(action)
> File "/usr/bin/duplicity", line 1477, in do_backup
> restore(col_stats)
> File "/usr/bin/duplicity", line 733, in restore
> restore_get_patched_rop_iter(col_stats)):
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 558, in Write_ROPaths
> for ropath in rop_iter:
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 521, in integrate_patch_iters
> for patch_seq in collated:
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 389, in yield_tuples
> setrorps(overflow, elems)
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 378, in setrorps
> elems[i] = iter_list[i].next()
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 121, in difftar2path_iter
> tarinfo_list = [tar_iter.next()]
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 339, in next
> self.set_tarfile()
> File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line
> 333, in set_tarfile
> self.current_fp = self.fileobj_iter.next()
> File "/usr/bin/duplicity", line 769, in get_fileobj_iter
> backup_set.volume_name_dict[vol_num],
> KeyError: 1
>
>
> lsb_release -d
> Description: Ubuntu 18.04.1 LTS
>
> dpkg-query -W deja-dup duplicity
> deja-dup 37.1-2fakesync1
> duplicity 0.7.18.2+bzr1367-0ubuntu1~ubuntu18.04.1
>
> To manage notifications about this bug go to:
> htt...

[mariana xavier (marianaxrp)]

This message appears no matter what files are in the backup folder, both of them triggers this error:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1560, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1546, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1398, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1424, in do_backup
    action).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 684, in set_values
    backend_filename_list = self.backend.list()
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 395, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 79, in uexc
    return ufn(unicode(e).encode('utf-8'))
 UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 25: ordinal not in range(128)