Backup fails with "an unknown error"

[Springnuts]

Trying to a backup to a network drive. Yesterday Backup decided to do the preiodic full backup. But it fails, and I get the following error information with the "unknown error" message:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1555, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1541, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1393, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1522, in do_backup
    check_last_manifest(col_stats) # not needed for full backup
  File "/usr/bin/duplicity", line 1227, in check_last_manifest
    last_backup_set.check_manifests()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 201, in check_manifests
    remote_manifest = self.get_remote_manifest()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 242, in get_remote_manifest
    return manifest.Manifest().from_string(manifest_buffer)
  File "/usr/lib/python2.7/dist-packages/duplicity/manifest.py", line 188, in from_string
    vi = VolumeInfo().from_string(match.group(1))
  File "/usr/lib/python2.7/dist-packages/duplicity/manifest.py", line 388, in from_string
    raise VolumeInfoError("Start or end index not set")
 VolumeInfoError: Start or end index not set

lsb_release -rd >
Description: Ubuntu 18.04 LTS
Release: 18.04

apt-cache policy deja-dup >
deja-dup:
  Installed: 37.1-2fakesync1
  Candidate: 37.1-2fakesync1
  Version table:
 *** 37.1-2fakesync1 500
        500 http://la-mirrors.evowise.com/ubuntu bionic/main amd64 Packages
        100 /var/lib/dpkg/status

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: deja-dup 37.1-2fakesync1
ProcVersionSignature: Ubuntu 4.15.0-22.24-generic 4.15.17
Uname: Linux 4.15.0-22-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.2
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Sat Jun 9 21:14:08 2018
InstallationDate: Installed on 2016-02-26 (834 days ago)
InstallationMedia: Ubuntu 15.10 "Wily Werewolf" - Release amd64 (20151021)
SourcePackage: deja-dup
UpgradeStatus: Upgraded to bionic on 2018-05-03 (37 days ago)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[TC (datagrab)]

Ran into the same, or similar, issue. It turned out the problem was caused by insufficient permissions. Found a solution here https://unix.stackexchange.com/questions/68079/mount-cifs-network-drive-write-permissions-and-chown

Below is the error message I received. Would be helpful to the user if the error returned stated that the destination was not writable.

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1555, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1541, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1393, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1511, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 572, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 454, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 146, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 172, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 453, in <lambda>
    vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 342, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 395, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 79, in uexc
    return ufn(unicode(e).encode('utf-8'))
 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 37: ordinal not in range(128)

[xrs (xrs)]

I have exactly the same bug as described in the bug description.

I'm using Ubuntu 18.10.

[sambo63 (ridge1234-au)]

Similar problem for me too.

While deja-dup was doing a fresh backup, my system crashed (Ubuntu 18.04.2 LTS). Then on retry I get this trace message each time I try to run deja-dup.

# uname -a

Linux comp1-mysys 4.15.0-51-generic #55-Ubuntu SMP Wed May 15 14:27:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

# lsb_release -crid
Distributor ID: Ubuntu
Description: Ubuntu 18.04.2 LTS
Release: 18.04
Codename: bionic

# apt-cache policy duplicity deja-dup
duplicity:
  Installed: 0.7.17-0ubuntu1.1
  Candidate: 0.7.17-0ubuntu1.1
  Version table:
 *** 0.7.17-0ubuntu1.1 500
        500 http://au.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     0.7.17-0ubuntu1 500
        500 http://au.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
deja-dup:
  Installed: (none)
  Candidate: 37.1-2fakesync1ubuntu0.1
  Version table:
     37.1-2fakesync1ubuntu0.1 500
        500 http://au.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     37.1-2fakesync1 500
        500 http://au.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages

# duplicity --version
duplicity 0.7.17

# deja-dup --version
deja-dup 40.1

Error Message:

---------------------------

Failed with an unknown error.

Traceback (innermost last):
  File "/snap/deja-dup/172/bin/duplicity", line 1581, in <module>
    with_tempdir(main)
  File "/snap/deja-dup/172/bin/duplicity", line 1567, in with_tempdir
    fn()
  File "/snap/deja-dup/172/bin/duplicity", line 1419, in main
    do_backup(action)
  File "/snap/deja-dup/172/bin/duplicity", line 1537, in do_backup
    full_backup(col_stats)
  File "/snap/deja-dup/172/bin/duplicity", line 577, in full_backup
    globals.backend)
  File "/snap/deja-dup/172/bin/duplicity", line 459, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/snap/deja-dup/172/lib/python2.7/site-packages/duplicity/asyncscheduler.py", line 146, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/snap/deja-dup/172/lib/python2.7/site-packages/duplicity/asyncscheduler.py", line 172, in __run_synchronously
    ret = fn(*params)
  File "/snap/deja-dup/172/bin/duplicity", line 458, in <lambda>
    vol_num: put(tdp, dest_filename, vol_num),
  File "/snap/deja-dup/172/bin/duplicity", line 347, in put
    backend.put(tdp, dest_filename)
  File "/snap/deja-dup/172/lib/python2.7/site-packages/duplicity/backend.py", line 395, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/snap/deja-dup/172/lib/python2.7/site-packages/duplicity/util.py", line 79, in uexc
    return ufn(unicode(e).encode('utf-8'))
 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 37: ordinal not in range(128)

[Filofel (filofel)]

I see something that looks *exactly* identical to the problem sambo63 has (#6):

Same uname -a,
same everything except for:

deja-dup --version
deja-dup 37.0

I get:

Backup Failed
Failed with an unknown error.

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1555, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1541, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1393, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1511, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 572, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 454, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 146, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 172, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 453, in <lambda>
    vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 342, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 395, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 79, in uexc
    return ufn(unicode(e).encode('utf-8'))
 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 37: ordinal not in range(128)

[Michael Terry (mterry)]

Commenters that have errors which talk about Unicode or ascii or codecs: Is be interested to hear if trying the latest Deja Dup snap (which bundles duplicity 0.8 which claims to fix that) helps:

snap install deja-dup --classic --candidate

The original OP report was actually for a different issue about a VolumeInfoError. Which also seems to be a duplicity bug, not deja-dup

[Filofel (filofel)]

Michael (#7):
18.04.2 x64:
I uninstalled deja-dup and duplicity then did what you suggested (snap install etc.).
It mostly solved the problem I had (#6).
Thanks!

Still, the backup ended with the following error message:

Backup Finished

Could not back up the following files. Please make sure you are able to open them.

/home/philippe/.cache/deconf
/home/philippe/.dbus

Anything I could do to fix those?

[jervin (jervin)]

Are those files you need backed up?  Otherwise, you can put them in the
"folders to be ignored" list.

On 7/11/19 4:19 PM, Filofel wrote:
> Michael (#7):
> 18.04.2 x64:
> I uninstalled deja-dup and duplicity then did what you suggested (snap install etc.).
> It mostly solved the problem I had (#6).
> Thanks!
>
> Still, the backup ended with the following error message:
>
> Backup Finished
>
> Could not back up the following files. Please make sure you are able to
> open them.
>
> /home/philippe/.cache/deconf
> /home/philippe/.dbus
>
> Anything I could do to fix those?
>
--
John F. Ervin
<email address hidden>
407-679-6514

[Filofel (filofel)]

#9:

> Are those files you need backed up? Otherwise, you can put them in the
> "folders to be ignored" list.

Problem is, I'd rather not put them in the "ignored" list *if* they actually should be backed up and something else should be done instead (like changing permissions).

All I can tell I didn't have this error when running off previous Ubuntu versions.
But I couldn't tell whether deja-dup successfully saved them or automatically skipped them back then.

[Rajesh Chaudhary (rajeshisnepali)]

For the first time, I also encountered the same issue. I'm in Ubuntu 20.04

[Barry Martin (barry-martin-3-c)]

As much as I hate to be a "me too" message the same 'failed with unknown error' occurred with last night's automated backup, as well as the manual retests. Running Ubuntu 18.04. Nothing was changed here, although I did create a script on two Raspberry Pi's to monitor the status of accessing the remote NAS -- RPi with Motion, storage to the NAS; this Ubuntu system with storage to the same NAS, different directory. (No issues with the RPi writing.)

[Barry Martin (barry-martin-3-c)]

Update on what I found....

The easiest and fastest way to ‘correct’ is to create a new directory to back up to and change the storage location to the new back up point. I looked at permissions as suggested in one of the posts above:

barry@NZXT:~$ getfacl /media/NSA320_Backups/DejaDup_NZXT_02
getfacl: Removing leading '/' from absolute path names
# file: media/NSA320_Backups/DejaDup_NZXT_02
# owner: barry
# group: barry
user::rwx
group::rwx
other::rwx

barry@NZXT:~$ ls -lS /media/NSA320_Backups
total 0
drwxrwxrwx 2 barry barry 0 Apr 29 2020 Backend_3
drwxrwxrwx 2 barry barry 0 Oct 25 19:24 DejaDup_NZXT_01
drwxrwxrwx 2 barry barry 0 Dec 17 02:36 DejaDup_NZXT_02

barry@NZXT:~$ ls -lS /media/NSA320_Backups/DejaDup_NZXT_02
<snip>
-rwxrwxrwx 1 barry barry 187 Jul 30 03:33 duplicity-inc.20210729T082208Z.to.20210730T082213Z.manifest
-rwxrwxrwx 1 barry barry 187 Aug 10 03:33 duplicity-inc.20210809T082232Z.to.20210810T082229Z.manifest

To me looks like everyone and their uncle can do what they want (so chmod 777?). No idea. LIS, I created and pointed to a new directory and it’s making a full backup now.

[Barry Martin (barry-martin-3-c)]

Tested a theory this morning: try a restore from the old/failed backup. Was thinking maybe it had lost the login credentials to the NAS; nope: gave the same error message: "Restore Failed" and lines 1555, 1541, 1393, etc. (same as when attempted to create a backup). Again I have no idea what this means other that "it broke" but maybe is useful to the programmers .

As I said, creating a new backup location fixes (more patches) the problem.