Can not restore files. Error message: failed with unknown error.

[Rafayel PAremuzyan]

OS: Ubuntu 20.04
version: deja-dup 40.7
Output of the command dconf dump /org/gnome/deja-dup/
/]
backend='google'
last-backup='2020-10-12T03:10:25+00'
last-restore='2020-10-12T08:53:10+00'
last-run='2020-10-12T08:53:10+00'
nag-check='2020-10-12T03:10:25+00'
periodic=true
prompt-check='disabled'

[drive]
folder='rafopar-Precision-7510'
icon='. GThemedIcon drive-harddisk-usb drive-harddisk drive drive-harddisk-usb-symbolic drive-harddisk-symbolic drive-symbolic'
name='Rafo_Backup'
uuid='8908d407-982e-4725-902e-8baed07d67fe'

[file]
migrated=true

[gcs]
folder='rafopar-Precision-7510'

[google]
folder='rafopar-Precision-7510'

[local]
folder='rafopar-Precision-7510'

[openstack]
container='rafopar-Precision-7510'

[rackspace]
container='rafopar-Precision-7510'

[remote]
folder='rafopar-Precision-7510'

[s3]
folder='rafopar-Precision-7510'

The command "DEJA_DUP_DEBUG=1 deja-dup --restore | tail -n 1000 > /tmp/deja-dup.log" produces
the file /tmp/deja-dup.log, but after the error message it is still active (i.e. didn't exit), and the /tmp/deja-dup.log file is empty.

After installing a fresh Ubuntu 20.04 (previous was Ubuntu 18.04), I can not restore my files.

Steps:
* Restore
* "Restore From Where:, Here I select my backup location "Rafo_Bacup", and correct directory "rafopar-Precision-7510"
* "Restore From When", I chose the most recent, although, I have tried different other days as well
* Then status is "Preparing", after about 5 minutes, it prompts to a password, as soon I enter the password, it tells "Failed with unknown error". Doesn't show any log message.

In the deja-dup "Help" it tells that I can use command-line tool duplicity.
So following instructions I did
rafopar@rafopar-Precision-7510:/$ duplicity --gio file:///media/rafopar/Rafo_Backup/rafopar-Precision-7510/ /Work_Test/ > ~/duplicity_log

It does something, i.e. consumes lots of CPU, but no single file is written in the /Work_Test/ directory

Anyway bellow are error messages and outputs of the command

*Error messages*
Warning: Option --gio is pending deprecation and will be removed in a future release.
Use of default filenames is strongly suggested.
GnuPG passphrase for decryption:

Giving up after 1 attempts. Error: Error opening file “/tmp/duplicity-1za80bqb-tempdir/mktemp-gsmqt28z-2612”: No such file or directory

*And in the log file I see*
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sun Aug 2 19:41:47 2020
Cleanup of temporary directory /tmp/duplicity-1za80bqb-tempdir failed - this is probably a bug.

[Michael Terry (mterry)]

Hello! I'm sorry this is happening to ya. Let's figure this out together.

Deja Dup's error message "failed with an unknown error" usually means that duplicity failed underneath it but didn't give us a useful error code or message to show.

> Error: Error opening file “/tmp/duplicity-1za80bqb-tempdir/mktemp-gsmqt28z-2612”: No such file or directory

Hmm, that sounds like bug 1177381. Which doesn't have any current leads for what causes it...

OK. Let's talk about some options to try.

= Using newer versions =

You can install the snap for either deja-dup or duplicity:
snap install deja-dup --classic
snap install duplicity --classic

The benefit of using the latest Deja Dup is that it has a nicer restore interface that lets you browse the backup files and choose one to restore. (Assuming duplicity lets you get that far.)

The benefit of using the latest duplicity is just in case it has fixed any bugs around this. Might be worth a shot. (Deja Dup's snap bundles in an internal updated version of duplicity too, but doesn't expose it outside its own snap -- installing the duplicity snap will let you call it on the command line like normal.)

= Checking File Sizes =

Are the backup volume files all OK? Check size and types using commands like:

du -sh /media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-*
file /media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-*

We're just looking for any files that don't look like the other ones. Maybe bizarrely small, or are not reported as "GPG symmetrically encrypted data (AES cipher)" -- right, your backup is encrypted?

This would be looking for any volume files that somehow got mangled. Wouldn't tell us why, but might point at what we need to do to recover.

Does your storage location hold one big full backup, or is it like, a long history of backups with incrementals and all that jazz?

= Restoring By Hand =

There are instructions here for this: https://wiki.gnome.org/Apps/DejaDup/Help/Restore/WorstCase#Restoring_by_Hand

This is not a fun approach but can be necessary.

[Rafayel PAremuzyan (rafopar)]

Dear Michael,

Thank you for your suggestions.

*You can install the snap for either deja-dup or duplicity:snap install
deja-dup --classicsnap install duplicity --classic*
The deja-dup didn't install saying it is already installed, and the
duplicity was updated.
However this didn't solve the issue.

*du -sh /media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-*file
/media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-**
In the output of the command du -sh /media/rafopar/Rafo_Backup/
rafopar-Precision-7510/duplicity-* | sort -h > du_output
I see different types of files:
I) *.manifest, those are mostly 4.0K, an example line from the output is
this
4.0K
   /media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-inc.20200814T021733Z.to.20200814T072101Z.manifest
though there are few *.mainfest files with larger, 8.0K, 32K, 48K, 52K and
one file with 60M

II) duplicity-inc.*.difftat=r.gz
file sizes varry smalles is 336K, and going up to 50M, 51 M
50M
    /media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-full.20200802T234147Z.vol10077.difftar.gz

III) duplicity-full.*difftar.gz
one files 3.3M and the rest is 50M and 51M

IV) duplicity-new-signatures.*.sigtar.gz
Those are mostly more them 55M, though there are three with 16M, 20 and 25M

V) 4.9G
   /media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-full-signatures.20200802T234147Z.sigtar.gz
This is only file with duplicity-full-signatures*
In case it might be useful, I am attached the full output of *du* command
(du_output).

In the output of the file command

Mostly they look like
/media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-full.20200802T234147Z.vol10001.difftar.gz:
                    gzip compressed data, last modified: Mon Aug 3 07:34
:15 2020, max compression, original size modulo 2^32 54555648

There are also a lot of different types, e.g.
/media/rafopar/Rafo_Backup/rafopar-Precision-7510/duplicity-full.20200802T234147Z.vol10180.difftar.gz:
                    gzip compressed data, last modified: Mon Aug 3 07:45
:21 2020, max compression, original size modulo 2^32 154684416 gzip
compressed data, unknown method, extra field, from FAT filesystem (MS-DOS,
OS/2, NT), original size modulo 2
^32 154684416
I am attaching the file command output too "file_command_log"

I haven't looked the "Restore it by Hand" instructions, but will do it
next...

Than you,
Rafayel

On Mon, Oct 12, 2020 at 12:25 PM Michael Terry <email address hidden>
wrote:

> Hello! I'm sorry this is happening to ya. Let's figure this out
> together.
>
> Deja Dup's error message "failed with an unknown error" usually means
> that duplicity failed underneath it but didn't give us a useful error
> code or message to show.
>
> > Error: Error opening file “/tmp/duplicity-1za80bqb-tempdir/mktemp-
> gsmqt28z-2612”: No such file or directory
>
> Hmm, that sounds like bug 1177381. Which doesn't have any current leads
> for what causes it...
>
> OK. Let's talk about some options to try.
>
> = Using newer versions =
>
> You can install the snap for either deja-dup or duplicity:
> snap install deja-dup --classic
> snap install duplicity --classic
>
> The benefit of using the la...

[Michael Terry (mterry)]

Rafayel, I don't see anything obviously bad from the "file" or "du" command output. So that's good.

You mention the deja-dup snap is already installed? Does that work any better? If you run it like "snap run deja-dup" and try to restore, does it at least let you browse through your files?

[Rafayel PAremuzyan (rafopar)]

Dear Michael,

it turned out the new version actually worked!!

What was confusing is that, the command snap

snap install deja-dup --classic
snap install duplicity --classic

To my surprise it installed duplicity and the deja-dup (which was
installed earlier)
in the /snap/bin/deja and /snap/bin/duplicity

And actually I ended up having two versions of them, one in /usr/bin,
and another /snap/bin,
calling deja-dup, or duplicity was launching those located in /usr/bin.
So this was the reason that I told it didn't help.

Today I realized this, and through the synaptic I removed those
installed in /usr/bin.
Currently files are being restored, and soon they will be finished, I
see restored files!!

As you pointed, I also think the default version of duplicity in Ubuntu
20.04 (at least in my case) has a bug,
Now I am not 100% sure which version it was, but I believe it was 0.8.12.

Thank you a lot for your help.

With Regards,
Rafo

On 10/13/20 5:05 PM, Michael Terry wrote:
> Rafayel, I don't see anything obviously bad from the "file" or "du"
> command output. So that's good.
>
> You mention the deja-dup snap is already installed? Does that work any
> better? If you run it like "snap run deja-dup" and try to restore, does
> it at least let you browse through your files?
>

[Michael Terry (mterry)]

Ah! Good! Mostly... 🙂 I’ll close this ticket and will have to look through duplicity release notes to see if anything obvious needs to be backported.

Glad it worked out.

[Rafayel PAremuzyan (rafopar)]

Thank you a lot for your help.

Now I have all files successfully restored!

Rafo

On 10/13/20 7:18 PM, Michael Terry wrote:
> Ah! Good! Mostly... 🙂 I’ll close this ticket and will have to look
> through duplicity release notes to see if anything obvious needs to be
> backported.
>
> Glad it worked out.
>
> ** Changed in: deja-dup
> Status: New => Invalid
>