Déjà Dup consistently fails to restore from Google Drive to Hard Drive - ApiRequestError

[giraffe]

Déjà Dup fails to restore from Google Drive to Hard Drive

I am trying to restore a backup of my Ubuntu machine's home directory from Google drive to a FAT formatted harddrive. After deja-dup runs the restore process for some time, it errors with:

Restore Failed
Giving up after 5 attempts. ApiRequestError: Cannot download file: {'x-guploader-uploadid': 'ADPycdsAgcrS17FH_WktdKcFutvCmDpVvrX01ATLIsIMU46mNKdxF7MjZha4tFWWjGgOol1Jyg4Qug0Unw6O-DCy7os', 'vary': 'Origin, X-Origin', 'content-type': 'application/json; charset=UTF-8', 'date': 'Sun, 18 Jul 2021 21:32:55 GMT', 'expires': 'Sun, 18 Jul 2021 21:32:55 GMT', 'cache-control': 'private, max-age=0', 'content-length': '290', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}

I would appreciate any help with solving this issue. Please let me know what additional information I should provide so I can help you help me.

--------------------------------------------------------
1.The distribution of Linux you're using:

Description: Ubuntu 20.04.2 LTS

--------------------------------------------------------
2.The version of deja-dup:

40.7

--------------------------------------------------------
3.The output of dconf dump /org/gnome/deja-dup/ :

[/]
backend='google'
delete-after=182
exclude-list=['/home/MY_USERNAME/.local/share/Trash']
include-list=['/home/MY_USERNAME']
last-backup='2021-07-18T07:25:12+00'
last-run='2021-07-18T07:25:12+00'
nag-check='2021-06-10T02:27:10+00'
periodic=true
periodic-period=1
prompt-check='2020-06-29T17:53:58+00'

[drive]
folder='MY_USERNAME-K501UX'

[file]
migrated=true

[gcs]
folder='MY_USERNAME-K501UX'

[goa]
type='google'

[google]
folder='extended_linux_backup'

[local]
folder='MY_USERNAME-K501UX'

[openstack]
container='MY_USERNAME-K501UX'

[rackspace]
container='MY_USERNAME-K501UX'

[remote]
folder='MY_USERNAME-K501UX'

[s3]
folder='MY_USERNAME-K501UX'

The attatched file has the output from step 4 (The file /tmp/deja-dup.log after running DEJA_DUP_DEBUG=1 deja-dup --restore | tail -n 1000 > /tmp/deja-dup.log)

[Michael Terry (mterry)]

Hello and thank you for your report! Sorry that Deja Dup is giving you trouble.

Looks like Google was giving you a 403 error code, which is a "not authorized" code. Maybe the auth token was timing out during the restore, and the underlying backup tool we use (duplicity) didn't refresh its token?

There have been some decent changes to duplicity's Google Drive backend since your version (0.8.11). Could you try maybe using the latest deja-dup snap, which bundles a newer duplicity in it?

snap install deja-dup --classic

And then try restoring?

[giraffe (chessbucket)]

I downloaded the classic version from snap and tried to use that (I have not purged the non-classic deja-dup version, fyi). The classic version GUI is extremely laggy and unresponsive when trying to restore - both trying to restore using my laptop and desktop. When I get to the "Restore to Where" screen, I get a warning (red exclamation mark) that "Backups does not have permissions to restore the following files." After around 10 minutes of the deja-dup application just freezing at this pop-up, I just kill it from System Monitor. I have been through this whole process several times but the end result is always a frozen deja-dup application on the "Restore to where" screen.

What would you suggest I try? What information should I provide?

[giraffe (chessbucket)]

I retried restoring with the classic version using the GUI. The Restore failed with the window saying: "Giving up after 5 attempts. ApiRequestError: None"

[Michael Terry (mterry)]

Yikes ok... The "classic" version is what I'd call the snap version. Passing --classic is a technical detail, which gives the snap access to the full file system.

I am concerned at how much worse it was for you... but that's not your immediate concern. It didn't help, so let's go back to square one.

First, let's remove the snap version that didn't help:
snap remove deja-dup

Then... you received a 403 error from Google. That means some sort of permission / authentication issue. Can you open the "Passwords and Keys" app and delete the "Google credentials for D�j� Dup" entry in there? That will delete any saved credentials for Google Drive that Deja Dup is keeping.

Then the next time you try to restore, it should ask you to re-grant Deja Dup access to Google Drive. Maybe that will help, to have fresh authentication tokens.

[giraffe (chessbucket)]

Thanks for sticking with me Michael.

I removed the "snap" version of deja-dup.
I removed the Google credentials for deja-dup in "Passwords and Keys".
When trying to restore, it indeed asked me to re-grant Deja Dup access to Google Drive. Same story though as with my first post. I get a "Restore Failed screen":

Giving up after 5 attempts. ApiRequestError: Cannot download file: {'x-guploader-uploadid': 'ADPycduaZY19ZloJoT-B7hE0hoGnRNcck0aMpCvD1nmBppbHrwVlGrmkbE4gBDBBk_X7lS3I5QYjlxYNS_geQB7jwuI', 'vary': 'Origin, X-Origin', 'content-type': 'application/json; charset=UTF-8', 'date': 'Thu, 29 Jul 2021 03:08:28 GMT', 'expires': 'Thu, 29 Jul 2021 03:08:28 GMT', 'cache-control': 'private, max-age=0', 'content-length': '290', 'server': 'UploadServer', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '403'}

[Michael Terry (mterry)]

Hi sorry for the silence. It's in part because I was busy and in part because I'm running out of ideas.

OK here's one - you say this happens after a while. So maybe... some access token is expiring mid restore.

Can you try restoring smaller portions of the backup? It might finish restoring before you hit this error.

This would be easier on version 42.0 or higher because it introduced a "browse & restore" interface. You can get that by downloading the snap version again.

But even on 40.7, you can do it on the console, with restrictions. You can run something like "deja-dup --restore path/to/file" and it will restore that file in-place (I don't believe 40.7 asks to restore it to a new place, it will always restore to original location with this command line method).

If you already know the different paths to restore (like... /home/xxx/Pictures or whatever), that might be good. Else the snap version lets you browse around and restore to new folder as well.

[Launchpad Janitor (janitor)]

[Expired for Déjà Dup because there has been no activity for 60 days.]