Authentication error with google drive

Bug #1747646 reported by Sam on 2018-02-06
184
This bug affects 35 people
Affects Status Importance Assigned to Milestone
Duplicity
Undecided
Unassigned
Déjà Dup
High
Unassigned
deja-dup (Ubuntu)
Medium
Unassigned

Bug Description

Hi,

I'm having an issue using deja-dup to back up my laptop to my google drive.

When backing only a small folder (eg: ~/Documents) every thing works fine.
However when trying to backup my whole home folder (/home/myusername) I get an "Invalid credentials" error after 5-10 min.

The back up seems to start normally and my destination folder starts filling with "duplicity-full...difftar.gz" files, but eventually I get the Invalid Credentials error.

lsb_release -d
Fedora release 27 (Twenty Seven)

rpm -q deja-dup duplicity
deja-dup-37.1-1.fc27.x86_64
duplicity-0.7.16-1.fc27.x86_64

deja-dup.log attached.
I cannot attach deja-dup.gsettings since i can only join 1 file. Will send if needed.

Sam (scaum) wrote :
Sam (scaum) wrote :

I tried today using duplicity (followed this guide https://6ftdan.com/danielpclark/2016/04/21/encrypted-linux-backup-with-google-drive-and-duplicity/) and everything went fine.

For some reason the issue seems linked to deja-dup

Michael Terry (mterry) wrote :

Ah that blog post is using a different duplicity backend than deja-dup is using. Can you try something for me?

Instead of running "duplicity .... gdocs://<email address hidden>/backup" can you do "duplicity .... gio+google-drive://<email address hidden>/backup"? You'll need to do that inside your desktop session (i.e. not from cron).

Let me know if that hits the same bug you're seeing with deja-dup.

Changed in deja-dup:
status: New → Incomplete
Sam (scaum) wrote :

Hi,

Thank you for your answer. Using your command gives me another error (not same as with deja-dup).

export GOOGLE_DRIVE_SETTINGS=~/.duplicity/credentials
duplicity --exclude-filelist ~/.duplicity/ignore ~/ gio+google-drive://myemail/test
Traceback (innermost last):
  File "/usr/bin/duplicity", line 1559, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1545, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1381, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib64/python2.7/site-packages/duplicity/commandline.py", line 1140, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/lib64/python2.7/site-packages/duplicity/commandline.py", line 1015, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/lib64/python2.7/site-packages/duplicity/backend.py", line 223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib64/python2.7/site-packages/duplicity/backend.py", line 209, in get_backend_object
    return factory(pu)
  File "/usr/lib64/python2.7/site-packages/duplicity/backends/giobackend.py", line 97, in __init__
    self.remote_file.make_directory_with_parents(None)
 Error: g-io-error-quark: Operation unsupported (15)

The "test" folder was already created in my drive account at "root" level (next to the backup folder I am using with the other command).

Vej (vej) on 2018-02-13
Changed in deja-dup:
status: Incomplete → New
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ubuntu:
status: New → Confirmed
Jacob Marciniec (foxyjacob) wrote :

Bumping this... I will gladly pitch in $10 to whoever fixes this. This bug affects me and I really would love to see it solved. I can help with any diagnostics.

https://www.bountysource.com/issues/58724362-authentication-error-with-google-drive

summary: - Authentication error with google drive
+ Authentication error with google drive [$10]
tags: added: bounty

I noticed that I encounter this error when I am connecting via a VPN service, and do not encounter this error when I am not. If the reason is that Google rejects authenticating from suspicious IPs with the kind of authentication deja-dup and GNOME's online accounts use, I don't know if the team can do anything about it.

If this is indeed the case, I think a workaround by connecting from a non-blacklisted IP may work.

Johannes Choo (jhanschoo) wrote :

I would like to retract my report: my backups appear to not have completed.

Usul_ (usul-) wrote :

This happens to me to. I am not using a personal google account but a corporate one.
After 5 or 10 minutes I get an authentication error.
I am on ubuntu 18.04 using deja-dup 37-1 and duplicity 0.7.17

Cody Hodges (cody35367) wrote :

Perhaps the OAuth token is expiring during long backup jobs and needs to be renewed? Not sure what it is based on because our times vary widely.

https://developers.google.com/drive/api/v3/handle-errors#401_invalid_credentials

This happens to me after first-time backup running for about 45 minutes. Then it fails with (same as OP's log):
Giving up after 5 attempts. Error: gdata-service-error-quark: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

Ubuntu 18.04.01 using:
deja-dup 37.1-2fakesync1
duplicity 0.7.17-0ubuntu1

Paul White (paulw2u) on 2018-11-15
affects: ubuntu → deja-dup (Ubuntu)
Vej (vej) on 2018-12-17
Changed in deja-dup:
status: New → Triaged
importance: Undecided → High
Changed in deja-dup (Ubuntu):
status: Confirmed → Triaged
importance: Undecided → Medium
tags: added: bionic
Changed in deja-dup (Ubuntu):
milestone: none → bionic-updates
Vej (vej) on 2018-12-17
summary: - Authentication error with google drive [$10]
+ Authentication error with google drive
Prabhat Soni (ubuntudev2018) wrote :

This bug seems to not have been fixed yet. Nobody seems to have posted its solution on the internet.
Description : I have a weekly backup scheduled. I do this on my google drive of my gmail account.

The backup happens for about 10 minutes. I'm unable to unmount my google drive from my computer this time and the computer tells me that some operations are keeping th volume busy.

Then I get this error message:-
This has been happening since 1 month continously and everytime I get this error message

Giving up after 5 attempts. Error: gdata-service-error-quark: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}
 (4)

Clyton (clyton) wrote :

Getting the same error on ubuntu-gnome 18.10 but I can see the backup files in google-drive. I don't know which part of the backup has failed
Giving up after 5 attempts. Error: gdata-service-error-quark: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

Andrew C. Dingman (adingman) wrote :

I'm seeing exactly the same thing today on Fedora 29. It works fine on a new laptop with very little to back up, but a few gigs in to backing up my desktop I get the same error reported by the last two commenters.

Andrew C. Dingman (adingman) wrote :

Actually, slightly different. Probably not meaningfully so, but here it is just in case:

Giving up after 5 attempts. Error: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

Andrew C. Dingman (adingman) wrote :

The same error happens with Duplicity on the command line, but it sucks less because resuming works.

$ duplicity --exclude=/home/<me>/{.cache,kolab,box.iu,Download/releases} --encrypt-key <mykey> /home/<me>/ gio+google-drive://<user>@gmail.com/<my_hostname>
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
Error accessing possibly locked file /home/<me>/Documents/<somedir>/<somefile>.pdf
Attempt 1 failed. Error: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

Attempt 2 failed. Error: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

Attempt 3 failed. Error: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

Attempt 4 failed. Error: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

Giving up after 5 attempts. Error: Authentication required: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "authError",
    "message": "Invalid Credentials",
    "locationType": "header",
    "location": "Authorization"
   }
  ],
  "code": 401,
  "message": "Invalid Credentials"
 }
}

$ duplicity --exclude=/home/<me>/{.cache,kolab,box.iu,Download/releases} --encrypt-key <mykey> /home/<me>/ gio+google-drive://<user>@gmail.com/<my_hostname>
Local and Remote metadata are synchronized, no sync needed.
Last full backup left a partial set, restarting.
Last full backup date: Mon Dec 31 15:56:49 2018
GnuPG passphrase:
RESTART: Volumes 220 to 221 failed to upload before termination.
         Restarting backup at volume 220.
Restarting after volume 219, file Music/<band>/<album>/<track>.flac, block 209
Error accessing possibly locked file /home/<me>/Documents/<somedir>/<somefile>.pdf

Andrew C. Dingman (adingman) wrote :

The bit about a possibly locked file can safely be ignored. For whatever reason, the PDF in question had mode 260.

Michael Terry (mterry) on 2019-01-05
tags: added: restore
tags: added: backup
removed: restore
Michael Terry (mterry) wrote :

I suspect this is an issue with gnome-online-accounts and how it is driving the authentication. I've filed an upstream bug: https://gitlab.gnome.org/GNOME/gnome-online-accounts/issues/36

Andrew C. Dingman (adingman) wrote :

Possibly, but I also notice that the gio backend doesn't implement _retry_cleanup(). Since re-running the same duplicity command can continue successfully without needing to re-start goa-daemon, I suspect there's something that could be done there to re-authenticate the connection.

Unfortunately, my Python skills are both limited and rusty. So far, all I've been able to do is confirm that simply re-invoking __init__() doesn't do the trick. I'll keep banging on it from that side as I have time.

Michael Terry (mterry) wrote :

I moved the above bug to GVFS: https://gitlab.gnome.org/GNOME/gvfs/issues/360

In that bug, I've attached a Python script that reproduces the problem. So this is not unique to duplicity or deja-dup.

However, we might reasonably work around it in duplicity by remounting or similar when such an error occurs. Maybe in _retry_cleanup as Andrew suggests.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Bug attachments