Using onedrive backend fails with TypeError

Bug #1543976 reported by ThomasL.
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Duplicity
New
Undecided
Unassigned

Bug Description

I'm using duply to call duplicity on a Debian Jessie system. It uploads a few hundred MB each day to onedrive and was working for months without any issues. The last successful backup was on Jan 26th and since this date (almost) all commands always fail with the same error message:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1580, in <module>
    if "Forced assertion for testing" in str(e):
TypeError: __str__ returned non-string (type Error)

Yes, it fails MOST of the time, not always. Today the following command executed successfully about 2-3 times in 50 tries. So I think it's quite unlikely it's a local problem or an issue in the duplicity onedrive backend. Also a colleague has exactly the same issue since about the same date.

/usr/bin/duplicity collection-status --name duply_owncloud --encrypt-key B44F3F99 --sign-key B44F3F99 --verbosity 9 --gpg-options --compress-algo=bzip2 --ssl-no-check-certificate onedrive://backups/ownCloud

I modified line 1580 to print out a full backtrace and this is the complete output:

# /usr/bin/duplicity collection-status --name duply_owncloud --encrypt-key B44F3F99 --sign-key B44F3F99 --verbosity 9 --gpg-options --compress-algo=bzip2 --ssl-no-check-certificate onedrive://backups/ownCloud
Using archive dir: /root/.cache/duplicity/duply_owncloud
Using backup name: duply_owncloud
Import of duplicity.backends.azurebackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.copycombackend Succeeded
Import of duplicity.backends.dpbxbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.giobackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.hubicbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.lftpbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.multibackend Succeeded
Import of duplicity.backends.ncftpbackend Succeeded
Import of duplicity.backends.onedrivebackend Succeeded
Import of duplicity.backends.par2backend Succeeded
Import of duplicity.backends.pydrivebackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.ssh_paramiko_backend Succeeded
Import of duplicity.backends.ssh_pexpect_backend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.sxbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Using temporary directory /tmp/duplicity-4yt4Ft-tempdir
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1530, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1524, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1362, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 1093, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 209, in get_backend_object
    return factory(pu)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/onedrivebackend.py", line 90, in __init__
    self.initialize_oauth2_session()
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/onedrivebackend.py", line 129, in initialize_oauth2_session
    user_info_response = self.http_client.get(self.API_URI + 'me')
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 469, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests_oauthlib/oauth2_session.py", line 257, in request
    headers=headers, data=data, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 420, in send
    raise SSLError(e, request=request)
SSLError: <unprintable SSLError object>

I've no experience in Python but for me this looks like an issue with SSL. Most likely provoked by the server side.

And this is the output on a successful run (some minutes later)

# /usr/bin/duplicity collection-status --name duply_owncloud --encrypt-key B44F3F99 --sign-key B44F3F99 --verbosity 9 --gpg-options --compress-algo=bzip2 --ssl-no-check-certificate onedrive://backups/ownCloud
Using archive dir: /root/.cache/duplicity/duply_owncloud
Using backup name: duply_owncloud
Import of duplicity.backends.azurebackend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.copycombackend Succeeded
Import of duplicity.backends.dpbxbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.giobackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.hubicbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.lftpbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.multibackend Succeeded
Import of duplicity.backends.ncftpbackend Succeeded
Import of duplicity.backends.onedrivebackend Succeeded
Import of duplicity.backends.par2backend Succeeded
Import of duplicity.backends.pydrivebackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.ssh_paramiko_backend Succeeded
Import of duplicity.backends.ssh_pexpect_backend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.sxbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
OneDrive id for the configured directory "backups/ownCloud" is "folder.56f6b3babc8c79f2.56F6B3BABC8C79F2!1494"
Main action: collection-status
================================================================================
duplicity 0.7.03 (May 11, 2015)
Args: /usr/bin/duplicity collection-status --name duply_owncloud --encrypt-key B44F3F99 --sign-key B44F3F99 --verbosity 9 --gpg-options --compress-algo=bzip2 --ssl-no-check-certificate onedrive://backups/ownCloud
Linux srvt01 4.2.0-0.bpo.1-amd64 #1 SMP Debian 4.2.6-3~bpo8+2 (2015-12-14) x86_64
/usr/bin/python 2.7.9 (default, Mar 1 2015, 12:57:24)
[GCC 4.9.2]
================================================================================
Local and Remote metadata are synchronized, no sync needed.
797 files exist on backend
163 files exist in cache
Extracting backup chains from list of files: [u'duplicity-full.20150301T233003Z.manifest.gpg', u'duplicity-full.20150301T233003Z.vol1.difftar.gpg', u'duplicity-full.20150301T233003Z.vol10.difftar.gpg', u'duplicity-full.20150301T233003Z.vol11.difftar.gpg', u'duplicity-full.20150301T233003Z.vol12.difftar.gpg', u'duplicity-full.20150301T233003Z.vol13.difftar.gpg', u'duplicity-full.20150301T233003Z.vol14.difftar.gpg', u'duplicity-full.20150301T233003Z.vol15.difftar.gpg', u'duplicity-full.20150301T233003Z.vol16.difftar.gpg', u'duplicity-
-- LOTS OF LINES REMOVED --
Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Mon Jan 4 00:30:05 2016
Chain end time: Tue Jan 26 00:30:06 2016
Number of contained backup sets: 21
Total number of contained volumes: 103
 Type of backup set: Time: Num volumes:
                Full Mon Jan 4 00:30:05 2016 83
         Incremental Tue Jan 5 00:30:07 2016 1
         Incremental Wed Jan 6 00:30:06 2016 1
         Incremental Thu Jan 7 00:30:08 2016 1
         Incremental Fri Jan 8 00:30:06 2016 1
         Incremental Sat Jan 9 00:30:06 2016 1
         Incremental Sun Jan 10 00:30:10 2016 1
         Incremental Mon Jan 11 00:30:07 2016 1
         Incremental Tue Jan 12 00:30:08 2016 1
         Incremental Wed Jan 13 00:30:06 2016 1
         Incremental Thu Jan 14 00:30:07 2016 1
         Incremental Fri Jan 15 00:30:06 2016 1
         Incremental Sat Jan 16 00:30:07 2016 1
         Incremental Sun Jan 17 00:30:06 2016 1
         Incremental Mon Jan 18 00:30:06 2016 1
         Incremental Tue Jan 19 00:30:05 2016 1
         Incremental Wed Jan 20 00:30:06 2016 1
         Incremental Thu Jan 21 00:30:05 2016 1
         Incremental Fri Jan 22 00:30:07 2016 1
         Incremental Sun Jan 24 00:30:06 2016 1
         Incremental Tue Jan 26 00:30:06 2016 1
-------------------------
No orphaned or incomplete backup sets found.
Releasing lockfile <lockfile.LinkFileLock instance at 0x7f3d392ca200>
Using temporary directory /tmp/duplicity-9bGfwW-tempdir

Duplicity is absolutely perfect to backup to storage you don't trust (as onedrive). But it looks like the access to onedrive is so unreliable since a few weeks it's unusable. I've tried accessing onedrive using the web browser and this works perfect all the time and I still have about 800GB of free space on my onedrive account. Anyone else experiencing the same problems? Any ideas how to solve this?

My environment:
- duplicity 0.7.03
- duply v1.9.1
- Python 2.7.9
- Debian Jessie 64bit

Revision history for this message
ThomasL. (tht) wrote :

I've sniffed all the network traffic and this is what I get:

* TCP Connection is open
* Server sends his certificate
* Client responds with "Alert (Level: Fatal, Description: Unknown CA)"
* Client aborts the connection

So why do I get a certificate issue most of the time but not always? This smells like a server side issue. But then again, all my duplicity commandlines do have --ssl-no-check-certificate included. So this unknown CA should not be an issue for the client.

Revision history for this message
ThomasL. (tht) wrote :

Ok. Got it working again. For me it looks like onedrivebackend.py always validates ssl certificates. I added "verify=False" to all http requests done in that file and duplicity is working again.

This is, of course, not the solution. But at least we know for sure the problem is isolated to the ssl server certificate verification.

IN the backend file I found the following URLs:

    API_URI = 'https://apis.live.net/v5.0/'
    OAUTH_TOKEN_URI = 'https://login.live.com/oauth20_token.srf'
    OAUTH_AUTHORIZE_URI = 'https://login.live.com/oauth20_authorize.srf'
    OAUTH_REDIRECT_URI = 'https://login.live.com/oauth20_desktop.srf'

I tested all of them using wget. None of them reports an SSL related issue. So I don't think it's a certificate missing locally. But according to this issue here I assume python_requests is NOT using the system certificate store:
https://github.com/kennethreitz/requests/issues/2966

I exported all system certificates into a single bundle which I specified using the environment variable REQUESTS_CA_BUNDLE. This does not help, still the "same SSLError: <unprintable SSLError object>".

So clearly not an issue related to the backend module, it's an issue related to the CAs MS is using on most of their servers (but not all).

Is there a way to properly import the missing certificate? Or should onedrivebackend.py ignore the certificates?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.