UnicodeDecodeError: 'ascii' codec can't decode byte 0x85 in position 0: ordinal not in range(128) with PCA

[Xavier Poinsard]

Same kind of bug after last fix :
I am using duplicity 0.8.11.1607 from snap edge channel.
I am trying to use multi backend with one of them being the OVH PCA.
It fails when trying to write to the PCA with the following error :
MultiBackend: _put: write to store #0 (pca://backupPortableX-cold)
Writing cold_duplicity-full.20200315T171233Z.vol1.difftar.gpg
Backtrace of previous error: Traceback (innermost last):
  File "/snap/duplicity/86/lib/python3.6/site-packages/duplicity/backend.py", line 376, in inner_retry
    return fn(self, *args)
  File "/snap/duplicity/86/lib/python3.6/site-packages/duplicity/backend.py", line 547, in put
    self.__do_put(source_path, remote_filename)
  File "/snap/duplicity/86/lib/python3.6/site-packages/duplicity/backend.py", line 533, in __do_put
    self.backend._put(source_path, remote_filename)
  File "/snap/duplicity/86/lib/python3.6/site-packages/duplicity/backends/pcabackend.py", line 154, in _put
    open(util.fsdecode(source_path.name)))
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/swiftclient/client.py", line 1842, in put_object
    response_dict=response_dict)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/swiftclient/client.py", line 1691, in _retry
    service_token=self.service_token, **kwargs)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/swiftclient/client.py", line 1320, in put_object
    conn.putrequest(path, headers=headers, data=data)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/swiftclient/client.py", line 449, in putrequest
    return self.request('PUT', full_path, data, headers, files)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/swiftclient/client.py", line 439, in request
    files=files, **self.requests_args)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/swiftclient/client.py", line 422, in _request
    return self.request_session.request(*arg, **kwarg)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
    resp = self.send(prep, **send_kwargs)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
    r = adapter.send(request, **kwargs)
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/requests/adapters.py", line 460, in send
    for i in request.body:
  File "/snap/duplicity/86/usr/lib/python3/dist-packages/swiftclient/utils.py", line 279, in __next__
    chunk = self.content.read(self.chunk_size)
  File "/snap/duplicity/86/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
 UnicodeDecodeError: 'ascii' codec can't decode byte 0x85 in position 0: ordinal not in range(128)

[Kenneth Loafman (kenneth-loafman)]

OK, this bug has to be in the python3-swiftclient module. The real problem is that Ubuntu is behind on versions. The latest version is 3.9.0 while the version in the snap is 3.5.0. Are you required to use the snap version? If not you might want to try one of the installs below.

* Stable tarball install - https://launchpad.net/duplicity/+download
* Daily duplicity builds - https://launchpad.net/~duplicity-team/+archive/ubuntu/daily-dev-trunk
* Latest pip builds - “sudo pip install duplicity"

[Kenneth Loafman (kenneth-loafman)]

One suggestion is to try the rclone backend. You'll need to configure it before running duplicity, but it may work for you. It's in the snap you have and should be up-to-date. Search for rclone in the man page.

[Xavier Poinsard (xpoinsard)]

I just tried without snap, and using duplicity 0.8.11-0ubuntu1ppa1608~ubuntu18.04.1 package and python3-swiftclient 1:3.9.0-0ubuntu1 package :

MultiBackend: _put: write to store #0 (pca://backupPortableX-cold)
Writing cold_duplicity-full.20200316T142347Z.vol1.difftar.gpg
Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 376, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 547, in put
    self.__do_put(source_path, remote_filename)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 533, in __do_put
    self.backend._put(source_path, remote_filename)
  File "/usr/lib/python3/dist-packages/duplicity/backends/pcabackend.py", line 154, in _put
    open(util.fsdecode(source_path.name)))
  File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1908, in put_object
    response_dict=response_dict)
  File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1748, in _retry
    service_token=self.service_token, **kwargs)
  File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1375, in put_object
    conn.putrequest(path, headers=headers, data=data)
  File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 479, in putrequest
    return self.request('PUT', full_path, data, headers, files)
  File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 469, in request
    files=files, **self.requests_args)
  File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 452, in _request
    return self.request_session.request(*arg, **kwarg)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 460, in send
    for i in request.body:
  File "/usr/lib/python3/dist-packages/swiftclient/utils.py", line 298, in __next__
    chunk = self.content.read(self.chunk_size)
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
 UnicodeDecodeError: 'ascii' codec can't decode byte 0x85 in position 0: ordinal not in range(128)

[Kenneth Loafman (kenneth-loafman)]

Thanks for making the effort!

I'd like a log file, please. Run duplicity with -v9 and redirect to a log file. Then attach, DO NOT COPY PASTE, the log file to this bug report. Munge passwords. Munge paths as little as possible. Especially, do no change the structure of the URLs.

[Kenneth Loafman (kenneth-loafman)]

OK, I think I have a fix after scrounging through the swiftclient code. The file was not being opened in binary mode, so a few char fix, and we may have a client.