UnicodeDecodeError when backing up to Ubuntu One in some locales

[João Antonio Santana]

[Impact]
Most non-English users cannot back up to Ubuntu One (the default in Deja Dup) at all.

When backing up to Ubuntu One, the duplicity Ubuntu One backend will return unicode filenames. This can cause UnicodeDecodeErrors in some locales when mixed with duplicity's UTF-8 translated strings.

For example, pt_BR is affected because it uses non-ascii characters.

As for Ubuntu, Quantal and Raring are affected. Precise and below don't seem to care.

[Test Case]
sudo apt-get install language-pack-pt-base
LANGUAGE=pt_BR duplicity --no-encryption /bin u1+http://utf8bug

[Fix]
In u1backend.py, an ".encode('utf-8')" needs to be added in the list() function.

https://code.launchpad.net/~mterry/duplicity/u1-utf8/+merge/138275

[Exception]
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1404, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1397, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1277, in main
    globals.archive_dir).set_values()
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 691, in set_values
    self.get_backup_chains(partials + backend_filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 814, in get_backup_chains
    map(add_to_sets, filename_list)
  File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 808, in add_to_sets
    log.Debug(_("File %s is not part of a known set; creating new set") % (filename,))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal not in range(128)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Michael Terry (mterry)]

I've uploaded to raring and quantal-proposed. Subscribing ubuntu-sru.

[Launchpad Janitor (janitor)]

This bug was fixed in the package duplicity - 0.6.20-0ubuntu2

---------------
duplicity (0.6.20-0ubuntu2) raring; urgency=low

  * debian/patches/03u1backend.dpatch:
    - Update with a fix for a UnicodeDecodeError crash (LP: #1080423)
 -- Michael Terry <email address hidden> Wed, 05 Dec 2012 16:16:44 -0500

[Pilot6 (hanipouspilot)]

When I try to backup using 0.6.20 I get
UnsupportedBackendScheme: scheme not supported in url: u1+http://deja-dup/Pilot6-2 in Deja Dup

[Michael Terry (mterry)]

Pilot6, did you try running 0.6.20 without the raring deja-dup version? 0.6.20 changed the U1 backend and you need different dependencies or it won't work. I'd say wait until the quantal-proposed fix for 0.6.19 comes through. The archive admins will leave a message here when they approve the package for testing.

[Alexandr Makovksy (mailboxmak)]

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1404, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1397, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1248, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 994, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 161, in get_backend
    return _backends[pu.scheme](pu)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/u1backend.py", line 74, in __init__
    self.create_volume()
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 328, in iterate
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/u1backend.py", line 161, in create_volume
    answer = auth.request(self.volume_uri, http_method="PUT")
  File "/usr/lib/python2.7/dist-packages/ubuntuone-couch/ubuntuone/couch/auth.py", line 152, in request
    url, method=http_method, headers=headers, body=request_body)
  File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1543, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1293, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1229, in _conn_request
    conn.connect()
  File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1018, in connect
    raise socket.error, msg
error: [Errno 111] Connection refused

[Brian Murray (brian-murray)]

Hello João, or anyone else affected,

Accepted duplicity into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/duplicity/0.6.19-0ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Pilot6 (hanipouspilot)]

Update works for me. Thanks!

[Pilot6 (hanipouspilot)]

I mean 0.6.19 in quantal.

[Ares Aguilar Sotos (serary93)]

0.6.19 works for me in Ubuntu 12.10.
Thanks!

[Colin Watson (cjwatson)]

duplicity (0.6.19-0ubuntu2.2) quantal-proposed; urgency=low

  * debian/patches/09volcorruption.dpatch:
    - Backport a fix for a restart test that caused FTBFS on armhf

 -- Michael Terry <email address hidden> Fri, 11 Jan 2013 13:59:19 -0500

duplicity (0.6.19-0ubuntu2.1) quantal-proposed; urgency=low

  [ Michael Terry ]
  * debian/patches/07u1utf8.dpatch:
    - Update with a fix for a UnicodeDecodeError crash (LP: #1080423)
  * debian/patches/09volcorruption.dpatch:
    - Fix some data-corruption issues when resuming an interrupted
      backup (LP: #1091269)

  [ Chris J Arges ]
  * debian/patches/08caching.dpatch:
    - Backport caching work done by Steve Atwell. (LP: #1013446)

 -- Michael Terry <email address hidden> Wed, 05 Dec 2012 16:39:01 -0500