UnicodeEncode error from "Backup Statistics"

[Lars Renström]

When duplicity tries to do an incremental backup today it crashes with the following Traceback:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1493, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1487, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1336, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1469, in do_backup
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 656, in incremental_backup
    print_statistics(diffdir.stats, bytes_written)
  File "/usr/bin/duplicity", line 594, in print_statistics
    print diffdir.stats.get_stats_logstring(_("Backup Statistics"))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 27: ordinal not in range(128)

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: duplicity 0.6.23-1ubuntu2
ProcVersionSignature: Ubuntu 3.13.0-14.34-generic 3.13.5
Uname: Linux 3.13.0-14-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.13.2-0ubuntu5
Architecture: amd64
CurrentDesktop: Unity
Date: Sun Mar 2 17:40:44 2014
InstallationDate: Installed on 2013-04-30 (305 days ago)
InstallationMedia: Ubuntu 13.04 "Raring Ringtail" - Release amd64 (20130424)
SourcePackage: duplicity
UpgradeStatus: Upgraded to trusty on 2014-01-14 (46 days ago)

[Michael Terry (mterry)]

Interesting. I can get that error by inserting a str() -- i.e. making that line like the following:
    print str(diffdir.stats.get_stats_logstring(_("Backup Statistics")))

But normally, it works for me. Your environment is UTF-8, so print should be able to print the unicode. And xterm seems able to handle it in my testing (didn't show unicode characters right, but it didn't die).

Do you have any ideas why your shell may not be able to handle unicode output? How are you running duplicity?

[Michael Terry (mterry)]

Ah, if LANG=C, I get your crash... So you must be running in a very bare bones environment?

[Lars Renström (lars-renstrom)]

No, I am running a 14.04 on a desktop updated from 13.10. This is what I get from the terminal:

lars@matilda:~$ echo $LANG
sv_SE.UTF-8

Duplicity is running a backup once every week automatically, but lately it has been dying with the error described above.

[Alexander Kubashin (kubashin-a)]

I have same error when running backup from deja-dup.

$ echo $LANG
ru_RU.UTF-8

Deja-dup backup was broken more than month ago on my 14.04 system.

[Alexander Kubashin (kubashin-a)]

When I changed the system language to English, backups run without errors.

P.S. Last line of errors for ru_RU.UTF-8 locale:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-25: ordinal not in range(128)

[faguirre (rodriguesaguirre)]

Yes, same error.

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1493, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1487, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1336, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1457, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 564, in full_backup
    print_statistics(diffdir.stats, bytes_written)
  File "/usr/bin/duplicity", line 594, in print_statistics
    print diffdir.stats.get_stats_logstring(_("Backup Statistics"))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 21: ordinal not in range(128)

$ echo $LANG
pt_BR.UTF-8

[Óscar Rodríguez Ríos (ingorr01)]

Hi,

Same here, since the upgrade from 12.04 to 14.04 every scheduled backup fails with this error.

PS My system language is Galician gl_ES.UTF-8

[Ioannis Iliadis - Ilousis (ioannis.ilousis)]

If System Language is set to English (System Settings → User Accounts → Language → English), then back-up completes flawlessly. This is confirmation to the observation of Alexander Kubashin (kubashin-a).

For el_GR.UTF-8 (as for ru_RU.UTF-8), the last line of errors is:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 16-25: ordinal not in range(128)

This bug, maybe it is the same as Bug #1276327 and similar to Bug #989496 and Bug #554412 .

[Alberto Salvia Novella (es20490446e)]

It renders essential functionality of the package (or a dependent one) broken.

[Daniel Holm (danielholm)]

Anthing I can do to help fixing this?

[Michael Terry (mterry)]

I found the problem, it was that we were passing unicode to the 'print' Python command. I've attached a branch that fixes it to this bug and uploaded a fix to trusty.

[Launchpad Janitor (janitor)]

This bug was fixed in the package duplicity - 0.6.23-1ubuntu4

---------------
duplicity (0.6.23-1ubuntu4) trusty; urgency=medium

  * debian/patches/skip-flaky-test.patch:
    - One of the tests is failing on ppc64el because the test relies on
      guessing how many volumes duplicity will split a bunch of random
      data into. But that's an inherently sketchy guess to make.
      Just disable for now until we can work with upstream on a better
      fix.
 -- Michael Terry <email address hidden> Wed, 09 Apr 2014 09:51:13 -0400

[Coeur Noir (coeur-noir)]

Hello,

upgraded from 13.10 to 14.04 yesterday.

Since déjà dup (daily) backup no longer works. It worked smoothly with 13.10

gerald@Asgard:~$ echo $LANG
fr_FR.UTF-8

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1470, in do_backup
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 637, in incremental_backup
    globals.backend)
  File "/usr/bin/duplicity", line 424, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 145, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 171, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 423, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp, dest_filename, vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 314, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/giobackend.py", line 145, in put
    self.copy_file('put', source_file, target_file)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 330, in iterate
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/giobackend.py", line 130, in copy_file
    log.Info(_("Writing %s") % target.get_parse_name())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 15: ordinal not in range(128)

[Carl Englund (englundc)]

I have duplicity 0.6.23-1ubuntu4, having this problem:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1458, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 542, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 424, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 145, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 171, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 423, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp, dest_filename, vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 314, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/giobackend.py", line 145, in put
    self.copy_file('put', source_file, target_file)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 330, in iterate
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/giobackend.py", line 130, in copy_file
    log.Info(_("Writing %s") % target.get_parse_name())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 16: ordinal not in range(128)

[Galindro (bruno-galindro)]

This is occuring with en_US.UTF-8 too:

https://bugs.launchpad.net/ubuntu/+source/duplicity/+bug/1431322