UnicodeEncodeError: 'ascii' codec can't encode character u'\ufeff'

[Galindro]

Guys,

   This error is ocurring when I try to execute a full backup to Windows Azure.

================================================================================
duplicity 0.7.01 ($reldate)
Args: /usr/bin/duplicity --archive-dir /backup/duplicity_cache/ --no-encryption --volsize 1000 --extra-clean --verbosity 8 --name s3 /backup/s3/ azure://aws-s3
Linux SV-BACKUP-AMAZON 3.13.0-46-generic #77-Ubuntu SMP Mon Mar 2 18:23:39 UTC 2015 x86_64 x86_64
/usr/bin/python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2]
================================================================================

AsyncScheduler: running task synchronously (asynchronicity disabled)
Writing duplicity-full.20150312T104506Z.vol5.difftar.gz
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1497, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1491, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1340, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1466, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 536, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 418, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 145, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 171, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 417, in <lambda>
    async_waiters.append(io_scheduler.schedule_task(lambda tdp, dest_filename, vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 308, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 387, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 78, in uexc
    return ufn(str(e))
UnicodeEncodeError: 'ascii' codec can't encode character u'\ufeff' in position 76: ordinal not in range(128)

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty

# dpkg -l |grep duplicity
ii duplicity 0.7.01-0ubuntu0ppa1063~ubuntu14.04.1 amd64 encrypted bandwidth-efficient backup

# locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

[Galindro (bruno-galindro)]

Any return?

[Michael Terry (mterry)]

That character (0xfeff) is used at the beginning of a document to indicate byte-order. Looks like our assumptions about the encoding of messages that the backends give us is wrong (we're getting unicode from the backend, but expecting ascii). I'm guessing we should check in utils.uexc whether we already have unicode.

[KevinZ (kzembower)]

I also have this problem with duplicity 0.6.23:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1494, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1488, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1337, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1366, in do_backup
    sync_archive(decrypt)
  File "/usr/bin/duplicity", line 1100, in sync_archive
    remote_metafiles, ignored, rem_needpass = get_metafiles(remlist)
  File "/usr/bin/duplicity", line 992, in get_metafiles
    pr = file_naming.parse(fn)
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 391, in parse
    pr = check_inc()
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 332, in check_inc
    t2 = str2time((m1 or m2).group("end_time"), short)
  File "/usr/lib/python2.7/dist-packages/duplicity/file_naming.py", line 281, in str2time
    t = dup_time.genstrtotime(timestr.upper())
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 278, in genstrtotime
    return override_curtime - intstringtoseconds(timestr)
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 190, in intstringtoseconds
    error()
  File "/usr/lib/python2.7/dist-packages/duplicity/dup_time.py", line 181, in error
    raise TimeException(bad_interval_string % interval_string)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 8: ordinal not in range(128)

As far as I can tell, this bug has been present in a similar form since 2012. This is on a Ubuntu 14.04.4 LTS system.

Thanks for looking into this. Let me know if there's any other information I can find.

[Wolf (drechsel)]

Mine looks similar - ubuntu 10.04:

saving data fails with unknown error:

Traceback (innermost last):
  File "/usr/bin/duplicity", line 1555, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1541, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1393, in main
    do_backup(action)
  File "/usr/bin/duplicity", line 1511, in do_backup
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 572, in full_backup
    globals.backend)
  File "/usr/bin/duplicity", line 454, in write_multivol
    (tdp, dest_filename, vol_num)))
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 146, in schedule_task
    return self.__run_synchronously(fn, params)
  File "/usr/lib/python2.7/dist-packages/duplicity/asyncscheduler.py", line 172, in __run_synchronously
    ret = fn(*params)
  File "/usr/bin/duplicity", line 453, in <lambda>
    vol_num: put(tdp, dest_filename, vol_num),
  File "/usr/bin/duplicity", line 342, in put
    backend.put(tdp, dest_filename)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 395, in inner_retry
    % (n, e.__class__.__name__, util.uexc(e)))
  File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 79, in uexc
    return ufn(unicode(e).encode('utf-8'))
 UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 33: ordinal not in range(128)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.