UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2886: ordinal not in range(128)

[Marius Gedminas]

I got this in the mail:

    From: Cron Daemon <root@...>
    To: root@...
    Subject: Cron <root@lims> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
    Date: Thu, 25 Feb 2016 02:11:47 +0200 (EET)

    /etc/cron.daily/apt:
    Traceback (most recent call last):
      File "/usr/bin/unattended-upgrade", line 1255, in <module>
        main(options)
      File "/usr/bin/unattended-upgrade", line 1205, in main
        pkgs, pkg_install_success, pkgs_kept_back, mem_log, logfile_dpkg)
      File "/usr/bin/unattended-upgrade", line 752, in send_summary_mail
        body += fp.read()
      File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2886: ordinal not in range(128)

According to /var/log/apt/history.log.1.gz, the packages upgraded that day were

    Start-Date: 2016-02-25 02:06:22
    Upgrade: libgnutls-openssl27:amd64 (2.12.23-12ubuntu2.4, 2.12.23-12ubuntu2.5), libssl1.0.0:amd64 (1.0.1f-1ubuntu2.16, 1.0.1f-1ubuntu2.17), libvirt0:amd64 (1.2.2-0ubuntu13.1.16, 1.2.2-0ubuntu13.1.17), libssl-dev:amd64 (1.0.1f-1ubuntu2.16, 1.0.1f-1ubuntu2.17), ca-certificates:amd64 (20141019ubuntu0.14.04.1, 20160104ubuntu0.14.04.1), libgnutls26:amd64 (2.12.23-12ubuntu2.4, 2.12.23-12ubuntu2.5), libssl-doc:amd64 (1.0.1f-1ubuntu2.16, 1.0.1f-1ubuntu2.17), openssl:amd64 (1.0.1f-1ubuntu2.16, 1.0.1f-1ubuntu2.17)
    End-Date: 2016-02-25 02:06:36

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: unattended-upgrades 0.82.1ubuntu2.4
Uname: Linux 2.6.32-042stab108.8 x86_64
ApportVersion: 2.14.1-0ubuntu3.19
Architecture: amd64
Date: Tue Mar 8 08:26:05 2016
PackageArchitecture: all
ProcEnviron:
 LC_CTYPE=lt_LT.UTF-8
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=lt_LT.UTF-8
 SHELL=/bin/bash
SourcePackage: unattended-upgrades
UpgradeStatus: Upgraded to trusty on 2016-02-06 (30 days ago)

[Marius Gedminas (mgedmin)]

/var/log/unattended-upgrades/unattended-upgrades-dpkg_2016-02-25_02:06:20.971350.log contains this line:

    Adding debian:TÜRKTRUST_Elektronik_Sertifika_Hizmet_Sağlayıcısı_H5.pem

the Ü is at byte position 2888 of 10432 according to vim (which uses a 1-based offset and probably points to the last byte of multi-byte characters).

[Marius Gedminas (mgedmin)]

I got this error email from four different OpenVZ-based VPSes, but not from any other servers. I wonder what's peculiar about their configuration. Locale used by cron?

[Daniel Kraft (daniel-kraft)]

This still exists in 16.04 and is hitting uns every day.

[Daniel Kraft (daniel-kraft)]

This is the traceback from the logs in 16.04:

apt.systemd.daily[957]: Traceback (most recent call last):
apt.systemd.daily[957]: File "/usr/bin/unattended-upgrade", line 1473, in <module>
apt.systemd.daily[957]: main(options)
apt.systemd.daily[957]: File "/usr/bin/unattended-upgrade", line 1411, in main
apt.systemd.daily[957]: log_content = get_dpkg_log_content(logfile_dpkg, install_start_time)
apt.systemd.daily[957]: File "/usr/bin/unattended-upgrade", line 1079, in get_dpkg_log_content
apt.systemd.daily[957]: for line in fp.readlines():
apt.systemd.daily[957]: File "/usr/lib/python3.5/codecs.py", line 321, in decode
apt.systemd.daily[957]: (result, consumed) = self._buffer_decode(data, self.errors, final)
apt.systemd.daily[957]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 758: invalid start byte

Could it be that dpkg logs according to locale settings? These are the ones on this host:

LANG=de_DE
LANGUAGE=de_DE:
LC_CTYPE=de_DE.UTF-8
LC_NUMERIC="de_DE"
LC_TIME="de_DE"
LC_COLLATE="de_DE"
LC_MONETARY="de_DE"
LC_MESSAGES="de_DE"
LC_PAPER="de_DE"
LC_NAME="de_DE"
LC_ADDRESS="de_DE"
LC_TELEPHONE="de_DE"
LC_MEASUREMENT="de_DE"
LC_IDENTIFICATION="de_DE"
LC_ALL=

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Marius Gedminas (mgedmin)]

Last night's update gave me two more of these errors, both in OpenVZ containers.

[Marius Gedminas (mgedmin)]

It looks like the unattended upgrade itself was successful, it's just that instead of the usual email showing me the upgrade log I get this traceback.