etckeeper runs bzr without a locale set from cron script

[Fabien Tassin]

Binary package hint: bzr

on oneiric:

/etc/cron.daily/etckeeper:
bzr: failed to report crash using apport:
     AssertionError()
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 34: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 936, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 1136, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 697, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 719, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/builtins.py", line 707, in run
    no_recurse, action=action, save=not dry_run)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 49, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 628, in smart_add
    for subf in sorted(os.listdir(abspath)):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 34: ordinal not in range(128)

bzr 2.4b2 on python 2.7.2rc1 (Linux-2.6.39-3-generic-x86_64-with-
    Ubuntu-11.10-oneiric)
arguments: ['/usr/bin/bzr', 'add', '-q', '.']
plugins: bash_completion[2.4b2], builddeb[2.7.0dev], bzrtools[2.4.0],
    changelog_merge[2.4b2], etckeeper[unknown], launchpad[2.4b2],
    netrc_credential_store[2.4b2], news_merge[2.4b2], weave_fmt[unknown]
encoding: 'ANSI_X3.4-1968', fsenc: 'ANSI_X3.4-1968', lang: None

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.
etckeeper warning: bzr add failed

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: bzr 2.4.0~beta2-2ubuntu1
ProcVersionSignature: Ubuntu 2.6.39-3.10-generic 2.6.39
Uname: Linux 2.6.39-3-generic i686
NonfreeKernelModules: nvidia
Architecture: i386
Date: Thu Jun 2 14:56:20 2011
InstallationMedia: Ubuntu 10.04 "Lucid Lynx" - Alpha i386 (20100331)
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_US:en
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/usr/bin/tcsh
SourcePackage: bzr
UpgradeStatus: Upgraded to oneiric on 2011-05-09 (24 days ago)

[Martin Packman (gz)]

As I said in bug 393038 comment 7 this issue is filed as bug 715547.

What's interesting in your case here is that you have `fsenc: 'ANSI_X3.4-1968'` (ie, Bazaar is being told that all your filenames are ascii-only) even though you have `LANG=en_US.UTF-8`. I suggest you double check your environment to make sure that your locales are correctly installed, and that there aren't any LC_* variables overriding that setting.

It may be you need to file a bug against etckeeper if it's to blame for the cron job being run in the C locale rather than your set LANG preference.

[Fabien Tassin (fta)]

please re-read my description: it doesn't impact me when i'm logged in, it impacts etckeeper which is a *cron* job, running as root, i.e. with the default/limited env.
And it's a regression on oneiric, that very same box was running natty before (and maveric and lucid even before) without any etckeeper/bzr problem.

[Fabien Tassin (fta)]

Here is the env:

UPSTART_INSTANCE=
UPSTART_JOB=anacron
TERM=linux
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
SHELL=/bin/sh
PWD=/

[Fabien Tassin (fta)]

bzr dies while listing /etc/ssl/certs/

it now contains tons of certificates from Mozilla, some with weird filenames:

lrwxrwxrwx 1 root root 104 May 31 08:20 T??B??TAK_UEKAE_K??k_Sertifika_Hizmet_Sa??lay??c??s??_-_S??r??m_3.pem -> /usr/share/ca-certificates/mozilla/T??B??TAK_UEKAE_K??k_Sertifika_Hizmet_Sa??lay??c??s??_-_S??r??m_3.crt

[Jelmer Vernooij (jelmer)]

On Sun, 2011-06-05 at 18:31 +0000, Fabien Tassin wrote:
> *** This bug is a duplicate of bug 715547 ***
> https://bugs.launchpad.net/bugs/715547
>
> please re-read my description: it doesn't impact me when i'm logged in, it impacts etckeeper which is a *cron* job, running as root, i.e. with the default/limited env.
> And it's a regression on oneiric, that very same box was running natty before (and maveric and lucid even before) without any etckeeper/bzr problem.
As far as I know this code hasn't really changed in Oneiric - do you
perhaps now have problematic files in your /etc whereas you didn't
previously?

Cheers,

Jelmer

[Fabien Tassin (fta)]

if you really think bzr is not to blame, then etckeeper should stop using bzr and move back to git like in debian.
etckeeper needs to be able to handle anything in /etc, without LANG or anything in a user profile, it's a system tool.

[Jelmer Vernooij (jelmer)]

On Mon, 2011-06-06 at 10:44 +0000, Fabien Tassin wrote:
> if you really think bzr is not to blame, then etckeeper should stop using bzr and move back to git like in debian.
> etckeeper needs to be able to handle anything in /etc, without LANG or anything in a user profile, it's a system tool.
I'm not arguing that this is not a bug in bzr; at the very least it
should give a better error message.

My question was about finding out whether or not this is is a bug that's
been present for a while or if it is a regression in oneiric.

Cheers,

Jelmer

[Fabien Tassin (fta)]

it's probably an older bug, I don't have all those weird certs in any of my production servers, just my dev-boxes and pre-prod servers.

compare:
http://packages.ubuntu.com/natty/all/ca-certificates/filelist
and:
http://packages.ubuntu.com/oneiric/all/ca-certificates/filelist

[Martin Packman (gz)]

Fabien you're right there's another bug here, but etckeeper could use a valid locale when running bzr.

[Martin Pitt (pitti)]

Hello Fabien, or anyone else affected,

Accepted etckeeper into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Tom Ellis (tellis)]

I tested etckeeper from -proposed and it fixes the issue for me.

ubuntu@ch02b12:~$ apt-cache policy etckeeper
etckeeper:
  Installed: 0.56ubuntu2.1
  Candidate: 0.56ubuntu2.1
  Version table:
 *** 0.56ubuntu2.1 0
        500 http://us.archive.ubuntu.com/ubuntu/ oneiric-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     0.56ubuntu2 0
        500 http://archive.ubuntu.com/ubuntu/ oneiric/main amd64 Packages

[Launchpad Janitor (janitor)]

This bug was fixed in the package etckeeper - 0.56ubuntu2.1

---------------
etckeeper (0.56ubuntu2.1) oneiric-proposed; urgency=low

  * etckeeper: LP: #872553, #791839
    - ensure that LANG is set, default to UTF8, necessary for bzr to
      function properly in non-interactive shells (eg, cron); user's
      local LANG will override if set
 -- Dustin Kirkland <email address hidden> Wed, 12 Oct 2011 12:18:25 -0500