bzr related tests fail in LXC with unexpected LANG environment

[Gary Poster]

This could be regarded as a bug in launchpad tests, but for now I'll characterize it as a bug in lpsetup because I can fix the bug ("fix" the bug?) by changing lp-setup-lxc-test. Maybe it ought to be fixed in Launchpad, or in lpsetup by setting up the container or host differently. I like the lp-setup-lxc-test fix at the moment.

In some cases (it has happened twice in the past few days, but not consistently), the host container on EC2 will have a LANG setting that the Lucid container cannot support. When this happens and you ssh into the containers, the ssh command sends the LANG through. This causes multiple test failures because of bzr being unhappy. See http://pastebin.ubuntu.com/1037316/ . Benji and I investigated the errors that complained about the following: "RuntimeError: Bzr server did not start correctly. See stdout and stderr reported above." These proved to be the same basic failure state as the first one in the list:

======================================================================
FAILURE: lp.codehosting.tests.test_lpserve.TestLaunchpadServe.test_successful_start_then_stop
worker ID: worker-15
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: log: {{{
37.119 creating repository in file:///tmp/testbzr-mCY92c.tmp/.bzr/.
37.124 creating branch <bzrlib.branch.BzrBranchFormat7 object at 0x9fb70cc> in file:///tmp/testbzr-mCY92c.tmp/
37.139 trying to create missing lock '/tmp/testbzr-mCY92c.tmp/.bzr/checkout/dirstate'
37.140 opening working tree '/tmp/testbzr-mCY92c.tmp'
}}}

Traceback (most recent call last):
  File "/var/lib/buildbot/slaves/slave/lucid-devel/build/lib/lp/codehosting/tests/test_lpserve.py", line 83, in test_successful_start_then_stop
    self.assertFinishedCleanly(result)
  File "/var/lib/buildbot/slaves/slave/lucid-devel/build/lib/lp/codehosting/tests/test_lpserve.py", line 30, in assertFinishedCleanly
    self.assertEqual((0, '', ''), tuple(result))
AssertionError: not equal:
a = (0, '', '')
b = (0,
 '',
 'bzr: warning: unsupported locale setting\n bzr could not set the application locale.\n Although this should be no problem for bzr itself, it might\n cause problems with some plugins. To investigate the issue,\n look at the output of the locale(1p) tool.\n')

Note that the host's LANG is not sent when you use lxc-console, so this can be confusing because it will seem that you cannot duplicate the error.

As mentioned above, I temporarily fixed the problem by explicitly removing LANG when ssh calls into the containers to start tests. I did this by adding "env -u LANG", as in the below.

$ cat `which lp-setup-lxc-test`
#!/bin/sh
# Copyright 2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).

# Test Launchpad using LXC ephemeral instances.

# Assumptions:
# * This script is run as root.

# We intentionally generate a very long line for the
# lxc-start-ephemeral command below because ssh does not propagate
# quotes the way we want. E.g.,
# touch a; touch b; ssh localhost -- ls "a b"
# succeeds, when it should say that the file "a b" does not exist.

set -uex
lxc-start-ephemeral -u buildbot -S '/var/lib/buildbot/.ssh/launchpad_lxc_id_rsa' -o lptests -- \
    "env -u LANG xvfb-run --error-file=/var/tmp/xvfb-errors.log \
    --server-args='-screen 0 1024x768x24' \
    -a $PWD/bin/test --shuffle --subunit $@"

Until convinced otherwise, I think that this solution is fine.