tox py34 tests failing locally when LANG=C

[John L. Villalovos]

Running tox locally I see the following:

$ tox -v -v -epy34
<snip>
{0} ironic.tests.unit.api.v1.test_chassis.TestPost.test_create_chassis_unicode_description [0.044698s] ... ok

Captured stdout:
~~~~~~~~~~~~~~~~
{4} ironic.tests.unit.api.v1.test_nodes.TestListNodes.test_get_nodes_by_provision_state_not_allowed [] ... inprogress
{5} ironic.tests.unit.api.v1.test_nodes.TestPatch.test_add_chassis_id [] ... inprogress
{2} ironic.tests.unit.api.v1.test_nodes.TestListNodes.test_get_boot_device [] ... inprogress
{3} ironic.tests.unit.api.v1.test_nodes.TestPut.test_power_change_when_being_cleaned [] ... inprogress
{11} ironic.tests.unit.api.test_base.TestBase.test_api_setup [] ... inprogress
{9} ironic.tests.unit.api.test_acl.TestACL.test_non_admin_with_admin_header [] ... inprogress
{6} ironic.tests.unit.api.v1.test_chassis.TestPatch.test_remove_non_existent_property_fail [] ... inprogress
{8} ironic.tests.unit.api.v1.test_drivers.TestDriverProperties.test_driver_properties_fake [] ... inprogress
{10} ironic.tests.unit.api.test_acl.TestACL.test_authenticated [] ... inprogress
{1} ironic.tests.unit.api.v1.test_nodes.TestListNodes.test_sort_key_invalid [] ... inprogress
{7} ironic.tests.unit.api.v1.test_chassis.TestPost.test_create_chassis_doesnt_contain_id [] ... inprogress
Traceback (most recent call last):
  File "/home/john/openstack/ironic/.tox/py34/bin/subunit-trace", line 11, in <module>
    sys.exit(main())
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/os_testr/subunit_trace.py", line 324, in main
    stream.run(result)
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/subunit/v2.py", line 275, in run
    self._parse_packet(result)
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/subunit/v2.py", line 344, in _parse_packet
    self._parse(packet, result)
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/subunit/v2.py", line 471, in _parse
    route_code=route_code, timestamp=timestamp)
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/testtools/testresult/real.py", line 516, in status
    target.status(**kwargs)
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/testtools/testresult/real.py", line 415, in status
    domap(methodcaller('status', *args, **kwargs), self.targets)
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/testtools/testresult/real.py", line 390, in domap
    return list(map(*args, **kwargs))
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/testtools/testresult/real.py", line 655, in status
    self.on_test(self._inprogress.pop(key))
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/os_testr/subunit_trace.py", line 186, in show_outcome
    print_attachments(stream, test)
  File "/home/john/openstack/ironic/.tox/py34/lib/python3.4/site-packages/os_testr/subunit_trace.py", line 123, in print_attachments
    stream.write(" %s\n" % line)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 39-41: ordinal not in range(128)

Slowest Tests:

[Errno 32] Broken pipe
Test id Runtime (s)
----------------------------------------------------------------------------------------- -----------
ironic.tests.unit.api.test_acl.TestACL.test_authenticated 0.374
ironic.tests.unit.api.v1.test_chassis.TestListChassis.test_empty 0.350
ironic.tests.unit.api.test_acl.TestACL.test_non_admin_with_admin_header 0.339
ironic.tests.unit.api.test_acl.TestACL.test_non_admin 0.224
ironic.tests.unit.api.test_base.TestVersion.test_parse_headers_ok 0.220
ironic.tests.unit.api.test_hooks.TestNoExceptionTracebackHook.test_hook_without_traceback 0.205
ironic.tests.unit.api.test_base.TestVersion.test_lessthan 0.202
ironic.tests.unit.api.test_base.TestBase.test_api_setup 0.188
ironic.tests.unit.api.test_hooks.TestContextHook.test_context_hook_admin 0.187
ironic.tests.unit.api.test_hooks.TestContextHook.test_context_hook_noauth_token_removed 0.185
ERROR: InvocationError: '/home/john/openstack/ironic/.tox/py34/bin/ostestr'
py34 finish: runtests after 8.52 seconds
_____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
ERROR: py34: commands failed

[John L. Villalovos (happycamp)]

This seems to be related to LANG=C in my environment.

When I did: LANG=en_US.UTF-8

It worked.

Now why does it break with LANG=C?

[Robert Collins (lifeless)]

LANG=C implies that you can't output non-ascii text at all. Don't do that :)

[Matthew Treinish (treinish)]

Heh, yeah that's what my attitude was when this was reported.

But, that 1 line change in subunit-trace fixes it for John and doesn't seem to break anything else, so why not.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229527
Committed: https://git.openstack.org/cgit/openstack/os-testr/commit/?id=65e1e1867c30fc70e4963df7f5bd9a15acf54a68
Submitter: Jenkins
Branch: master

commit 65e1e1867c30fc70e4963df7f5bd9a15acf54a68
Author: Matthew Treinish <email address hidden>
Date: Wed Sep 30 13:00:23 2015 -0400

    Force utf8 encoding on subunit attachments output

    On certain environments if the locale is set to ASCII when printing an
    attachment things will explode because it's potentially trying to use
    unicode chars outside of ASCII. This commit forces the attachment
    output to be unicode encoded to avoid this issue.

    Change-Id: I79ff7482ec3b1fd3ce83b8acf3137119b3db39a9
    Closes-Bug: #1501415

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229520
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=3e22cc4ea0de9960b2892c4f7546681c02a504ff
Submitter: Jenkins
Branch: master

commit 3e22cc4ea0de9960b2892c4f7546681c02a504ff
Author: John L. Villalovos <email address hidden>
Date: Wed Sep 30 09:39:46 2015 -0700

    Python 3.4 unit tests fail with LANG=C

    After the move to os-testr the 'tox -epy34' unit tests will fail when
    LANG=C is set in the environment.

    Set LC_ALL=en_US.UTF-8 in the testenv to solve the issue. This is
    in-line with our curent method of running the unit tests in US English
    so that logging, messages, and exceptions can be validated by unit
    tests.

    Closes-Bug: #1501415
    Change-Id: Ifefdd67c9e5edf22f3b7a1fce56f21c788ac9ead

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/ironic 4.3.0 release.