5 unicode unit test failures when building Debian package

[Thomas Goirand]

======================================================================
FAIL: keystone.tests.test_exception.ExceptionTestCase.test_invalid_unicode_string
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: pythonlogging:'': {{{Adding cache-proxy 'keystone.tests.test_cache.CacheIsolatingProxy' to backend.}}}

Traceback (most recent call last):
  File "/tmp/buildd/keystone-2014.1~b3/keystone/tests/test_exception.py", line 100, in test_invalid_unicode_string
    self.assertIn('%(attribute)', e.message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 327, in assertIn
    self.assertThat(haystack, Contains(needle))
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: '%(attribute)' not in 'Expecting to find xx in \xe7a va. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.'

======================================================================
FAIL: keystone.tests.test_exception.ExceptionTestCase.test_unicode_string
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: pythonlogging:'': {{{Adding cache-proxy 'keystone.tests.test_cache.CacheIsolatingProxy' to backend.}}}

Traceback (most recent call last):
  File "/tmp/buildd/keystone-2014.1~b3/keystone/tests/test_exception.py", line 92, in test_unicode_string
    self.assertIn(u'\u2013', e.message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 327, in assertIn
    self.assertThat(haystack, Contains(needle))
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 404, in assertThat
    mismatch_error = self._matchHelper(matchee, matcher, message, verbose)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 454, in _matchHelper
    mismatch = matcher.match(matchee)
  File "/usr/lib/python2.7/dist-packages/testtools/matchers/_basic.py", line 285, in match
    if self.needle not in matchee:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 29: ordinal not in range(128)

======================================================================
FAIL: keystone.tests.test_exception.SecurityErrorTestCase.test_invalid_unicode_string
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: pythonlogging:'': {{{Adding cache-proxy 'keystone.tests.test_cache.CacheIsolatingProxy' to backend.}}}

Traceback (most recent call last):
  File "/tmp/buildd/keystone-2014.1~b3/keystone/tests/test_exception.py", line 100, in test_invalid_unicode_string
    self.assertIn('%(attribute)', e.message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 327, in assertIn
    self.assertThat(haystack, Contains(needle))
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: '%(attribute)' not in 'Expecting to find xx in \xe7a va. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.'

======================================================================
FAIL: keystone.tests.test_exception.SecurityErrorTestCase.test_unicode_string
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: pythonlogging:'': {{{Adding cache-proxy 'keystone.tests.test_cache.CacheIsolatingProxy' to backend.}}}

Traceback (most recent call last):
  File "/tmp/buildd/keystone-2014.1~b3/keystone/tests/test_exception.py", line 92, in test_unicode_string
    self.assertIn(u'\u2013', e.message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 327, in assertIn
    self.assertThat(haystack, Contains(needle))
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 404, in assertThat
    mismatch_error = self._matchHelper(matchee, matcher, message, verbose)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 454, in _matchHelper
    mismatch = matcher.match(matchee)
  File "/usr/lib/python2.7/dist-packages/testtools/matchers/_basic.py", line 285, in match
    if self.needle not in matchee:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 29: ordinal not in range(128)

======================================================================
FAIL: keystone.tests.test_wsgi.LocalizedResponseTest.test_static_translated_string_is_Message
----------------------------------------------------------------------
Traceback (most recent call last):
_StringException: pythonlogging:'': {{{Adding cache-proxy 'keystone.tests.test_cache.CacheIsolatingProxy' to backend.}}}

Traceback (most recent call last):
  File "/tmp/buildd/keystone-2014.1~b3/keystone/tests/test_wsgi.py", line 247, in test_static_translated_string_is_Message
    gettextutils.Message)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 372, in assertIsInstance
    self.assertThat(obj, matcher, msg)
  File "/usr/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    raise mismatch_error
MismatchError: 'The request you have made requires authentication.' is not an instance of Message

[David Stanek (dstanek)]

I did a couple tests and couldn't reproduce. Which Debian release are you using? Are you using the default version of Python or did you install something different?

[Thomas Goirand (thomas-goirand)]

I'm using Debian Sid for all of my uploads, and this was using Keystone from Havana beta 3. However, this happens also in Wheezy for the backport of Keystone (Havana release). You can see the build log over here:

https://havana.dev-debian.pkgs.enovance.com/job/keystone/87/consoleFull

As you can see, it's the same exact unit test failures.

If you wish to reproduce the Debian environment, you can do, in an up-to-date Sid:

apt-get build-dep keystone
apt-get install git-buildpackage
git clone git://git.debian.org/openstack/keystone.git
cd keystone
git checkout -b debian/icehouse origin/debian/icehouse
./debian/rules gen-orig-xz
git-buildpackage

You can replace debian/icehouse by debian/havana if you wish.

I hope the above helps.

Thomas Goirand (zigo)

[David Stanek (dstanek)]

I'm having trouble doing a lot of the debian specific things, but there is a patch (defines-gettext-function-to-avoid-ftbfs.patch) that was applied to my local repo. Removing that patch seemed to make the tests work for me. Is that patch needed?

[Thomas Goirand (thomas-goirand)]

Thanks a lot David, this helped me a lot. This bug can be closed.

[David Stanek (dstanek)]

I marked this an invalid as it wasn't a current bug in Keystone. The problem was a Debian specific patch that looks to be a fix for an issue we may have had in the past.