The tests test_cache_layer_domain_crud, test_cache_layer_project_crud, and test_cache_layer_role_crud in keystone.tests.test_backend.IdentityTests may fail depending on which tests have run previously. In particular, if no tests have run previously, then they all fail. Here's an example running each test separately using the KVS backend:
% tox -e py27 -- keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud
py27 develop-inst-nodeps: /opt/stack/keystone
py27 runtests: commands[0] | python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud
running testr
running=${PYTHON:-python} -m subunit.run discover \
-t ./ ./keystone/tests \
--list
running=${PYTHON:-python} -m subunit.run discover \
-t ./ ./keystone/tests \
--load-list /tmp/tmpCi8sRv
======================================================================
FAIL: keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud
tags: worker-0
----------------------------------------------------------------------
Traceback (most recent call last):
File "keystone/tests/test_backend.py", line 2682, in test_cache_layer_role_crud
self.assertDictEqual(role_ref, self.assignment_api.get_role(role_id))
File "keystone/tests/core.py", line 423, in assertDictEqual
self.assertEqual(d1, d2, msg)
File "/opt/stack/keystone/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 322, in assertEqual
self.assertThat(observed, matcher, message)
File "/opt/stack/keystone/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 417, in assertThat
raise MismatchError(matchee, matcher, mismatch, verbose)
MismatchError: !=:
reference = {'id': u'bf17c51445794329987eda59f9ffd64f',
'name': u'ea760f7d48aa4c22a6dcaa54098fd2a5'}
actual = {'id': u'bf17c51445794329987eda59f9ffd64f',
'name': u'815c02c26b3048308b1fb4be22412882'}
======================================================================
FAIL: process-returncode
tags: worker-0
----------------------------------------------------------------------
Binary content:
traceback (test/plain; charset="utf8")
Ran 3 (+1) tests in 0.143s (+0.004s)
FAILED (id=151, failures=2)
error: testr failed (1)
ERROR: InvocationError: '/opt/stack/keystone/.tox/py27/bin/python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_role_crud'
____________________________________________________________________________________ summary _____________________________________________________________________________________
ERROR: py27: commands failed
% tox -e py27 -- keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud
py27 develop-inst-nodeps: /opt/stack/keystone
py27 runtests: commands[0] | python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud
running testr
running=${PYTHON:-python} -m subunit.run discover \
-t ./ ./keystone/tests \
--list
running=${PYTHON:-python} -m subunit.run discover \
-t ./ ./keystone/tests \
--load-list /tmp/tmp4QfzG9
======================================================================
FAIL: keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud
tags: worker-0
----------------------------------------------------------------------
Traceback (most recent call last):
File "keystone/tests/test_backend.py", line 2636, in test_cache_layer_project_crud
project, self.assignment_api.get_project(project_id))
File "keystone/tests/core.py", line 480, in assertDictContainsSubset
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python2.7/unittest/case.py", line 412, in fail
raise self.failureException(msg)
AssertionError: Mismatched values: 'name', expected: '4c6b94aaf6324a2c8d3da20fc19ced63', actual: u'aebeb924708f4cc7a326043d240be3af'
======================================================================
FAIL: process-returncode
tags: worker-0
----------------------------------------------------------------------
Binary content:
traceback (test/plain; charset="utf8")
Ran 3 (+1) tests in 0.180s (+0.040s)
FAILED (id=152, failures=2)
error: testr failed (1)
ERROR: InvocationError: '/opt/stack/keystone/.tox/py27/bin/python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_project_crud'
____________________________________________________________________________________ summary _____________________________________________________________________________________
ERROR: py27: commands failed
% tox -e py27 -- keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud
py27 develop-inst-nodeps: /opt/stack/keystone
py27 runtests: commands[0] | python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud
running testr
running=${PYTHON:-python} -m subunit.run discover \
-t ./ ./keystone/tests \
--list
running=${PYTHON:-python} -m subunit.run discover \
-t ./ ./keystone/tests \
--load-list /tmp/tmpY27vIC
======================================================================
FAIL: keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud
tags: worker-0
----------------------------------------------------------------------
Traceback (most recent call last):
File "keystone/tests/test_backend.py", line 2586, in test_cache_layer_domain_crud
domain_ref, self.assignment_api.get_domain(domain_id))
File "keystone/tests/core.py", line 480, in assertDictContainsSubset
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python2.7/unittest/case.py", line 412, in fail
raise self.failureException(msg)
AssertionError: Mismatched values: 'name', expected: u'2414056b220a4e44ae11c9f4fb00376e', actual: u'a9e3a4ff47844a0f9f4911c9d7dbfa0e'
======================================================================
FAIL: process-returncode
tags: worker-0
----------------------------------------------------------------------
Binary content:
traceback (test/plain; charset="utf8")
Ran 3 (+1) tests in 0.143s (-0.034s)
FAILED (id=153, failures=2)
error: testr failed (1)
ERROR: InvocationError: '/opt/stack/keystone/.tox/py27/bin/python setup.py testr --testr-args=keystone.tests.test_backend_sql.SqlIdentity.test_cache_layer_domain_crud'
____________________________________________________________________________________ summary _____________________________________________________________________________________
ERROR: py27: commands failed
The results are similar for the SQL backend. I haven't tried the other backends.
This test order dependency is particularly when running the tests in parallel since the test execution order is nondeterministic.
Whoops, I ran the SQL backend in the example instead of the KVS as stated. Anyhow, the results are the same.