unit tests fail based on wall clock time

Bug #1464366 reported by Brant Knudson
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
Medium
Brant Knudson

Bug Description

We've got a lot of tests that depend on how long the test takes to run. Tests can take a long time just because you have a slow or overloaded system, or maybe you're trying to step through it with the debugger.

The tests that fail generally don't care about the time and aren't attempting to verify performance, but still require that the test run quickly enough.

Tests shouldn't depend on the wall clock time, just like they shouldn't depend on any external factors.

Here's an example of a failing test:

keystone.tests.unit.test_auth.AuthWithRemoteUser.test_scoped_remote_authn
-------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/test_auth.py", line 741, in test_scoped_remote_authn
        enforce_audit_ids=False)
      File "keystone/tests/unit/test_auth.py", line 104, in assertEqualTokens
        timeutils.parse_isotime(b['access']['token']['expires']))
      File "keystone/tests/unit/core.py", line 521, in assertCloseEnoughForGovernmentWork
        self.assertTrue(abs(a - b).seconds <= delta, msg)
      File "/home/jenkins/workspace/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/case.py", line 678, in assertTrue
        raise self.failureException(msg)
    AssertionError: False is not true : 2015-06-11 13:34:46+00:00 != 2015-06-11 13:34:50+00:00 within 3 delta

It took 4 seconds rather than 3.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/190790

Changed in keystone:
status: New → In Progress
tags: added: test-improvement
Changed in keystone:
importance: Undecided → Medium
Changed in keystone:
status: In Progress → Fix Committed
Changed in keystone:
milestone: none → liberty-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in keystone:
milestone: liberty-1 → 8.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.