Use testr instead of nose as the unittest runner.

Bug #1177924 reported by Clark Boylan on 2013-05-08
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Ceilometer
Undecided
Angus Salkeld
Cinder
Medium
Michael Kerrin
Glance
Undecided
Unassigned
OpenStack DBaaS (Trove)
Low
Unassigned
OpenStack Dashboard (Horizon)
Wishlist
Unassigned
OpenStack Identity (keystone)
Wishlist
David Stanek
OpenStack Object Storage (swift)
Wishlist
Richard Hawkins
django-openstack-auth
Wishlist
Unassigned
manila-ui
Wishlist
Unassigned
python-ceilometerclient
Undecided
Angus Salkeld
python-cinderclient
Medium
Unassigned
python-heatclient
Fix Released
Medium
Steve Baker
python-keystoneclient
Wishlist
Clark Boylan

Bug Description

We want to start using testr as our test runner instead of nose so that we can start running tests in parallel. For the projects that have switched we have seen improvements to test speed and quality.

As part of getting set for that, we need to start using testtools and fixtures so provide the plumbing and test isolation needed for automatic parallelization. The work can be done piecemeal - with testtools and fixtures being added first, and then tox/run_tests being ported to us testr/subunit instead of nose.

This work was semi tracked during Grizzly with this https://blueprints.launchpad.net/openstack-ci/+spec/grizzly-testtools blueprint. I am opening this bug so that we can track migration to testr on a per project basis.

CVE References

Changed in python-heatclient:
assignee: nobody → Steve Baker (steve-stevebaker)
importance: Undecided → Medium
Angus Salkeld (asalkeld) wrote :
Changed in ceilometer:
assignee: nobody → Angus Salkeld (asalkeld)
Changed in python-ceilometerclient:
assignee: nobody → Angus Salkeld (asalkeld)
Clark Boylan (cboylan) wrote :

https://review.openstack.org/#/c/18637/ implements the use of testr for python-keystoneclient. It is currently in need of reviews.

Changed in python-keystoneclient:
assignee: nobody → Clark Boylan (cboylan)
status: New → In Progress
Changed in cinder:
importance: Undecided → Medium
status: New → Confirmed
Changed in python-cinderclient:
importance: Undecided → Medium
status: New → Confirmed

Fix proposed to branch: master
Review: https://review.openstack.org/29043

Changed in ceilometer:
status: New → In Progress

Fix proposed to branch: master
Review: https://review.openstack.org/29044

Fix proposed to branch: master
Review: https://review.openstack.org/29045

Fix proposed to branch: master
Review: https://review.openstack.org/29046

Fix proposed to branch: master
Review: https://review.openstack.org/29047

Gabriel Hurley (gabriel-hurley) wrote :

I'll be interested to see how this works out since Horizon has fairly different testing infrastructure than most projects since it relies on Django's test harnass as well as selenium...

Changed in horizon:
importance: Undecided → Wishlist
milestone: none → havana-1
status: New → Triaged

Reviewed: https://review.openstack.org/29043
Committed: http://github.com/openstack/ceilometer/commit/06093adfafa330a0087cb834ada16fd33510e11b
Submitter: Jenkins
Branch: master

commit 06093adfafa330a0087cb834ada16fd33510e11b
Author: Angus Salkeld <email address hidden>
Date: Thu May 9 14:46:36 2013 +1000

    Get all tests to use tests.base.TestCase

    part of bug 1177924

    Change-Id: If8124e2ca21bf9766adf770e30a630437c6fcd95

Changed in ceilometer:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/29045
Committed: http://github.com/openstack/ceilometer/commit/8da9ac25fe95c24eeae9a5654ea05cae504ef56d
Submitter: Jenkins
Branch: master

commit 8da9ac25fe95c24eeae9a5654ea05cae504ef56d
Author: Angus Salkeld <email address hidden>
Date: Tue May 14 14:20:53 2013 +1000

    remove unused tests/skip module

    part of bug 1177924
    Change-Id: I9760d81b1bb52a3847cd2c6cc595dad29042a701

Reviewed: https://review.openstack.org/29044
Committed: http://github.com/openstack/ceilometer/commit/e9a30f61fe1f51a8758c4587e8822723fefc3c4e
Submitter: Jenkins
Branch: master

commit e9a30f61fe1f51a8758c4587e8822723fefc3c4e
Author: Angus Salkeld <email address hidden>
Date: Fri May 17 09:52:34 2013 +1000

    Change from unittest to testtools

    part of bug 1177924

    Change-Id: I99be2c48dc7143015883b89130083c8ea11abeec
    Signed-off-by: Doug Hellmann <email address hidden>

Angus Salkeld (asalkeld) wrote :
Changed in python-ceilometerclient:
status: New → Fix Committed

Reviewed: https://review.openstack.org/29046
Committed: http://github.com/openstack/ceilometer/commit/6693c9fb04722de9b911fd51a708d70158760178
Submitter: Jenkins
Branch: master

commit 6693c9fb04722de9b911fd51a708d70158760178
Author: Angus Salkeld <email address hidden>
Date: Sat May 18 17:26:38 2013 +1000

    Use fixtures in the tests

    TempDir() makes sure all files created under it are deleted
    at the end of the tests.
    FakeLogger() hides logs when the test passes and displays logs
    for failed tests.

    part of bug 1177924
    Change-Id: I07acb66daa1932d7864a5431f1b64570b747ce5a

Reviewed: https://review.openstack.org/29047
Committed: http://github.com/openstack/ceilometer/commit/1a723a05c74361d8fe52220061b0697de8003221
Submitter: Jenkins
Branch: master

commit 1a723a05c74361d8fe52220061b0697de8003221
Author: Angus Salkeld <email address hidden>
Date: Sat May 18 18:06:38 2013 +1000

    Replace nose with testr

    - add .testr.conf (in base dir and nova_tests)
    - remove all references to nose
    - adjust paths: A couple of tests would assume the current directory
      was ./tests/ but it is now ./
    - don't run the tests in parallel as the db tests have one connection
      per test class.

    nova_tests hackery:
    It seems testtools/testr imports everything and I just don't see a way of
    running things in tests/ and nova_tests/ seperately but in one .test.conf.
    So if you want to use testr directly you will need to:
     testr run
     cd nova_tests
     testr run

    part of bug 1177924
    Change-Id: I41875dcf94463fa5f9c07a7840c37089226c59ad

Changed in horizon:
milestone: havana-1 → havana-2

Fix proposed to branch: master
Review: https://review.openstack.org/30291

Changed in cinder:
assignee: nobody → Michael Kerrin (michael-kerrin-w)
status: Confirmed → In Progress
Thierry Carrez (ttx) on 2013-05-29
Changed in ceilometer:
milestone: none → havana-1
status: Fix Committed → Fix Released
Dolph Mathews (dolph) on 2013-05-29
Changed in python-keystoneclient:
importance: Undecided → Wishlist
Clark Boylan (cboylan) on 2013-06-04
Changed in python-keystoneclient:
status: In Progress → Fix Committed
Dolph Mathews (dolph) on 2013-06-07
Changed in keystone:
importance: Undecided → Wishlist
status: New → Confirmed
Clark Boylan (cboylan) on 2013-06-11
Changed in python-cinderclient:
status: Confirmed → Fix Committed
Clark Boylan (cboylan) on 2013-06-11
Changed in python-heatclient:
status: New → Fix Committed
Kieran Spear (kspear) on 2013-06-11
Changed in horizon:
assignee: nobody → Kieran Spear (kspear)

Reviewed: https://review.openstack.org/30291
Committed: http://github.com/openstack/cinder/commit/930f5891b0815e1b49b9b2cc840e0c24b2796e84
Submitter: Jenkins
Branch: master

commit 930f5891b0815e1b49b9b2cc840e0c24b2796e84
Author: Michael Kerrin <email address hidden>
Date: Thu May 16 08:10:27 2013 +0000

    Fix up the test framework

    When trying to test a bug the cinder test framework was
    running all the tests when I just wanted one to run.

    Instead of fixing this problem in nose I tried to move cinder to
    testr. I copied a fair bit of code from nova to get this to work.
    This way at least if some one makes an improvement to the nova
    test framework cinder can improve here also.

    Note that now all the tests a thread safe. Added a concurrency argument
    to run_tests.sh that defaults to 1. You can specify your own level then
    and take the risk of intermittent failures. Tell jenkins to use a
    concurrency level of 1 to avoid failing gate jobs.

    Note One of these tests is broken and I have disabled for now. There
    is a bug for this here: https://bugs.launchpad.net/cinder/+bug/1173266

    Note: cinder.api.openstack.FaultWrapper is deprecated and is causing
    a circular import when I try and import cinder.tests.test_wsgi. This
    is fixed by only importing the openstack.FaultWrapper during the
    __init__ phase of the wsgi middleware.

    Fixes: bug 1183434
    Fixes: bug 1177924

    Change-Id: I5e10b55c5b236eb81a6a3e0e9ea56af8ca4ef8e1

Changed in cinder:
status: In Progress → Fix Committed
Dolph Mathews (dolph) on 2013-06-26
Changed in python-keystoneclient:
milestone: none → 0.3.0
status: Fix Committed → Fix Released

I am out of the office until 07/01/2013.

I will be on vacation until Monday, July 1st and will not be able to
respond to emails for the majority of my time away. Please contact Johanna
Koester, my manager, in case of emergencies.

Note: This is an automated response to your message "[Bug 1177924] Re: Use
testr instead of nose as the unittest runner." sent on 06/26/2013 7:51:29
AM.

This is the only notification you will receive while this person is away.

Dolph Mathews (dolph) wrote :

I think we should escalate this bug Johanna Koester ^

Changed in horizon:
milestone: havana-2 → havana-3
Changed in python-heatclient:
milestone: none → v0.2.3
Thierry Carrez (ttx) on 2013-07-17
Changed in cinder:
milestone: none → havana-2
status: Fix Committed → Fix Released
Changed in horizon:
milestone: havana-3 → none
Changed in keystone:
assignee: nobody → Morgan Fainberg (mdrnstm)
Changed in python-heatclient:
milestone: v0.2.3 → none
status: Fix Committed → Fix Released

Fix proposed to branch: master
Review: https://review.openstack.org/51594

Changed in keystone:
assignee: Morgan Fainberg (mdrnstm) → David Stanek (dstanek)
status: Confirmed → In Progress
Thierry Carrez (ttx) on 2013-10-17
Changed in ceilometer:
milestone: havana-1 → 2013.2
Thierry Carrez (ttx) on 2013-10-17
Changed in cinder:
milestone: havana-2 → 2013.2

Reviewed: https://review.openstack.org/51594
Committed: http://github.com/openstack/keystone/commit/59adb86b26394d95dca4e16ee6e099f8012310bf
Submitter: Jenkins
Branch: master

commit 59adb86b26394d95dca4e16ee6e099f8012310bf
Author: David Stanek <email address hidden>
Date: Thu Oct 10 13:36:03 2013 +0000

    Changes to testr as the test runner

    run_tests.sh has also been deprecated.

    Fixes-Bug: #1177924
    Change-Id: I15c1707eb6a62c74a0021a48b89ae15f03fcfea8

Changed in keystone:
status: In Progress → Fix Committed
Changed in trove:
status: New → Triaged
Thierry Carrez (ttx) on 2013-12-04
Changed in keystone:
milestone: none → icehouse-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-17
Changed in keystone:
milestone: icehouse-1 → 2014.1
Changed in trove:
importance: Undecided → Low
Erno Kuvaja (jokke) wrote :

This has been implemented in Glance change https://review.openstack.org/#/c/59699/

Changed in glance:
status: New → Fix Committed
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-10-16
Changed in glance:
milestone: juno-2 → 2014.2
Changed in horizon:
assignee: Kieran Spear (kspear) → Lin Hua Cheng (lin-hua-cheng)
Changed in django-openstack-auth:
assignee: nobody → Lin Hua Cheng (lin-hua-cheng)
Changed in horizon:
assignee: Lin Hua Cheng (lin-hua-cheng) → nobody
Changed in django-openstack-auth:
assignee: Lin Hua Cheng (lin-hua-cheng) → nobody
Changed in swift:
importance: Undecided → Wishlist
Changed in python-ceilometerclient:
status: Fix Committed → Fix Released
Changed in swift:
status: New → Triaged
Changed in python-cinderclient:
status: Fix Committed → Fix Released
David Lyle (david-lyle) on 2015-07-15
Changed in django-openstack-auth:
importance: Undecided → Wishlist
Changed in swift:
assignee: nobody → Richard Hawkins (richard-hawkins)

Reviewed: https://review.openstack.org/214206
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=9d7f71d5754c8b45f8e7c6ab80202de09933afb8
Submitter: Jenkins
Branch: master

commit 9d7f71d5754c8b45f8e7c6ab80202de09933afb8
Author: Richard Hawkins <email address hidden>
Date: Fri Aug 7 18:14:13 2015 -0500

    Modify functional tests to use ostestr/testr

    Defcore uses Tempest, which uses Test Repository.
    This change makes it easier for Defcore to pull functional
    tests from Swift and run them. Additionally, using testr
    allows tests to be run in parallel.

    Concurrency set to 1 for now, >1 causes failures for
    reasons that are still TBD.

    With switch to ostestr all the server logs are being sent to stdout
    which makes it completely unreadable. Suppressing the logs by default
    now with a flag to enable it if desired.

    Co-Authored-By: John Dickinson <email address hidden>
    Co-Authored-By: Robert Collins <email address hidden>
    Co-Authored-By: Matthew Oliver <email address hidden>
    Co-Authored-By: Ganesh Maharaj Mahalingam <email address hidden>

    Change-Id: I53ef4a116996a772cf1f3abc2eb0ad60047322d5
    Related-Bug: 1177924

Download full text (54.1 KiB)

Reviewed: https://review.openstack.org/264517
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=93ddaffaebb620bc712d52d0267194562dce02be
Submitter: Jenkins
Branch: feature/hummingbird

commit f53cf1043d078451c4b9957027bf3af378aa0166
Author: Ondřej Nový <email address hidden>
Date: Tue Jan 5 20:20:15 2016 +0100

    Fixed few misspellings in comments

    Change-Id: I8479c85cb8821c48b5da197cac37c80e5c1c7f05

commit 79222e327f9df6335b58e17a6c8dd0dc44b86c17
Author: ChangBo Guo(gcb) <email address hidden>
Date: Sat Dec 26 13:13:37 2015 +0800

    Fix AttributeError for LogAdapter

    LogAdapter object has no attribute 'warn' but has attribute
    'warning'.

    Closes-Bug: #1529321
    Change-Id: I0e0bd0a3dbc4bb5c1f0b343a8809e53491a1da5f

commit 684c4c04592278a280032002b5313b171ee7a4c0
Author: janonymous <email address hidden>
Date: Sun Aug 2 22:47:42 2015 +0530

    Python 3 deprecated the logger.warn method in favor of warning

    DeprecationWarning: The 'warn' method is deprecated, use 'warning'
    instead

    Change-Id: I35df44374c4521b1f06be7a96c0b873e8c3674d8

commit d0a026fcb8e8a9f5475699cc56e1998bdc4cd5ca
Author: Hisashi Osanai <email address hidden>
Date: Wed Dec 16 18:50:37 2015 +0900

    Fix duplication for headers in Access-Control-Expose-Headers

    There are following problems with Access-Control-Expose-Headers.

    * If headers in X-Container-Meta-Access-Control-Expose-Headers are
      configured, the headers are kept with case-sensitive string.
      Then a CORS request comes, the headers are merged into
      Access-Control-Expose-Headers as case-sensitive string even if
      there is a same header which is not case-sensitive string.

    * Access-Control-Expose-Headers is handled by a list.
      If X-Container/Object-Meta-XXX is configured in container/object
      and X-Container-Meta-Access-Control-Expose-Headers, same header
      is listed in Access-Control-Expose-Headers.

    This patch provides a fix for the problems.

    Change-Id: Ifc1c14eb3833ec6a851631cfc23008648463bd81

commit 0bcd7fd50ec0763dcb366dbf43a9696ca3806f15
Author: Bill Huber <email address hidden>
Date: Fri Nov 20 12:09:26 2015 -0600

    Update Erasure Coding Overview doc to remove Beta version

    The major functionality of EC has been released for Liberty and
    the beta version of the code has been removed since it is now
    in production.

    Change-Id: If60712045fb1af803093d6753fcd60434e637772

commit 84ba24a75640be4212e0f984c284faf4c894e7c6
Author: Alistair Coles <email address hidden>
Date: Fri Dec 18 11:24:34 2015 +0000

    Fix rst errors so that html docs are complete

    rst table format errors don't break the gate job
    but do cause sections of the documents to go missing
    from the html output.

    Change-Id: Ic8c9953c93d03dcdafd8f47b271d276c7b356dc3

commit 87f7e907ee412f5847f1f9ffca7a566fb148c6b1
Author: Matthew Oliver <email address hidden>
Date: Wed Dec 16 17:19:24 2015 +1100

    Pass HTTP_REFERER down to subrequests

    Currently a HTTP_REFERER (Referer) header isn't passed down to
    subreq...

tags: added: in-feature-hummingbird
Changed in trove:
assignee: nobody → Gábor Antal (gabor.antal)
Changed in horizon:
assignee: nobody → Gábor Antal (gabor.antal)
Download full text (30.2 KiB)

Reviewed: https://review.openstack.org/272201
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=f9b7fd3074b5b0e5d6ea879d4144f7bfeec5d46b
Submitter: Jenkins
Branch: feature/crypto

commit e13a03c379273ee10e678818078b9c40a96a7dc9
Author: Tim Burke <email address hidden>
Date: Wed Jan 20 16:06:26 2016 -0800

    Stop overriding builtin range

    Change-Id: I315f8b554bb9e96659b455f4158f074961bd6498

commit 0a404def7d54d1ef1c85c11a378052260c4fda4c
Author: John Dickinson <email address hidden>
Date: Wed Jan 20 15:19:35 2016 -0800

    remove unneeded duplicate dict keys

    Change-Id: I926d7aaa9df093418aaae54fe26e8f7bc8210645

commit 221f94fdd39fd2dcd9a2e5565adceab615d55913
Author: John Dickinson <email address hidden>
Date: Tue Jan 19 14:50:24 2016 -0800

    authors and changelog updates for 2.6.0

    Change-Id: Idd0ff9e70abc0773be183c37cd6125fe852da7c0

commit 58359269b0e971e52f0eb7f97221566ca2148014
Author: Samuel Merritt <email address hidden>
Date: Tue Dec 8 16:36:05 2015 -0800

    Fix memory/socket leak in proxy on truncated SLO/DLO GET

    When a client disconnected while consuming an SLO or DLO GET response,
    the proxy would leak a socket. This could be observed via strace as a
    socket that had shutdown() called on it, but was never closed. It
    could also be observed by counting entries in /proc/<pid>/fd, where
    <pid> is the pid of a proxy server worker process.

    This is due to a memory leak in SegmentedIterable. A SegmentedIterable
    has an 'app_iter' attribute, which is a generator. That generator
    references 'self' (the SegmentedIterable object). This creates a
    cyclic reference: the generator refers to the SegmentedIterable, and
    the SegmentedIterable refers to the generator.

    Python can normally handle cyclic garbage; reference counting won't
    reclaim it, but the garbage collector will. However, objects with
    finalizers will stop the garbage collector from collecting them* and
    the cycle of which they are part.

    For most objects, "has finalizer" is synonymous with "has a __del__
    method". However, a generator has a finalizer once it's started
    running and before it finishes: basically, while it has stack frames
    associated with it**.

    When a client disconnects mid-stream, we get a memory leak. We have
    our SegmentedIterable object (call it "si"), and its associated
    generator. si.app_iter is the generator, and the generator closes over
    si, so we have a cycle; and the generator has started but not yet
    finished, so the generator needs finalization; hence, the garbage
    collector won't ever clean it up.

    The socket leak comes in because the generator *also* refers to the
    request's WSGI environment, which contains wsgi.input, which
    ultimately refers to a _socket object from the standard
    library. Python's _socket objects only close their underlying file
    descriptor when their reference counts fall to 0***.

    This commit makes SegmentedIterable.close() call
    self.app_iter.close(), thereby unwinding its generator's stack and
    making it eligible for garbage collection.

    * in Python < 3...

tags: added: in-feature-crypto
Changed in trove:
assignee: Gábor Antal (gabor.antal) → nobody

Change abandoned by Alistair Coles (<email address hidden>) on branch: feature/crypto
Review: https://review.openstack.org/277950

John Dickinson (notmyname) wrote :

Brian, in Swift that's only the functests. Unit tests haven't been ported yet.

Changed in horizon:
assignee: Gábor Antal (gabor.antal) → nobody
Changed in manila-ui:
importance: Undecided → Wishlist
clayg (clay-gerrard) wrote :

fixed I53ef4a116996a772cf1f3abc2eb0ad60047322d5

Changed in swift:
status: Triaged → Confirmed
status: Confirmed → Fix Released
Akihiro Motoki (amotoki) wrote :

Horizon is one of projects which do not use testtools for testing. This is because we use Django test framework which depends on python standard unittest. testtools support in Django is out of scope of the horizon project and replacing horizon test from Django to testtools is not productive. we mark this bug as Won't Fix.

On the other hand, Django supports several test runners. Looking at the bug description, the motivation of switching to testtools is to run tests in parallel (and use subunit for output serialization though it is not explicitly mentioned). It is better to be explored separately.

Changed in horizon:
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers