parallelize unit tests

[Aaron Rosen]

It would be nice if we also did this for quantum as well to speed up the unit tests.

Hello,

The change (https://review.openstack.org/15078) to make testr Nova's
test runner merged in the last hour or so. What does this mean?

* Nova's unittests will now be run in X parallel processes where X is
  the number of CPUs available on your machine (which results in much
  faster testing).
* Parallel testing means you cannot count on your tests being run in
  the same order each time they are run or in the same process. Tests
  should be independent units.
  * Make sure you subclass nova.test.TestCase for your test classes to
    pull in the proper base setUp and tearDown stuff.
  * Use fixtures.Fixture to build robust fixtures that properly set
    and reset state before and after each test.
* Virtual envs will need to be rebuilt as dependencies have changed.
* Each test run is logged in subunit under .testrepository/$TEST_RUN_NUMBER.
  * Every test run during the test run will be captured in this log.
  * For each test in this log you will find the captured stdout,
    stderr, and logging for when that test was run (this content is also
    echoed to stdout when a test fails).
* Running tests shouldn't change too much. tox -epy2X will still work
  and most of the run_tests.sh functionality was preserved.
  * For more info on running tests see http://wiki.openstack.org/testr.
* The logs provided by Jenkins will include the console log and an
  html page with test details (that does not change). You will also find
  the subunit log for that test run.

This is a semi large change and there are differences between nose and
testr, non python2.6 test facilities like setUpClass will no longer be
available under python2.6 for example. But most of these changes
should be positive. Tests run faster, each test comes with very
verbose logs, tests end up being more robust as testr shouldn't have
the zero tests run but pass anyways problem. Additionally, because
tests are run in different orders and in parallel, testr will often
catch things like missing imports or conflicts between test cases (in
fact most of the work involved in making this happen was in fixing
test bugs).

It would be great if other projects used
https://review.openstack.org/15078 as a template for switching over to
testr as well. Let me know if you have questions.

Thanks,
Clark

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/24152

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/24152
Committed: http://github.com/openstack/quantum/commit/147038a61386fb2c94459499637341978032cc30
Submitter: Jenkins
Branch: master

commit 147038a61386fb2c94459499637341978032cc30
Author: Tim Miller <email address hidden>
Date: Sun Mar 10 15:23:46 2013 -0700

    Parallelize quantum unit testing:

    This change switches run_tests.sh to use testr
    in parallel mode, instead of nosetests, in order
    to enable unit tests to run in parallel.

    By default, the number of parallel test processes
    is set to the number of cores on the test machine.

    A similar change was recently merged to nova:
      https://review.openstack.org/#/c/15078/

    Some tests required updating to remove inter-testcase
    dependencies.

    bug #1099694

    Change-Id: Idfb923d424342a07dcba88d70b971683f549f763