[RFE] replace testrepository with stestr

[milan k]

Stestr[1] was forked from testr to fix some long outstanding bugs.
See Migration instructions[2]

Outcome of the Pike PTG QA sessions[3][4]

[1] https://github.com/mtreinish/stestr/
[2] https://github.com/mtreinish/stestr#migrating-from-testrepository
[3] https://etherpad.openstack.org/p/qa-ptg-pike
[4] https://etherpad.openstack.org/p/qa-ptg-pike-test-runners

[Vladyslav Drok (vdrok)]

Setting the status to triaged, as it seems that the docs already have the migration instructions.

[Pavlo Shchelokovskyy (pshchelo)]

the title of the bug is wrong - in fact we have to replace 'testrepository' with 'stestr' (which is a kind of a slimmed down fork more suitable for openstack projects)

ostestr is still the test runner/wrapper to use. ostestr < 1.0 was using testrepository as underlying test runner, but starting with ostestr 1.0.0 (already in g-r) it uses stestr instead. As a result, some small changes have to be made to projects to migrate unit test infrastructure to stestr:

- replace .testr.conf with .stestr.conf (script to migrate is included in stestr repo)
- coverage jobs might need to be rewritten a bit (as stestr has no coverage/setuptools integration similar to what testrepository has)

[Pavlo Shchelokovskyy (pshchelo)]

I believe the ironic part is fixed by this (already merged) patch

https://review.openstack.org/#/c/505165/

Setting to "fix released" for ironic.

[Pavlo Shchelokovskyy (pshchelo)]

ironic-inspector is a bit different

- it does not use the testrepository/ostestr etc in the first place, using Python built-in 'unittest' test runner instead
- it actually runs coverage for both py27 and py35 unit tests jobs

Migrating to ostestr has some clear benefits, like parallel test execution and better results filtering/reporting.

As part of the migration I propose to stop running coverage on py* jobs and instead add a proper coverage gate job. In order to maintain test behavior/rigorousness for stable branches, some changes to tox.ini will have to be backported there (or block the new coverage job from running on current stable releases..).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/508122

[Ruby Loo (rloo)]

I suppose we should approve this rfe then? :)

[Ruby Loo (rloo)]

Sam and I are good with it, so approving. Thanks for reporting and fixing! :)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/508122
Committed: https://git.openstack.org/cgit/openstack/ironic-inspector/commit/?id=7625ed68c8f8f946323b129ddd116e0261668a75
Submitter: Jenkins
Branch: master

commit 7625ed68c8f8f946323b129ddd116e0261668a75
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Thu Sep 28 10:43:07 2017 +0000

    Use ostestr unit test runner

    This patch moves ironic-inspector to use 'stestr' test runner
    via 'ostestr' wrapper.

    Effective changes:

    - unit tests are now being run in parallel by default
    - 'py*' jobs are not running code coverage measurements,
      which is left for 'cover' job only
    - 'cover' job now also produces html reports consumable by upstream CI,
      and fails under 90% of code coverage
    - users can leverage all the ostestr/stestr features

    Change-Id: I75f539457dbb1d811fdee69de52d1bcee4c6ebe6
    Depends-On: Ica838db23b5714c2da1e4313e1a7ece0ca241d44
    Closes-Bug: #1666232

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ironic-inspector 6.1.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/518015

[OpenStack Infra (hudson-openstack)]

Change abandoned by Milan Kováčik (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/518015
Reason: ^^

[OpenStack Infra (hudson-openstack)]

Change abandoned by Milan Kováčik (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/518015
Reason: this isn't needed anymore