[OSTF] Wrong discovering tests

[Yaroslav Lobankov]

FUEL RELEASE - 5.1
DEPLOYMENT MODE - multi-node

If name of a test file has combination of letters "ha", all tests in this file will be added to tests for HA mode.

If name of a test class in a test file has combination of letters "ha", all tests in this class will be added to tests for HA mode.

If name of a test in a test class has combination of letters "ha", this test will be added to tests for HA mode.

For example:
There is test file "test_sanity_sahara.py" in fuel_health/tests/sanity folder. All tests in this test file are discovered by nose as tests for sanity and ha.

From ostf.log:
2014-08-21 12:34:03 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_sahara.SanitySaharaTests.test_sanity_sahara added for sanity
2014-08-21 12:34:03 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_sahara.SanitySaharaTests.test_sanity_sahara added for ha

As the result of this I can't see the sanity test for Sahara in the "Health Check" tab on Web UI in the group "Sanity tests".

If I rename test file "test_sanity_murano.py" into "test_sanity_hamurano.py", all tests in this test file will be discovered by nose as tests for sanity and ha.

From ostf.log:
2014-08-21 13:31:29 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_hamurano.MuranoSanityTests.test_create_and_delete_service added for sanity
2014-08-21 13:31:29 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_hamurano.MuranoSanityTests.test_create_and_delete_service added for ha
2014-08-21 13:31:29 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_hamurano.MuranoSanityTests.test_get_list_categories added for sanity
2014-08-21 13:31:29 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_hamurano.MuranoSanityTests.test_get_list_categories added for ha
2014-08-21 13:31:29 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_hamurano.MuranoSanityTests.test_get_list_packages added for sanity
2014-08-21 13:31:29 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_hamurano.MuranoSanityTests.test_get_list_packages added for ha

If I rename "test_sanity_sahara.py" into "test_sanity_foo.py", class "SanitySaharaTests" into "SanityFooTests" and test "test_sanity_sahara" into "test_sanity_foo", everything will be OK.

From ostf.log:
2014-08-21 14:21:51 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_compute.SanityComputeTest.test_list_volumes added for sanity
2014-08-21 14:21:51 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_foo.SanityFooTests.test_sanity_foo added for sanity
2014-08-21 14:21:51 INFO (nose_discovery) fuel_health.tests.sanity.test_sanity_heat.SanityHeatTest.test_list_stacks added for sanity

After renaming the test file (as well as the class and the test) I can see the sanity test for Sahara in the "Health Check" tab on Web UI in the group "Sanity tests".

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/146870

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/146870
Committed: https://git.openstack.org/cgit/stackforge/fuel-ostf/commit/?id=70ecf3b93c406d3c77bb73d8dcbda5faf8a7fbf0
Submitter: Jenkins
Branch: master

commit 70ecf3b93c406d3c77bb73d8dcbda5faf8a7fbf0
Author: Sebastian Kalinowski <email address hidden>
Date: Tue Jan 13 14:51:57 2015 +0100

    Use regex to check if test belong to test set

    When discovering available tests regular expression will
    be used to check to which test set given test belongs.
    This prevent situation when 'test_sanity_sahara' was
    assigned to 'ha' test set because of occurrence of this words in
    'sahara'

    Change-Id: If7366da29da17404af3564e031e47e7ab39ba826
    Closes-Bug: #1359728