OSTF results are processed incorrectly after second and further runs

[Dennis Dmitriev]

Reproduced on CI , test 'ceph_ha_restart' : http://jenkins-product.srt.mirantis.net:8080/view/7.0_swarm/job/7.0.system_test.centos.thread_3/4/

Results of OSTF tests are not cleared, so if there is any failed test that must be failed for this particular test case (for example, in failover tests), then any other OSTF runs of another tests will also return this failed result.

From the CI job '7.0.system_test.centos.thread_3':
============
2015-06-16 02:06:34,070 - INFO fuel_web_client.py:135 -- Waiting 1200 sec. for passed OSTF HA tests.
...
AssertionError: Failed 1 OSTF tests; should fail 0 tests. Names of failed tests: [{u'Check that required services are running (failure)': u'Some nova services have not been started.. Please refer to OpenStack logs for more details.'}]
============

Test 'Check that required services are running' is in 'sanity' group that was run before 'ha' group with expected failed test.

OSTF test results should be filtered regarding running tests.

[Dennis Dmitriev (ddmitriev)]

For 7.0: https://review.openstack.org/#/c/195543/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/6.1
Review: https://review.openstack.org/198326

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/198326
Committed: https://git.openstack.org/cgit/stackforge/fuel-qa/commit/?id=66719a8bc8efe757e071040b06c987af14c3ca47
Submitter: Jenkins
Branch: stable/6.1

commit 66719a8bc8efe757e071040b06c987af14c3ca47
Author: Dennis Dmitriev <email address hidden>
Date: Thu Jun 25 15:01:54 2015 +0300

    Skip OSTF results that are not belong to running test_sets

    To check just necessary test sets, results for other test sets
    should be skipped.

    Change-Id: I366d289664c9a50ede9359bda95a09f33e58c61e
    Closes-Bug:#1468682

[Igor Shishkin (teran)]

Moving to 7.0 since 6.1 is already released.

[Igor Shishkin (teran)]

Looks like not to 7.0 but 6.1-updates