OSTF results are processed incorrectly after second and further runs

Bug #1468682 reported by Dennis Dmitriev on 2015-06-25
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
High
Dennis Dmitriev
6.1.x
High
Dennis Dmitriev
7.0.x
High
Dennis Dmitriev

Bug Description

Reproduced on CI , test 'ceph_ha_restart' : http://jenkins-product.srt.mirantis.net:8080/view/7.0_swarm/job/7.0.system_test.centos.thread_3/4/

Results of OSTF tests are not cleared, so if there is any failed test that must be failed for this particular test case (for example, in failover tests), then any other OSTF runs of another tests will also return this failed result.

From the CI job '7.0.system_test.centos.thread_3':
============
2015-06-16 02:06:34,070 - INFO fuel_web_client.py:135 -- Waiting 1200 sec. for passed OSTF HA tests.
...
AssertionError: Failed 1 OSTF tests; should fail 0 tests. Names of failed tests: [{u'Check that required services are running (failure)': u'Some nova services have not been started.. Please refer to OpenStack logs for more details.'}]
============

Test 'Check that required services are running' is in 'sanity' group that was run before 'ha' group with expected failed test.

OSTF test results should be filtered regarding running tests.

Dennis Dmitriev (ddmitriev) wrote :

Reviewed: https://review.openstack.org/198326
Committed: https://git.openstack.org/cgit/stackforge/fuel-qa/commit/?id=66719a8bc8efe757e071040b06c987af14c3ca47
Submitter: Jenkins
Branch: stable/6.1

commit 66719a8bc8efe757e071040b06c987af14c3ca47
Author: Dennis Dmitriev <email address hidden>
Date: Thu Jun 25 15:01:54 2015 +0300

    Skip OSTF results that are not belong to running test_sets

    To check just necessary test sets, results for other test sets
    should be skipped.

    Change-Id: I366d289664c9a50ede9359bda95a09f33e58c61e
    Closes-Bug:#1468682

Changed in fuel:
status: In Progress → Fix Committed
Igor Shishkin (teran) wrote :

Moving to 7.0 since 6.1 is already released.

Igor Shishkin (teran) wrote :

Looks like not to 7.0 but 6.1-updates

tags: added: non-release
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments