Race conditions in neutron_tempest_plugin/scenario/test_security_groups.py

Bug #1801306 reported by Federico Ressi on 2018-11-02
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
High
Unassigned

Bug Description

I found a couple of race conditions in neutron-tempest-plugin affecting recular CI build when executing test jobs with neutron-tempest-plugin test suite. These race conditions are mostly because two test case wrongly modify default security group causing random failures in other scenario test cases or also failing themself.

1) Test case test_default_sec_grp_scenarios shouldn't change default security group rule at below lines:
https://github.com/openstack/neutron-tempest-plugin/blob/cf38b77328dbf94f1323f96f68aa77124a6f4a7b/neutron_tempest_plugin/scenario/test_security_groups.py#L109-L112
This could caouse other tests to fail. Modifying default resources should be forbidden. There is no difference between default security group and another group, therefore the test makes no sense at all. Tests that do some changes on resources should create its own target resources.

2) Test case test_protocol_number_rule list for security groups and takes the first one, then it changes it at line
https://github.com/openstack/neutron-tempest-plugin/blob/cf38b77328dbf94f1323f96f68aa77124a6f4a7b/neutron_tempest_plugin/scenario/test_security_groups.py#L147-L149
The problem here is that the first security group that is taken could be the default security group instead of the one created by the test. This would cause a failure in the same test and even worst in another test later.

Changed in neutron:
assignee: nobody → Federico Ressi (fressi-redhat)
Changed in neutron:
status: New → In Progress
Federico Ressi (fressi-redhat) wrote :

I started fixing this problem here: https://review.openstack.org/#/c/612733/5

zhaobo (zhaobo6) on 2018-11-05
Changed in neutron:
importance: Undecided → High
tags: added: tempest
Federico Ressi (fressi-redhat) wrote :

There are two patches in progress to fix this bug:

- https://review.openstack.org/#/c/612733 - to cleanup CI status
- https://review.openstack.org/#/c/615813 - complete test case refactory to improve code quality and readybility

Reviewed: https://review.openstack.org/612733
Committed: https://git.openstack.org/cgit/openstack/neutron-tempest-plugin/commit/?id=6f9bb77d9872a1cac4e1815ce13d97b8a143b3a8
Submitter: Zuul
Branch: master

commit 6f9bb77d9872a1cac4e1815ce13d97b8a143b3a8
Author: Federico Ressi <email address hidden>
Date: Tue Oct 23 17:35:47 2018 +0200

    Fix race condition in test_security_groups

    Listing security groups and taking the first one to change
    rules doesn't give any warranty listed security group is
    exactly the one used to create servers ports.

    This avoids this problem by creating a security group and
    passing it to server instead of having to search for it
    after the server has been created.

    Despite a test cases require a general refactory this should
    at least fix the CI job intermittent failures related to
    this test.

    Partial-Bug: #1801306
    Change-Id: I5ecb3c8711e7455e7f1588943a2ffd08ce154578

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

Changed in neutron:
assignee: Federico Ressi (fressi-redhat) → nobody
status: In Progress → New
tags: added: timeout-abandon

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.openstack.org/615813
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers