Functional NetlinkLib tests can interfere with other tests

[Slawek Kaplonski]

It may happen that tests from neutron.tests.functional.agent.linux.test_netlink_lib.NetlinkLibTestCase can interfere with e.g. tests from neutron.tests.functional.agent.test_firewall module and fail because of that.
It may happen that one of NetlinkLib tests deletes some conntrack entries and expects ONLY some specific ones to be still in conntrack. But there are also other entries added by some other tests.

Example of such failure: http://logs.openstack.org/02/637802/2/check/neutron-functional-python27/3d611ea/logs/testr_results.html.gz

Additional conntrack entry (6, u'udp', 46935, 18650, u'2001:db8:aaaa::1', u'2001:db8:aaaa::2', 71) comes probably from one of tests in neutron.tests.functional.agent.test_firewall module.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/638635

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/638635
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=798c6c731fc5872ade8b5eb1f571a2c002c8c0fb
Submitter: Zuul
Branch: master

commit 798c6c731fc5872ade8b5eb1f571a2c002c8c0fb
Author: Slawek Kaplonski <email address hidden>
Date: Fri Feb 22 12:34:54 2019 +0100

    [Functional tests] Change way how conntrack entries are checked

    In netlink_lib functional tests module there are listed conntrack
    entries and those entries are assert to some expected list.
    It may happen that sometimes some additional entries from other
    tests will also be in the list and that cause failures of
    netlink_lib tests.

    So this patch changes way how those assertions are done. For now
    it will check if each of expected entries is in entries list and
    in case of delete entries tests, it will also check if any of
    deleted entries isn't actually in list.

    Change-Id: I30c18f141a8356b060902e6493ba0657b21619ad
    Closes-Bug: #1817295

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/649044

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/649045

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/649046

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/649045
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=19912a30bd4d28dd30b54083eaa136953cc5a174
Submitter: Zuul
Branch: stable/queens

commit 19912a30bd4d28dd30b54083eaa136953cc5a174
Author: Slawek Kaplonski <email address hidden>
Date: Fri Feb 22 12:34:54 2019 +0100

    [Functional tests] Change way how conntrack entries are checked

    In netlink_lib functional tests module there are listed conntrack
    entries and those entries are assert to some expected list.
    It may happen that sometimes some additional entries from other
    tests will also be in the list and that cause failures of
    netlink_lib tests.

    So this patch changes way how those assertions are done. For now
    it will check if each of expected entries is in entries list and
    in case of delete entries tests, it will also check if any of
    deleted entries isn't actually in list.

    Change-Id: I30c18f141a8356b060902e6493ba0657b21619ad
    Closes-Bug: #1817295
    (cherry picked from commit 798c6c731fc5872ade8b5eb1f571a2c002c8c0fb)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/649046
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ad1d4358ec4e885d4e46c56cfa298f24b9f3f0a2
Submitter: Zuul
Branch: stable/pike

commit ad1d4358ec4e885d4e46c56cfa298f24b9f3f0a2
Author: Slawek Kaplonski <email address hidden>
Date: Fri Feb 22 12:34:54 2019 +0100

    [Functional tests] Change way how conntrack entries are checked

    In netlink_lib functional tests module there are listed conntrack
    entries and those entries are assert to some expected list.
    It may happen that sometimes some additional entries from other
    tests will also be in the list and that cause failures of
    netlink_lib tests.

    So this patch changes way how those assertions are done. For now
    it will check if each of expected entries is in entries list and
    in case of delete entries tests, it will also check if any of
    deleted entries isn't actually in list.

    Change-Id: I30c18f141a8356b060902e6493ba0657b21619ad
    Closes-Bug: #1817295
    (cherry picked from commit 798c6c731fc5872ade8b5eb1f571a2c002c8c0fb)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/649044
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=289f66bd7424d2cbfc4d3caa8ef7369c2f18b177
Submitter: Zuul
Branch: stable/rocky

commit 289f66bd7424d2cbfc4d3caa8ef7369c2f18b177
Author: Slawek Kaplonski <email address hidden>
Date: Fri Feb 22 12:34:54 2019 +0100

    [Functional tests] Change way how conntrack entries are checked

    In netlink_lib functional tests module there are listed conntrack
    entries and those entries are assert to some expected list.
    It may happen that sometimes some additional entries from other
    tests will also be in the list and that cause failures of
    netlink_lib tests.

    So this patch changes way how those assertions are done. For now
    it will check if each of expected entries is in entries list and
    in case of delete entries tests, it will also check if any of
    deleted entries isn't actually in list.

    Change-Id: I30c18f141a8356b060902e6493ba0657b21619ad
    Closes-Bug: #1817295
    (cherry picked from commit 798c6c731fc5872ade8b5eb1f571a2c002c8c0fb)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.7 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.0.6 release.