Neutron Tempest Pagination test cases fail if run in parallel

[Sampat Ponnaganti]

Neutron tempest Pagination tet cases fail if run in parallel.

##### Issue
The pagination test cases see items created by other test cases and hence expected vs actual differ and the test cases fail.

##### Proposed solution:
1. Update pagination test cases to query neutron resource only for a specific project.

OR

2. Check the project ID in expected test data and use that Project ID to match the results in Actual. Ignore any other Project ID
Open to further discussion or any other solution.

For Eg:

###### These two test cases fail:

Test case 1:
 neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links[id-b4293e59-d794-4a93-be09-38667199ef68]

```code
Traceback
neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links[id-b4293e59-d794-4a93-be09-38667199ef68] Traceback: Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/test_trunk.py", line 353, in test_list_pagination_page_reverse_with_href_links self._test_list_pagination_page_reverse_with_href_links() File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1132, in inner return f(self, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1123, in inner return f(self, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1346, in _test_list_pagination_page_reverse_with_href_links self.assertSameOrder(expected_resources, reversed(resources)) File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1160, in assertSameOrder self.assertEqual(len(original), len(actual)) File "/usr/local/lib/python3.6/dist-packages/testtools/testcase.py", line 411, in assertEqual self.assertThat(observed, matcher, message) File "/usr/local/lib/python3.6/dist-packages/testtools/testcase.py", line 498, in assertThat raise mismatch_error testtools.matchers._impl.MismatchError: 5 != 6
```

Test case 2: neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_href_links[id-dcd02a7a-f07e-4d5e-b0ca-b58e48927a9b]

```code
Traceback (most recent call last):
 File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/test_trunk.py", line 337, in test_list_pagination_with_href_links
 self._test_list_pagination_with_href_links()
 File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1132, in inner
 return f(self, *args, **kwargs)
 File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1123, in inner
 return f(self, *args, **kwargs)
 File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1312, in _test_list_pagination_with_href_links
 self._test_list_pagination_iteratively(self._list_all_with_hrefs)
 File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1241, in _test_list_pagination_iteratively
 len(expected_resources), sort_args
 File "/usr/local/lib/python3.6/dist-packages/neutron_tempest_plugin/api/base.py", line 1302, in _list_all_with_hrefs
 self.assertEqual(1, len(resources_))
 File "/usr/local/lib/python3.6/dist-packages/testtools/testcase.py", line 411, in assertEqual
 self.assertThat(observed, matcher, message)
 File "/usr/local/lib/python3.6/dist-packages/testtools/testcase.py", line 498, in assertThat
 raise mismatch_error
testtools.matchers._impl.MismatchError: 1 != 0
```

###### Reason for failure:
More neutron trunk ports are being returned than expected.

Code has to be fixed for 2 and 3 below such that trunks are returned only for one project (specify project_id in GET trunks call). A parallel test might be creating trunks so this test case is failing.

2. test_list_pagination_page_reverse_with_href_links
     -> Expected returns trunks for project: 864acee2d6c64faa8750cfe53437a158
     -> Actual paginated returns trunk for more projects: 89e63227c3b6405498f8fb1973cd055d and 864acee2d6c64faa8750cfe53437a158

3. test_list_pagination_with_href_links
     Same issue as in 2.

###### Able to run the tests if run in serial mode: concurrency = 1
{0} neutron_tempest_plugin.api.test_networks.NetworksSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links [3.404399s] ... ok{0}
neutron_tempest_plugin.api.test_networks.NetworksSearchCriteriaTest.test_list_pagination_with_href_links [9.412321s] ... ok

{0} neutron_tempest_plugin.api.test_networks.NetworksSearchCriteriaTest.test_list_pagination_with_marker [4.566547s] ... ok{0}
neutron_tempest_plugin.api.test_ports.PortsSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links [4.636547s] ... ok

{0} neutron_tempest_plugin.api.test_subnets.SubnetsSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links [1.971421s] ... ok{0}
neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links [0.881594s] ... ok

{0} neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_href_links [2.173380s] ... ok{0}
neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_asc [0.180619s] ... ok

{0}
neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_desc [0.176105s] ... ok

======
Totals
======
Ran: 9 tests in 107.7512 sec.

Passed: 9
Skipped: 0
Expected Fail: 0
Unexpected Success: 0
Failed: 0
Sum of execute time for each test: 27.4029 sec.
==============
Worker Balance
==============

Worker 0 (9 tests) => 0:01:21.001099

[Martin Kopec (mkopec)]

I'm changing the project to neutron as it seems the problem is within neutron tempest plugin, not tempest.

[Sampat Ponnaganti (sp810x)]

@Martin, Thanks.

@Neutron-tempest-plugin team, could you comment on this bug if this is a valid bug? I'll submit a patchset.

[Lajos Katona (lajos-katona)]

I am not sure that using project_id would help as (at least as I know) we have 2 projects: admin & tenant, so possibly other resources (trunks in this case) will have the same project_id.
What about changing names to be more specific for the given testclass:
https://opendev.org/openstack/neutron-tempest-plugin/src/branch/master/neutron_tempest_plugin/api/test_trunk.py#L317
If I understand the names are in a list here:
https://opendev.org/openstack/neutron-tempest-plugin/src/branch/master/neutron_tempest_plugin/api/base.py#L1203

so when in test_trunk.py the names are used in the loop to create trunks, just add __name__ to the fetched name chunks:
for name in cls.resource_names:
    ....
    cls.create_trunk(parent_port, name='%s%s' % (name, cls.__name__))

[Sampat Ponnaganti (sp810x)]

Thanks, I'll change the trunk names to be more specific to the test case and check if that fixes the issue.