[tempest] Openstack resources are not deleted post tempest cleanup
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Shared File Systems Service (Manila) |
Invalid
|
Undecided
|
Martin Kopec | ||
tempest |
Fix Released
|
Medium
|
Martin Kopec |
Bug Description
Tempest cleanup utility does not always delete and destroy used OpenStack resources.
**Reproduce Steps:
$ tempest cleanup --init-saved-state
$ execute selected manila upstream api tests
$ tempest cleanup
**User impact: Tests may fail due to insufficient resources.
**Failure example:
setUpClass (manila_
-------
Captured traceback:
~~~~~~~~~~~~~~~~~~~
b'Traceback (most recent call last):'
b' File "/usr/lib/
b' six.reraise(etype, value, trace)'
b' File "/usr/lib/
b' raise value'
b' File "/usr/lib/
b' cls.setup_
b' File "/usr/lib/
b' credential_
b' File "/usr/lib/
b' creds = getattr(
b' File "/usr/lib/
b" return self.get_
b' File "/usr/lib/
b' credentials.
b' File "/usr/lib/
b' router = self._create_
b' File "/usr/lib/
b' resp_body = self.routers_
b' File "/usr/lib/
b' return self.create_
b' File "/usr/lib/
b' resp, body = self.post(req_uri, req_post_data)'
b' File "/usr/lib/
b" return self.request(
b' File "/usr/lib/
b' self._error_
b' File "/usr/lib/
b' raise exceptions.
b'tempest.
b"Details: {'type': 'IpAddressGener
b''
**Reference: tempest.conf & saved_state.json files are attached
**Screen captures:
(overcloud) [stack@undercloud-0 testing_dir]$ ll saved_state.json
-rw-rw-r--. 1 stack stack 2675 May 23 17:56 saved_state.json
(overcloud) [stack@undercloud-0 testing_dir]$ openstack network list
+------
| ID | Name | Subnets |
+------
| 03e522a1-
| 0b53019a-
| 0fab7ada-
| 110d202f-
| 166b710f-
| 249d773b-
...
(overcloud) [stack@undercloud-0 testing_dir]$ tempest cleanup
Begin cleanup
Process 0 projects
(overcloud) [stack@undercloud-0 testing_dir]$ openstack network list
+------
| ID | Name | Subnets |
+------
| 03e522a1-
| 0b53019a-
| 0fab7ada-
| 110d202f-
| 166b710f-
| 249d773b-
...
Changed in manila: | |
assignee: | nobody → Martin Kopec (mkopec) |
Changed in tempest: | |
status: | Triaged → In Progress |
Tempest cleanup works with two types of services, project ones [1] and global ones [2]. First cleanup iterates over the projects which are not in saved_state.json and removes all resources related to that projects. Then it iterates over the global services and removes them.
If a resource is associated with a project which has been already removed and the resource is from project services category, it won't be cleaned.
I tried to run some of the manila tests and after cleanup I ended up with 50 networks which were associated with already non-existing projects and therefore they weren't cleaned. I guess this could happen if a test case removes a project but doesn't remove the resources associated with the project first. I'm gonna have a look how it can be solved.
[1] https:/ /opendev. org/openstack/ tempest/ src/commit/ 77febec7848d2d0 31954619b6ce442 c9723d0043/ tempest/ cmd/cleanup_ service. py#L961 /opendev. org/openstack/ tempest/ src/commit/ 77febec7848d2d0 31954619b6ce442 c9723d0043/ tempest/ cmd/cleanup_ service. py#L988
[2] https:/