Credential locks are not released
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tempest |
New
|
Undecided
|
Unassigned |
Bug Description
When running tempest tests with a test_accounts_file, it very quickly runs out of accounts, resulting in InvalidCredentials errors with "Insufficient number of users provided.":
```
setUpClass (murano_
-------
Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
File "/lib/python3.
raise value.with_
File "/lib/python3.
cls.
File "/lib/python3.
creds = cls.get_
File "/lib/python3.
creds = cls.credentials
File "/lib/python3.
net_creds = self._get_creds()
File "/lib/python3.
free_hash = self._get_
File "/lib/python3.
return f(*args, **kwargs)
File "/lib/python3.
raise lib_exc.
tempest.
Details: Insufficient number of users provided. TestRepositoryS
```
It appears that locks are not released, as the account lock files (names are hashes, content is test class name) in /tmp/tempest-lock/ remain there after a test run is complete.
This may be related to https:/
description: | updated |
I can reliably reproduce this failure to clean up credentials on my environment with this test:
``` tempest_ tests.tests. api.application _catalog. artifacts. test_repository .TestRepository Sanity. test_get_ list_packages[ id-2818aaa0- 6613-4bd9- 8abe-0271312135 7a,smoke]
murano_
```
Which I believe is found in git+https:/ /opendev. org/openstack/ murano- tempest- plugin. git@2.6. 0
In my environment (2023.2/stable sunbeam), this test is skipped with a message:
``` tempest_ tests.tests. api.application _catalog. artifacts. test_repository .TestRepository Sanity) ... SKIPPED: Murano is not using GLARE backend. Skipping GL
{3} setUpClass (murano_
```
Every time I run tempest with this test, I see another file left in the /tmp/tempest-lock directory, such that after 4 runs, the directory looks like this:
``` k8s-0:/ tmp/tempest- lock# grep -r . a41790ccb98b316 10:TestReposito rySanity c7bb7dff60b8bf5 65:TestReposito rySanity 0b13a1bfa6c4bf8 fa:TestReposito rySanity b3342b9af240152 8a:TestReposito rySanity
root@tempest-
533bd1b808f1c89
7b03fe17fc92437
e5604fdc3b6b820
22a805d5d9f003a
```
These files accumulate regardless of whether other tests run in the same tempest run or not (eg. running with --regex smoke or running with --regex 'murano_ tempest_ tests.. *artifacts. test_repository .TestRepository Sanity' ).
These lock files that aren't cleaned up accumulate, and eventually cause tempest to fail to allocate accounts for tests.