ObjectACL tempest tests are flaky with preprov creds

[Andrea Frittoli]

Some of the tempest ObjectACL negative tests are flaky when executed using the pre provisioned credentials provider.

See for instance http://status.openstack.org/openstack-health/#/job/periodic-tempest-dsvm-neutron-full-test-accounts-master and http://status.openstack.org/openstack-health/#/job/periodic-tempest-dsvm-full-test-accounts-master.

Until this is fixed, the pre provisioned credentials provider cannot make it to the gate, and can only run in experimental and periodic jobs, which makes it susceptible of regressions.

[Andrea Frittoli (andrea-frittoli)]

All the flaky tests use the set_alt_auth_data functionality of auth.py, passing creds from an 'alt' user http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/object_storage/test_container_acl_negative.py#n90.
Tests which use set_alt_auth_data with no alt data (e.g. http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/object_storage/test_container_acl_negative.py#n58) do not suffer from the same issue.

I verified from the result DB that all the tests in the class run on the same worker, so there's no race possible within this class.

QUERY: select * from test_run_metadata, test_runs, tests where test_runs.id = test_run_metadata.test_run_id and test_runs.test_id = tests.id and run_id = 69874 and tests.test_id like '%ObjectACL%' and test_run_metadata.key = 'tags' and status = 'fail'.

[Andrea Frittoli (andrea-frittoli)]

AccountQuotasNegativeTest.test_user_modify_quota suffers from a similar issue as well

[Andrea Frittoli (andrea-frittoli)]

After the refactor of the test account generator tool, the issue have disappeared. While I don't understand what exactly was causing the original issue in the old account generator tool, I think it's safe to close this now.

[Andrea Frittoli (andrea-frittoli)]

Still working on this