Manila tempest tests on access-rules intermittently fail in the following way:
manila_tempest_tests.tests.api.test_rules.ShareIpRulesForNFSTest.test_create_delete_access_rule_with_cidr_6___2_38___6_[backend,positive]_StringException: pythonlogging:'': {{{
2019-02-14 18:11:56,686 29279 INFO [tempest.lib.common.rest_client] Request (ShareIpRulesForNFSTest:test_create_delete_access_rule_with_cidr): 200 POST http://158.69.70.18:8786/v2/a9073e7e64fa4dd082acddf3caaedcad/shares/cfd3cd7b-daa7-4aeb-bc0e-1051be13bccf/action 4.041s
2019-02-14 18:11:56,687 29279 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'x-openstack-manila-api-version': '2.38', 'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
Body: {"allow_access": {"access_level": null, "access_type": "ip", "access_to": "2001:db8:aec0:9:ee9e:5980::/89"}}
Response - Headers: {'status': '200', u'content-length': '305', 'content-location': 'http://158.69.70.18:8786/v2/a9073e7e64fa4dd082acddf3caaedcad/shares/cfd3cd7b-daa7-4aeb-bc0e-1051be13bccf/action', u'x-compute-request-id': 'req-bff8e22b-2a8f-449d-8dff-1b397a59255c', u'vary': 'X-OpenStack-Manila-API-Version', u'x-openstack-manila-api-version': '2.38', u'connection': 'close', u'date': 'Thu, 14 Feb 2019 18:11:56 GMT', u'content-type': 'application/json'}
Body: {"access": {"access_key": null, "share_id": "cfd3cd7b-daa7-4aeb-bc0e-1051be13bccf", "created_at": "2019-02-14T18:11:53.000000", "updated_at": null, "access_type": "ip", "access_to": "2001:db8:aec0:9:ee9e:5980::/89", "access_level": "rw", "state": "applying", "id": "0bae0d58-0507-4159-a8dc-0e48f5c9af29"}}
}}}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ddt.py", line 139, in wrapper
return func(self, *args, **kwargs)
File "/opt/stack/new/manila-tempest-plugin/manila_tempest_tests/tests/api/test_rules.py", line 182, in test_create_delete_access_rule_with_cidr
self.assertEqual("queued_to_apply", rule['state'])
File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 411, in assertEqual
self.assertThat(observed, matcher, message)
File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 498, in assertThat
raise mismatch_error
testtools.matchers._impl.MismatchError: 'queued_to_apply' != u'applying'
gouthamr remarked: "this suggests a race condition where the access rule is probably already being applied, but the test must account for this behavior - it can happen rarely when a user tries to add a rule right after another - and the act of applying prior rules will pick up the new rule immediately. It isn't necessarily harmful"
IOW the fix may be to our tempest tests rather than to the code under test if I understand him correctly.