ocata undercloud job failing on tempest tests

[Alex Schultz]

The following tests appear to be broken in the stable/ocata tripleo-ci-centos-7-undercloud-oooq jobs.
tempest.api.identity.v2.test_tenants.IdentityTenantsTest
tempest.api.identity.v2.test_users.IdentityUsersTest

http://logs.openstack.org/20/570420/1/gate/tripleo-ci-centos-7-undercloud-oooq/5a109a4/logs/tempest.html.gz
http://logs.openstack.org/99/570199/1/gate/tripleo-ci-centos-7-undercloud-oooq/5ddd70a/logs/tempest.html.gz

ft1.1: tempest.api.identity.v2.test_tenants.IdentityTenantsTest.test_list_tenants_returns_only_authorized_tenants[id-ecae2459-243d-4ba1-ad02-65f15dc82b78]_StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
2018-05-25 21:56:09,917 10574 INFO [tempest.lib.common.rest_client] Request (IdentityTenantsTest:test_list_tenants_returns_only_authorized_tenants): 403 GET http://192.168.24.1:35357/v2.0/tenants 0.697s
2018-05-25 21:56:09,917 10574 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'status': '403', u'content-length': '132', 'content-location': 'http://192.168.24.1:35357/v2.0/tenants', u'vary': 'X-Auth-Token', u'server': 'Apache', u'connection': 'close', u'date': 'Fri, 25 May 2018 21:56:09 GMT', u'content-type': 'application/json', u'x-openstack-request-id': 'req-7bb82822-86ae-423b-9d61-dacafaec2aca'}
        Body: {"error": {"message": "You are not authorized to perform the requested action: admin_required.", "code": 403, "title": "Forbidden"}}
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tempest/api/identity/v2/test_tenants.py", line 28, in test_list_tenants_returns_only_authorized_tenants
    resp = self.non_admin_tenants_client.list_tenants()
  File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v2/tenants_client.py", line 70, in list_tenants
    resp, body = self.get(url)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 285, in get
    return self.request('GET', url, extra_headers, headers)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 659, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 760, in _error_checker
    raise exceptions.Forbidden(resp_body, resp=resp)
tempest.lib.exceptions.Forbidden: Forbidden
Details: {u'message': u'You are not authorized to perform the requested action: admin_required.', u'code': 403, u'title': u'Forbidden'}

ft2.1: tempest.api.identity.v2.test_users.IdentityUsersTest.test_user_update_own_password[id-165859c9-277f-4124-9479-a7d1627b0ca7]_StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
2018-05-25 21:56:22,414 10574 INFO [tempest.lib.common.rest_client] Request (IdentityUsersTest:test_user_update_own_password): 404 PATCH http://192.168.24.1:35357/v2.0/OS-KSCRUD/users/68139b23f91843fca22fce5475696649 0.666s
2018-05-25 21:56:22,415 10574 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"user": {"original_password": "Y2#GZ^ed~!fqo=r", "password": "E2%3JB&W91VB1YB"}}
    Response - Headers: {'status': '404', u'content-length': '93', 'content-location': 'http://192.168.24.1:35357/v2.0/OS-KSCRUD/users/68139b23f91843fca22fce5475696649', u'vary': 'X-Auth-Token', u'server': 'Apache', u'connection': 'close', u'date': 'Fri, 25 May 2018 21:56:21 GMT', u'content-type': 'application/json', u'x-openstack-request-id': 'req-5268dccf-f26a-4bf4-9fd7-119f5f5e29d7'}
        Body: {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}
2018-05-25 21:56:22,760 10574 INFO [tempest.lib.common.rest_client] Request (IdentityUsersTest:_run_cleanups): 404 PATCH http://192.168.24.1:35357/v2.0/OS-KSCRUD/users/68139b23f91843fca22fce5475696649 0.341s
2018-05-25 21:56:22,761 10574 DEBUG [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"user": {"original_password": "E2%3JB&W91VB1YB", "password": "Y2#GZ^ed~!fqo=r"}}
    Response - Headers: {'status': '404', u'content-length': '93', 'content-location': 'http://192.168.24.1:35357/v2.0/OS-KSCRUD/users/68139b23f91843fca22fce5475696649', u'vary': 'X-Auth-Token', u'server': 'Apache', u'connection': 'close', u'date': 'Fri, 25 May 2018 21:56:22 GMT', u'content-type': 'application/json', u'x-openstack-request-id': 'req-c4db3a62-745a-4289-adbd-a285ec7c1187'}
        Body: {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}
}}}

traceback-1: {{{
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tempest/api/identity/v2/test_users.py", line 68, in _restore_password
    user_id, original_password=new_pass, password=old_pass)
  File "/usr/lib/python2.7/site-packages/tempest/api/identity/v2/test_users.py", line 40, in _update_password
    user_id, password=password, original_password=original_password)
  File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v2/users_client.py", line 125, in update_user_own_password
    resp, body = self.patch('OS-KSCRUD/users/%s' % user_id, patch_body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 317, in patch
    return self.request('PATCH', url, extra_headers, headers, body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 659, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 765, in _error_checker
    raise exceptions.NotFound(resp_body, resp=resp)
tempest.lib.exceptions.NotFound: Object not found
Details: {u'message': u'The resource could not be found.', u'code': 404, u'title': u'Not Found'}
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tempest/api/identity/v2/test_users.py", line 92, in test_user_update_own_password
    user_id, original_password=old_pass, password=new_pass)
  File "/usr/lib/python2.7/site-packages/tempest/api/identity/v2/test_users.py", line 40, in _update_password
    user_id, password=password, original_password=original_password)
  File "/usr/lib/python2.7/site-packages/tempest/lib/services/identity/v2/users_client.py", line 125, in update_user_own_password
    resp, body = self.patch('OS-KSCRUD/users/%s' % user_id, patch_body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 317, in patch
    return self.request('PATCH', url, extra_headers, headers, body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 659, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 765, in _error_checker
    raise exceptions.NotFound(resp_body, resp=resp)
tempest.lib.exceptions.NotFound: Object not found
Details: {u'message': u'The resource could not be found.', u'code': 404, u'title': u'Not Found'}

[Sagi (Sergey) Shnaidman (sshnaidm)]

passes now http://logs.openstack.org/91/564291/17/check/tripleo-ci-centos-7-undercloud-oooq/bd9bfcb/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-07-16_08_54_58