Comment 0 for bug 1626046

 fuel fuel-version
api: '1'
auth_required: true
feature_groups:
- experimental
- advanced
openstack_version: mitaka-9.0
release: '9.0'

Description:
With keystone configured for federated access (SAML) Sahara cannot create trusts due to 'unable to find role'

Steps to reproduce:
Configure keystone for federation such as SAML or OID

Expected Results:
federated users are ephemeral but consume a role (_member_) based on group membership, role based access should work.

actual results:
Sahara - RESP BODY: {"error": {"message": "Could not find role: 9fe2ff9ee4384b1894a90878d3e92bab", "code": 404, "title": "Not Found"}}

and finally:
Unable to create trust (reason: Could not find role: 9fe2ff9ee4384b1894a90878d3e92bab (HTTP 404)

 openstack role list (doesn't seem to make much sense Sahara can't find the _member_ role)
+----------------------------------+-----------------+
| ID | Name |
+----------------------------------+-----------------+
| 79fedf162a664cdbb6de7117d4998566 | ResellerAdmin |
| 87cf7f569672416db4027839aaa93eec | heat_stack_user |
| 897d116732174ee8be888aba12b7a550 | admin |
| 9fe2ff9ee4384b1894a90878d3e92bab | _member_ |
+----------------------------------+-----------------+

this goes for Murano and heat also - the same result - the federated users role cannot be found.

Sahara snippet:

 DEBUG keystoneclient.auth.identity.v3.base [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Making authentication request to http://172.25.60.5:5000/v3/auth/tokens get_auth_ref /usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py:188
2016-09-20 17:02:47.302 27869 DEBUG keystoneclient.auth.identity.v3.base [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Making authentication request to http://172.25.60.5:5000/v3/auth/tokens get_auth_ref /usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/v3/base.py:188
2016-09-20 17:02:47.412 27869 DEBUG keystoneclient.session [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] REQ: curl -g -i --insecure -X POST http://172.25.60.5:35357/v3/OS-TRUST/trusts -H "User-Agent: python-keystoneclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}cbd44946362e5abd7f0a4bd25512a85a178cc87c" -d '{"trust": {"impersonation": true, "trustor_user_id": "43951d0af69848268f07937c12fa36fd", "allow_redelegation": true, "roles": [{"name": "_member_"}], "trustee_user_id": "d29432ca2fdf4b0996b01347155934df", "project_id": "307a375b43274f779d89b0512824a054"}}' _http_log_request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:206
2016-09-20 17:02:47.560 27869 DEBUG keystoneclient.session [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] RESP: [404] Content-Length: 114 Vary: X-Auth-Token Server: Apache Connection: close Date: Tue, 20 Sep 2016 17:02:47 GMT Content-Type: application/json x-openstack-request-id: req-b74e985d-d32d-4857-8f3c-ec4c014fb107
RESP BODY: {"error": {"message": "Could not find role: 9fe2ff9ee4384b1894a90878d3e92bab", "code": 404, "title": "Not Found"}}
 _http_log_response /usr/lib/python2.7/dist-packages/keystoneclient/session.py:231
2016-09-20 17:02:47.561 27869 DEBUG keystoneclient.session [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Request returned failure status: 404 request /usr/lib/python2.7/dist-packages/keystoneclient/session.py:419
2016-09-20 17:02:47.562 27869 ERROR sahara.service.trusts [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Unable to create trust (reason: Could not find role: 9fe2ff9ee4384b1894a90878d3e92bab (HTTP 404) (Request-ID: req-b74e985d-d32d-4857-8f3c-ec4c014fb107))
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Error during rollback of cluster (reason: Failed to create trust
Error ID: 2d5ce02c-0fb6-4856-9d29-d0f085c0e993)
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Traceback (most recent call last):
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] File "/usr/lib/python2.7/dist-packages/sahara/service/ops.py", line 210, in wrapper
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] if _rollback_cluster(cluster, ex):
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] File "/usr/lib/python2.7/dist-packages/sahara/service/ops.py", line 238, in _rollback_cluster
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] _setup_trust_for_cluster(cluster)
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] File "/usr/lib/python2.7/dist-packages/sahara/service/ops.py", line 180, in _setup_trust_for_cluster
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] trusts.create_trust_for_cluster(cluster)
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] File "/usr/lib/python2.7/dist-packages/sahara/service/trusts.py", line 97, in create_trust_for_cluster
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] allow_redelegation=True)
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] File "/usr/lib/python2.7/dist-packages/sahara/service/trusts.py", line 75, in create_trust
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] raise ex.CreationFailed(_('Failed to create trust'))
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] CreationFailed: Failed to create trust
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Error ID: 2d5ce02c-0fb6-4856-9d29-d0f085c0e993
2016-09-20 17:02:47.698 27869 ERROR sahara.service.ops [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3]
2016-09-20 17:02:48.182 27869 INFO sahara.utils.cluster [req-3b19bf3e-6bbb-4e6f-88cb-921a3cf8bbca 43951d0af69848268f07937c12fa36fd 307a375b43274f779d89b0512824a054 - - -] [instance: none, cluster: 58f9e615-9ac1-4445-8c01-ac2d766b02c3] Cluster status has been changed. New status=Error

Reproducibility:
 configure keystone v3 API for federation

Workaround:
 unknown - I would especially like to know of any workaround this? any feedback is appreciated

Impact:
 federation users cannot access Sahara and Murano or heat, this would mean, for me, needing to create manual accounts for +500 students and revert to keystone v2 with users in SQL backend, breaks SSO, identity lifecycle management and UX.

Description of the environment:
 Operation system: Ubuntu with MOS packages
 Versions of components: fuel 9.0
 Reference architecture: HA with ceph
 Network model: GRE
 Related projects installed: Murano, Sahara
Additional information:
 here's something weird - EVERYTHING else works with federated users including swift with ceph hammer backend, but apparently this should not work until ceph jewel
https://bugs.launchpad.net/mos/10.0.x/+bug/1498552

How are the other services working with role based access as group members? - they are not relying yet on the shadow users bp, can the same approach be configured in murano, sahara, heat?

this bug is addressed here:
https://review.openstack.org/#/c/284943/

can this be backported to mitaka? or can I patch keystone manually in the meantime.