GET /v3/role_assignments?effective&include_names API fails with unexpected 500 error

[prashkre]

In an environment like ldap server as identity backend, where a group has role assignment but some users in group doesn't have "name" attribute configured in ldap. So while fetching effective role assignments with include_names, it is failing in below stack trace error.

2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi Traceback (most recent call last):
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 228, in __call__
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi result = method(req, **params)
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/assignment/controllers.py", line 999, in list_role_assignments_wrapper
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi return self.list_role_assignments(request)
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/controller.py", line 235, in wrapper
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi return f(self, request, filters, **kwargs)
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/assignment/controllers.py", line 956, in list_role_assignments
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi return self._list_role_assignments(request, filters)
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/assignment/controllers.py", line 945, in _list_role_assignments
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi include_names=include_names)
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi __ret_val = __f(*args, **kwargs)
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 948, in list_role_assignments
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi return self._get_names_from_role_assignments(role_assignments)
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 974, in _get_names_from_role_assignments
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi new_assign['user_name'] = _user['name']
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi KeyError: 'name'
2017-07-13 05:06:10.835 10460 ERROR keystone.common.wsgi

[Lance Bragstad (lbragstad)]

We have documentation that explains how to map attributes, like `name`, from LDAP to keystone [0]. I would be curious to see if you still end up with the same issue after reading and applying the configuration in those docs.

[0] https://docs.openstack.org/keystone/latest/configuration.html#using-an-ldap-server

[Lance Bragstad (lbragstad)]

Marking this as invalid for now. Please feel free to reopen this if there are further issues after following the instructions in comment #1.

[Matthew Edmonds (edmondsw)]

Yeah, those instructions were followed, but the problem here was that some users didn't have a value set in the property that was used for name. More specifically, the customer used a field that holds the email address as the name, and some users didn't have an email address. But even beyond that, we couldn't tell them to use a different LDAP attribute because there was no single attribute that consistently had a value for all users, even cn. You could argue that LDAP was misconfigured, but good luck getting that fixed in a large enterprise environment (which this was). You could argue that keystone was misconfigured, but in this case there was not a better LDAP attribute to use for name. So I'd like to see keystone handle this better somehow. Could keystone report a name of "" or "<missing>" or something when the attribute that is supposed to have the name is not found on a given resource?

[Lance Bragstad (lbragstad)]

Populating the name with a key like '<missing>' would work, but I think it would lead to confusion and it would push edge cases into clients. Are user's expected to always authenticate with IDs? Do they know their ID and not their username? What is the ID of the user in this case? I see two options if keystone has to deal with misconfigured LDAP installations.

1.) Keystone emits a known token in place of the name, like '<missing>'. What happens to scripts or tools that programmatically do things with the response? Do they have to hard-code how to deal with a special key?

2.) Keystone omits the name attribute in cases where the LDAP entry doesn't have the attribute. This would keep the clients from having to program around a misconfigured LDAP. At the same time, it's backward incompatible since the API guarantees a name attribute be returned.

For SQL installations, we require a name in order to function properly. I'm unsure letting LDAP slide on that requirement is a good thing long term.

[Matthew Edmonds (edmondsw)]

I think it would be perfectly reasonable to say that users without a name can't login. Maybe we could even go a step further and filter them within keystone, i.e. act like they don't exist. The more I think of that, the more I like it. It maintains the requirement that users have names, without disrupting users that do have names.

[Lance Bragstad (lbragstad)]

If we want to enforce the requirement of names, and a valid attribute mapping in case of LDAP deployments, do we bother with a fix? Having conditionals around an attribute keystone requires to be present seems like a contradiction. A fix here will also prolong proper configuration (of LDAP, not keystone).

Another possible route would be to document a way to do this on the LDAP side. If an LDAP deployment doesn't have a consistent attribute to use for names, then couldn't they iterate through all their users and create an attribute specifically for it (e.g. openstack-name)?

[Matthew Edmonds (edmondsw)]

"proper" configuration of LDAP isn't going to happen, so fixing this isn't prolonging that...

Similarly, documenting a way to address this on the LDAP side will do nothing. That's not going to happen either.

The only place this can be fixed is in keystone. And I don't see why it would be a contradiction to filter users that don't have fields we require... that just seems right to me.

[Lance Bragstad (lbragstad)]

Setting this to Low since there is a work around.

[Lance Bragstad (lbragstad)]

For what it's worth - this is change the response code from a 5XX response code to something else (either a 4XX or 2XX depending on the request). That should be fine according to the API WG guidelines [0].

[0] http://specs.openstack.org/openstack/api-wg/guidelines/api_interoperability.html#evaluating-api-changes

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/485302

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/485302
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=bb2b215b539b56c01e9a0c7acd9fd62ed1529ebc
Submitter: Jenkins
Branch: master

commit bb2b215b539b56c01e9a0c7acd9fd62ed1529ebc
Author: prashkre <email address hidden>
Date: Wed Jul 19 23:37:02 2017 +0530

    Filter users and groups in ldap

    All users and groups are required to have a name. Prior to this fix,
    Keystone was not properly enforcing this for LDAP users and groups.
    Keystone will now ignore users and groups that do not have a value for
    the LDAP attribute which Keystone has been configured to use for that
    entity's name.

    Closes-Bug: #1704205

    Change-Id: I424813785b7a355514ef42f1e4c6384b8a78b256

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/490138

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 12.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/490138
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=16d8f0d11f0ab9678b7e99f063fcce23d32c3c3b
Submitter: Jenkins
Branch: stable/ocata

commit 16d8f0d11f0ab9678b7e99f063fcce23d32c3c3b
Author: prashkre <email address hidden>
Date: Wed Jul 19 23:37:02 2017 +0530

    Filter users and groups in ldap

    All users and groups are required to have a name. Prior to this fix,
    Keystone was not properly enforcing this for LDAP users and groups.
    Keystone will now ignore users and groups that do not have a value for
    the LDAP attribute which Keystone has been configured to use for that
    entity's name.

    Closes-Bug: #1704205

    Change-Id: I424813785b7a355514ef42f1e4c6384b8a78b256
    (cherry picked from commit bb2b215b539b56c01e9a0c7acd9fd62ed1529ebc)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 11.0.4 release.