Error on _ldap_get_list without attrlist value

[Marcos Lobo]

Using keystone from master branch (keystone-2014.2.dev170.g2e49770) and configured with LDAP backend. Now, If you try this command:

$ keystone tenant-list
Authorization Failed: An unexpected error prevented the server from fulfilling your request: 'utf8' codec can't decode byte 0x97 in position 2: invalid start byte (Disable debug mode to suppress these details.) (HTTP 500)

The _ldap_get_list (/keystone/common/ldap/core.py) function has a problem when the attrlist attribute is None. This function raises an error like:

2014-08-18 16:19:31.861 26110 ERROR keystone.common.wsgi [-] 'utf8' codec can't decode byte 0x97 in position 2: invalid start byte
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi Traceback (most recent call last):
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/wsgi.py", line 214, in __call__
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi result = method(context, **params)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/token/controllers.py", line 99, in authenticate
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi context, auth)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/token/controllers.py", line 300, in _authenticate_local
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi user_id, tenant_id)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/token/controllers.py", line 379, in _get_project_roles_and_ref
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi user_id, tenant_id)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/assignment/core.py", line 186, in get_roles_for_user_and_project
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi user_role_list = _get_user_project_roles(user_id, project_ref)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/assignment/core.py", line 167, in _get_user_project_roles
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi tenant_id=project_ref['id'])
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/assignment/backends/ldap.py", line 131, in _get_metadata
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi tenant_id)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/assignment/backends/ldap.py", line 107, in _get_roles_for_just_user_and_project
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi (self.project._id_to_dn(tenant_id))
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/assignment/backends/ldap.py", line 555, in get_role_assignments
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi roles = self._ldap_get_list(tenant_dn, ldap.SCOPE_ONELEVEL)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/ldap/core.py", line 1422, in _ldap_get_list
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi return conn.search_s(search_base, scope, query, attrlist)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/ldap/core.py", line 926, in search_s
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi py_result = convert_ldap_result(ldap_result)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/ldap/core.py", line 154, in convert_ldap_result
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi for kind, values in six.iteritems(attrs))))
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/ldap/core.py", line 154, in <genexpr>
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi for kind, values in six.iteritems(attrs))))
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/ldap/core.py", line 123, in ldap2py
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi return utf8_decode(val)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib/python2.6/site-packages/keystone/common/ldap/core.py", line 84, in utf8_decode
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi return _utf8_decoder(value)[0]
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi return codecs.utf_8_decode(input, errors, True)
2014-08-18 16:19:31.861 26110 TRACE keystone.common.wsgi UnicodeDecodeError: 'utf8' codec can't decode byte 0x97 in position 2: invalid start byte

The problem is attrlist attribute is not validated before to send it to LDAP search.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/114986

[Marcos Lobo (marcos-fermin-lobo)]

Similar problem here https://bugs.launchpad.net/keystone/+bug/1364521 reported by Brent Roskos

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dolph Mathews (<email address hidden>) on branch: master
Review: https://review.openstack.org/114986
Reason: we had three bugs with three fixes for this (my apologies for not catching this sooner), and the chatter in the IRC channel today suggests that we're settling on https://bugs.launchpad.net/keystone/+bug/1355489 and https://review.openstack.org/#/c/119457/ for master and stable/icehouse