Failure when federated user name contains non ascii characters

[Jon Kåre Hellan]

When logging in with openid-connect, I get

 '{"error": {"message": "An unexpected error prevented the server from fulfilling your request: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128) (Disable debug mode to suppress these details.)", "code": 500, "title": "Internal Server Error"}}'

My name has an 'å'. I suspect there is a connection.

Coincidentally(?), if I do the following in python shell:

>>> unicode('Jon Kåre Hellan')

I get 'UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)'

This is on liberty, using federation in contrib. On master, federation has been moved up from contrib, but I couldn't see any code changes that would help.

Stack trace:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 248, in __call__
    result = method(context, **params)
  File "/usr/lib/python2.7/site-packages/keystone/contrib/federation/controllers.py", line 315, in federated_sso_auth
    protocol_id)
  File "/usr/lib/python2.7/site-packages/keystone/contrib/federation/controllers.py", line 297, in federated_authentication
    return self.authenticate_for_token(context, auth=auth)
  File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 385, in authenticate_for_token
    self.authenticate(context, auth_info, auth_context)
  File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 510, in authenticate
    auth_context)
  File "/usr/lib/python2.7/site-packages/keystone/auth/plugins/mapped.py", line 69, in authenticate
    self.identity_api)
  File "/usr/lib/python2.7/site-packages/keystone/auth/plugins/mapped.py", line 144, in handle_unscoped_token
    federation_api, identity_api)
  File "/usr/lib/python2.7/site-packages/keystone/auth/plugins/mapped.py", line 188, in apply_mapping_filter
    identity_provider, protocol, assertion)
  File "/usr/lib/python2.7/site-packages/keystone/contrib/federation/core.py", line 90, in evaluate
    mapped_properties = rule_processor.process(assertion_data)
  File "/usr/lib/python2.7/site-packages/keystone/contrib/federation/utils.py", line 470, in process
    new_local = self._update_local_mapping(local, direct_maps)
  File "/usr/lib/python2.7/site-packages/keystone/contrib/federation/utils.py", line 611, in _update_local_mapping
    new_value = self._update_local_mapping(v, direct_maps)
  File "/usr/lib/python2.7/site-packages/keystone/contrib/federation/utils.py", line 613, in _update_local_mapping
    new_value = v.format(*direct_maps)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)

[Steve Martinelli (stevemar)]

this sounds like it's very likely a bug, i'm not sure how much extra handling (if any) we do for non-ascii names in the federation mapping

[Steve Martinelli (stevemar)]

no patch yet, so i'm bumping to mitaka-3 as mitaka-2 is due tomorrow

[David Stanek (dstanek)]

Is this actively being worked on?

[Deliang Fan (vanderliang)]

@David Hi, I'm working on it, while it may takes some time because I am new fresh to federation.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/279908

[Steve Martinelli (stevemar)]

I tossed up a related patch, having trouble recreating the issue though

[David Stanek (dstanek)]

I filled out the patch to show the problem and provide a fix for it. The TL;DR version is that the header comes in a bytes (that could be ISO-8859-1). The mapping dictionary comes out of the DB as JSON. This is a bug since ``u'{}'.format(b'å')`` will raise a UnicodeDecodeError.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/279908
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=e913001cbedb4dd8748023ede31115a032de83f8
Submitter: Jenkins
Branch: master

commit e913001cbedb4dd8748023ede31115a032de83f8
Author: Steve Martinelli <email address hidden>
Date: Sat Feb 13 20:42:30 2016 -0500

    handle unicode names for federated users

    the previous logic that handled getting the assertions from
    the environment did not account for utf8 characters

    Co-Authored-By: David Stanek <email address hidden>
    Closes-Bug: 1525250

    Change-Id: I90f4885161a72758986a652e845b4017f9cdcfb7

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/keystone 9.0.0.0b3 development milestone.