Keystone ldap tree_dn does not support Chinese

[vegezcj]

Keystone ldap tree_dn does not support Chinese
My keystone.conf:

url = ldap://10.153.195.125
user = CN=Administrator,CN=Users,DC=h3c,DC=com
password = h3C123456
suffix = DC=h3c,DC=com
query_scope = sub
user_tree_dn = OU=华三,DC=h3c,DC=com
.......
.......

My tree_dn config ou is Chinese,I try login openstack dashboard
It throw:UnicodeDecodeError:'ascii' codes can't decode bytes 0xe5 in position 3:ordinal not in range(128)
I think tree_dn need unicode.

[Steve Martinelli (stevemar)]

silly question, does the ldap in question work with other application? i don't know the LDAP spec well enough to say if there is a character restriction for DNs, but I'm guessing not.

[vegezcj (453528434-f)]

I'm just given example,I have a demand, need to read the ldap user,but his ldap server ou contains Chinese,
then the above problems.
I modify '/keystone/common/ldap/core.py' ,self.tree_dn = unicode(self.tree_dn,'utf-8'）,solve this problem.
I think it will affect our Chinese users.so I commit this bug, this is not a silly question.

[Thomas Goirand (thomas-goirand)]

FYI, the person who opened #852536 in the Debian BTS (this bug is a forward of this) uses his own brewed patch like this:

Author: Sakirnth Nagarasa <email address hidden>
Description: Allows user with special character log on into dashboard

diff -Naurp keystone.orig/keystone/identity/backends/ldap/common.py
--- keystone.orig/keystone/identity/backends/ldap/common.py
+++ keystone/keystone/identity/backends/ldap/common.py
@@ -18,6 +18,8 @@ import functools
 import os.path
 import re
 import sys
+reload(sys)
+sys.setdefaultencoding('utf-8')
 import weakref

 import ldap.controls

I do think the above patch is strait forward and easy to understand. Comments from the Keystone team welcome.

[David Stanek (dstanek)]

Are you changing the default encoding like the Thomas mentioned? If so, where in the code are you doing it and what encoding are you changing it to be?

[Dolph Mathews (dolph)]

vegezcj: Steve was describing his own question in comment #1 as being silly (not the bug report).

[David Stanek (dstanek)]

I found a related oslo.config blueprint: https://blueprints.launchpad.net/oslo.config/+spec/values-unicode-support

So in thinking about this I think that a better approach might be to and an 'encoding' param to olso_config.cfg.ConfigParser.__call__ so that we can open that file as that encoding and return text instead of bytes.

Adding olso.config do see if they have had any thoughts about this kind of issue. I'd hate to start littering keystone with even more .decode() calls when that's really a bandaid.

[vegezcj (453528434-f)]

David: Thomas modify method syntax check is failed in the python3.x. I just got in touch with openstack, I think the lack of ability to modify oslo.config blueprint.I tried to change the way you say modify the 'encoding'param in the local openstack environment.
Or, about this problem ,you have any better suggestions.

[Lance Bragstad (lbragstad)]

Automatically unassigning due to inactivity.

[Doug Hellmann (doug-hellmann)]

Is this still a problem for more modern oslo.config?

[Launchpad Janitor (janitor)]

[Expired for OpenStack Identity (keystone) because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for oslo.config because there has been no activity for 60 days.]

[Colleen Murphy (krinkle)]

This seems to still be a problem at least in keystone in Queens https://bugs.launchpad.net/keystone/+bug/1810441

[Vidmantas (vidmantasvgtu)]

openstack rocky ldap keystone

  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 198: ordinal not in range(128)

[Vidmantas (vidmantasvgtu)]

user_tree_dn= OU =Infrastruktūros skyrius,OU=Informacinių technologijų ir sistemų centras,OU=ITSC,DC=domain,DC=local

[Audun Gangsto (audun-m)]

Still a problem on Stein.

[Bjoern (bjoern-t)]

Yes I can confirm this is still an issue where utf-8 sequences like \C3\B6h are present inside the DN. The DN was in my case even base64 encoded

[Aurelien Lourot (aurelien-lourot)]

The original issue (Keystone failing to tell oslo.config which encoding to use) still exist in Keystone, see analysis in lp:1933109 . I believe now these two bugs are duplicates.

#6 sounds promising and I couldn't find any Keystone Gerrit review attempting it. Should we give it a try?

Traceback of this issue seen on OpenStack Ussuri when having cyrillic chars in the LDAP config:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 103, in _inner
    return method(self, request)
  File "/usr/lib/python3/dist-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 358, in process_request
    resp = super(AuthContextMiddleware, self).process_request(request)
  File "/usr/lib/python3/dist-packages/keystonemiddleware/auth_token/__init__.py", line 409, in process_request
    data, user_auth_ref = self._do_fetch_token(
  File "/usr/lib/python3/dist-packages/keystonemiddleware/auth_token/__init__.py", line 445, in _do_fetch_token
    data = self.fetch_token(token, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/server/flask/request_processing/middleware/auth_context.py", line 252, in fetch_token
    self.token = self.token_provider_api.validate_token(
  File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/token/provider.py", line 145, in validate_token
    token = self._validate_token(token_id)
  File "<decorator-gen-26>", line 2, in _validate_token
  File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1359, in get_or_create_for_user_func
    return self.get_or_create(
  File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 957, in get_or_create
    with Lock(
  File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 187, in __enter__
    return self._enter()
  File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 94, in _enter
    generated = self._enter_create(value, createdtime)
 File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 180, in _enter_create
    return self.creator()
  File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 915, in gen_value
    created_value = creator(
  File "/usr/lib/python3/dist-packages/keystone/token/provider.py", line 179, in _validate_token
    token.mint(token_id, issued_at)
  File "/usr/lib/python3/dist-packages/keystone/models/token_model.py", line 580, in mint
    self._validate_token_user()
  File "/usr/lib/python3/dist-packages/keystone/models/token_model.py", line 503, in _validate_token_user
    if not self.user_domain.get('enabled'):
  File "/usr/lib/python3/dist-packages/keystone/models/token_model.py", line 139, in user_domain
    if self.user:
  File "/usr/lib/python3/dist-packages/keystone/models/token_model.py", line 133, in user
    self.__user = PROVIDERS.identity_api.get_user(self.user_id)
  File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 4...