keystone-ldap deployment missing "ldappool" module

[Shashinandan Kolar Srinivasa]

Hi,

 keystone-ldap-12 charm does not install "ldappool".

-----------------------
ubuntu@juju-1f7533-3-lxd-1:/var/log/juju$ sudo tail -f ../keystone/keystone.log
    plugin = ep.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/__init__.py", line 13, in <module>
    from keystone.identity.backends.ldap.core import * # noqa
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/core.py", line 26, in <module>
    from keystone.identity.backends.ldap import common as common_ldap
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 25, in <module>
    import ldappool
ModuleNotFoundError: No module named 'ldappool'
-----------------------

Had to install it manually.

---------------
ubuntu@juju-1f7533-3-lxd-1:/var/log/juju$ sudo python3 -m pip install ldappool
The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/ubuntu/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting ldappool
  Downloading https://files.pythonhosted.org/packages/eb/48/ab9f0ed864d42af507f0af755aff74e6b27f6092b244e41dd37bb577ac85/ldappool-2.4.0-py2.py3-none-any.whl
Requirement already satisfied: python-ldap>=3.0.0 in /usr/lib/python3/dist-packages (from ldappool)
Requirement already satisfied: PrettyTable<0.8,>=0.7.2 in /usr/lib/python3/dist-packages (from ldappool)
Installing collected packages: ldappool
Successfully installed ldappool-2.4.0
------------------------

Even with this manual intervention, keystone-ldap integration fails.

------
(ldappool): 2018-12-21 22:07:16,494 DEBUG Attempting to create a new connector to ldaps://<ourldap>.com (attempt 1)
(dogpile.lock): 2018-12-21 22:07:16,494 DEBUG Released creation lock
(keystone.common.wsgi): 2018-12-21 22:07:16,494 ERROR simple_bind() argument 1 must be str or None, not bytes
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/keystone/common/wsgi.py", line 148, in __call__
    result = method(req, **params)
  File "/usr/lib/python3/dist-packages/keystone/auth/controllers.py", line 67, in authenticate_for_token
    self.authenticate(request, auth_info, auth_context)
  File "/usr/lib/python3/dist-packages/keystone/auth/controllers.py", line 236, in authenticate
    auth_info.get_method_data(method_name))
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/password.py", line 31, in authenticate
    user_info = auth_plugins.UserAuthInfo.create(auth_payload, METHOD_NAME)
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/core.py", line 102, in create
    user_auth_info._validate_and_normalize_auth_data(auth_payload)
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/core.py", line 189, in _validate_and_normalize_auth_data
    auth_payload)
  File "/usr/lib/python3/dist-packages/keystone/auth/plugins/core.py", line 166, in _validate_and_normalize_auth_data
    user_name, domain_ref['id'])
  File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 116, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 416, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 426, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1220, in decorate
    should_cache_fn)
  File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 825, in get_or_create
    async_creator) as value:
  File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 154, in __enter__
    return self._enter()
  File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 94, in _enter
    generated = self._enter_create(createdtime)
  File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 145, in _enter_create
    created = self.creator()
  File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 792, in gen_value
    created_value = creator()
  File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1216, in creator
    return fn(*arg, **kw)
  File "/usr/lib/python3/dist-packages/keystone/identity/core.py", line 988, in get_user_by_name
    ref = driver.get_user_by_name(user_name, domain_id)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/core.py", line 97, in get_user_by_name
    return self.user.filter_attributes(self.user.get_by_name(user_name))
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 1532, in get_by_name
    res = self.get_all(query)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/core.py", line 319, in get_all
    hints=hints)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 1839, in get_all
    return super(EnabledEmuMixIn, self).get_all(ldap_filter, hints)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 1541, in get_all
    for x in self._ldap_get_all(hints, ldap_filter)]
  File "/usr/lib/python3/dist-packages/keystone/common/driver_hints.py", line 42, in wrapper
    return f(self, hints, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 1489, in _ldap_get_all
    with self.get_connection() as conn:
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 1249, in get_connection
    conn.simple_bind_s(user, password)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 892, in simple_bind_s
    clientctrls=clientctrls)
  File "/usr/lib/python3/dist-packages/keystone/identity/backends/ldap/common.py", line 747, in simple_bind_s
    with self._get_pool_connection() as conn:
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/ldappool/__init__.py", line 375, in connection
    conn = self._get_connection(bind, passwd)
  File "/usr/local/lib/python3.6/dist-packages/ldappool/__init__.py", line 325, in _get_connection
    conn = self._create_connector(bind, passwd)
  File "/usr/local/lib/python3.6/dist-packages/ldappool/__init__.py", line 266, in _create_connector
    self._bind(conn, bind, passwd)
  File "/usr/local/lib/python3.6/dist-packages/ldappool/__init__.py", line 230, in _bind
    conn.simple_bind_s(bind, passwd)
  File "/usr/local/lib/python3.6/dist-packages/ldappool/__init__.py", line 103, in simple_bind_s
    clientctrls)
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 1214, in simple_bind_s
    res = self._apply_method_s(SimpleLDAPObject.simple_bind_s,*args,**kwargs)
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 1196, in _apply_method_s
    return func(self,*args,**kwargs)
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 443, in simple_bind_s
    msgid = self.simple_bind(who,cred,serverctrls,clientctrls)
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 437, in simple_bind
    return self._ldap_call(self._l.simple_bind,who,cred,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls))
  File "/usr/lib/python3/dist-packages/ldap/ldapobject.py", line 313, in _ldap_call
    result = func(*args,**kwargs)
TypeError: simple_bind() argument 1 must be str or None, not bytes
-----------------------------

[James Page (james-page)]

version 12 of the charm will install ldappool but only for Mitaka or later.

FTR installing from pip is not a great idea - always try to install from the Ubuntu or UCA archives otherwise you are getting something that has never really been tested.

Which release of OpenStack are you trying to deploy?

[James Page (james-page)]

And then I realised this is for the keystone charm, not the keystone-ldap charm!

That said the preferred route to using LDAP is with the subordinate charm so please try with that instead.

[Shashinandan Kolar Srinivasa (shashinandan)]

Thanks for the update !
I was trying rocky release.
Deployed queens where this problem doesn't arise.

[Jason Hobbs (jason-hobbs)]

This affects rocky deployments using the keystone-ldap subordinate as well.

Before installing python3-ldappool, I get this error in /var/log/keystone/keystone.log:

http://paste.ubuntu.com/p/k5qHmDR8zM/

And I get keystone hook failures when trying to create users during relation joins.

After installing python3-ldappool, everything starts working.

Here is the bundle:
http://paste.ubuntu.com/p/qbtQ8xvBbY/

[Jason Hobbs (jason-hobbs)]

This is the crashdump from my failure.

[Jason Hobbs (jason-hobbs)]

Sub'd to field-critical since this affects all rocky deployments with ldap.

[James Page (james-page)]

Please reconsider the field-critical tag - lack of py3 support for keystone ldap was a release-noted issue last november:

  https://docs.openstack.org/charm-guide/latest/1811.html#python3-keystone-ldap-does-not-allow-bytes-for-dn-rdn-field-names

Package updates are being prepared, but this should not be a surprise that its non-functional.

[James Page (james-page)]

bug 1798184

[Jason Hobbs (jason-hobbs)]

We are sticking with field-critical on this.

Is bug 1798184 a blocker for this? Should this be a duplicate?

After installing python3-ldappool, everything starts working. That seems to indicate to me that the package should just be installed. Maybe that's a packaging issue that will be fixed in 1798184?

[James Page (james-page)]

There is an associated charm change alongside the packaging to support rocky; once the packages have been accepted by the ubuntu-sru team we can crack on with updates to the charm and testing.

[James Page (james-page)]

Duping this bug against the original charm bug for this problem

https://review.openstack.org/#/c/621275 fixes the charm; however it was blocked by issues in the packages which are being resolved under bug 1798184

[Corey Bryant (corey.bryant)]

@Jason, yes 1798184 is a blocker for this. I'm working on getting it verified atm.