Keystone create nonlocal user failed with sqlalchemy 1.2.X

[Zhenmei]

Keystone report error when executing 'openstack user list' with a ldap user
2019-05-24 02:17:20.717356 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi [req-e8dce895-b647-4699-8c89-b5b128976273 - - - - -] (exceptions.TypeError) Not a boolean value: u'544' [SQL: u'INSERT INTO user (id, domain_id, enabled, extra, default_project_id, created_at, last_active_at) VALUES (%(id)s, %(domain_id)s, %(enabled)s, %(extra)s, %(default_project_id)s, %(created_at)s, %(last_active_at)s)'] [parameters: [{'last_active_at': None, 'extra': {'email': <email address hidden>', 'description': u'tester'}, 'created_at': datetime.datetime(2019, 5, 24, 2, 17, 20, 583026), 'enabled': u'544', 'id': u'tester', 'default_project_id': None, 'domain_id': 'default'}]]: DBError: (exceptions.TypeError) Not a boolean value: u'544' [SQL: u'INSERT INTO user (id, domain_id, enabled, extra, default_project_id, created_at, last_active_at) VALUES (%(id)s, %(domain_id)s, %(enabled)s, %(extra)s, %(default_project_id)s, %(created_at)s, %(last_active_at)s)'] [parameters: [{'last_active_at': None, 'extra': {'email': <email address hidden>', 'description': u'tester'}, 'created_at': datetime.datetime(2019, 5, 24, 2, 17, 20, 583026), 'enabled': u'544', 'id': u'tester', 'default_project_id': None, 'domain_id': 'default'}]]
2019-05-24 02:17:20.717405 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi Traceback (most recent call last):
2019-05-24 02:17:20.717408 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 148, in __call__
2019-05-24 02:17:20.717410 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi result = method(req, **params)
2019-05-24 02:17:20.717412 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 67, in authenticate_for_token
2019-05-24 02:17:20.717415 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi self.authenticate(request, auth_info, auth_context)
2019-05-24 02:17:20.717417 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 236, in authenticate
2019-05-24 02:17:20.717419 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi auth_info.get_method_data(method_name))
2019-05-24 02:17:20.717422 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/plugins/password.py", line 37, in authenticate
2019-05-24 02:17:20.717424 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi password=user_info.password)
2019-05-24 02:17:20.717426 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 116, in wrapped
2019-05-24 02:17:20.717428 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi __ret_val = __f(*args, **kwargs)
2019-05-24 02:17:20.717430 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/notifications.py", line 521, in wrapper
2019-05-24 02:17:20.717433 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi result = f(wrapped_self, request, user_id, *args, **kwargs)
2019-05-24 02:17:20.717435 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 416, in wrapper
2019-05-24 02:17:20.717437 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return f(self, *args, **kwargs)
2019-05-24 02:17:20.717439 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 426, in wrapper
2019-05-24 02:17:20.717441 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return f(self, *args, **kwargs)
2019-05-24 02:17:20.717443 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 913, in authenticate
2019-05-24 02:17:20.717445 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi ref = self._shadow_nonlocal_user(ref)
2019-05-24 02:17:20.717447 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1270, in decorate
2019-05-24 02:17:20.717449 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi should_cache_fn)
2019-05-24 02:17:20.717451 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 864, in get_or_create
2019-05-24 02:17:20.717626 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi async_creator) as value:
2019-05-24 02:17:20.717640 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/lock.py", line 186, in __enter__
2019-05-24 02:17:20.717642 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return self._enter()
2019-05-24 02:17:20.717644 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/lock.py", line 93, in _enter
2019-05-24 02:17:20.717646 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi generated = self._enter_create(value, createdtime)
2019-05-24 02:17:20.717648 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/lock.py", line 179, in _enter_create
2019-05-24 02:17:20.717650 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return self.creator()
2019-05-24 02:17:20.717652 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 831, in gen_value
2019-05-24 02:17:20.717764 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi created_value = creator()
2019-05-24 02:17:20.717785 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1266, in creator
2019-05-24 02:17:20.717787 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return fn(*arg, **kw)
2019-05-24 02:17:20.717790 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 1409, in _shadow_nonlocal_user
2019-05-24 02:17:20.717792 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return PROVIDERS.shadow_users_api.create_nonlocal_user(user)
2019-05-24 02:17:20.717794 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/sql/core.py", line 516, in wrapper
2019-05-24 02:17:20.717796 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return method(*args, **kwargs)
2019-05-24 02:17:20.717886 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/identity/shadow_backends/sql.py", line 161, in create_nonlocal_user
2019-05-24 02:17:20.717890 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi return identity_base.filter_user(new_user_ref.to_dict())
2019-05-24 02:17:20.717892 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2019-05-24 02:17:20.717894 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi self.gen.next()
2019-05-24 02:17:20.717896 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1043, in _transaction_scope
2019-05-24 02:17:20.717898 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi yield resource
2019-05-24 02:17:20.717900 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2019-05-24 02:17:20.717902 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi self.gen.next()
2019-05-24 02:17:20.717904 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 653, in _session
2019-05-24 02:17:20.717906 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi self.session.rollback()
2019-05-24 02:17:20.717908 2019-05-24 02:17:20.710 9992 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__

[Zhenmei (zma-4)]

When I change the sqlalchemy to 1.1.11, then it works. But with sqlalchemy 1.2.x version, then it failed. It maybe caused by that sqlalchemy Boolean datatype now enforces strict True/False/None values.
reference: https://docs.sqlalchemy.org/en/13/changelog/migration_12.html

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/661183

[Colleen Murphy (krinkle)]

From discussion on the proposed fix, this seems to actually be an issue of an incorrectly configured keystone deployment and not an issue with the sqlalchemy version. Using the user_enabled_mask ldap config option is the correct fix for converting an integer returned from the ldap schema to a boolean that sqlalchemy can use.

Marking this bug as incomplete to leave it open for other operators to signal that this affects them too and deserves an upstream fix, otherwise this bug will expire.