User operations with LDAP Identity and enabled_mask/user_enabled_default fail

[Brant Knudson]

Start with devstack configured for LDAP.

Change configuration so that user_enabled_mask=2 and user_enabled_default=512 and restart Keystone server

Try to create a user.

 $ keystone user-create --name blktest2 --tenant demo --pass test2pwd
 -- fails

First problem is, can't get a token.

(keystone.common.wsgi): 2013-08-08 09:55:38,954 ERROR wsgi __call__ 'enabled'
Traceback (most recent call last):
  File "/opt/stack/keystone/keystone/common/wsgi.py", line 240, in __call__
    result = method(context, **params)
  File "/opt/stack/keystone/keystone/token/controllers.py", line 80, in authenticate
    context, auth)
  File "/opt/stack/keystone/keystone/token/controllers.py", line 242, in _authenticate_local
    username, DEFAULT_DOMAIN_ID)
  File "/opt/stack/keystone/keystone/common/manager.py", line 44, in _wrapper
    return f(*args, **kw)
  File "/opt/stack/keystone/keystone/identity/backends/ldap.py", line 87, in get_user_by_name
    ref = identity.filter_user(self.user.get_by_name(user_name))
  File "/opt/stack/keystone/keystone/common/ldap/core.py", line 360, in get_by_name
    res = self.get_all(query)
  File "/opt/stack/keystone/keystone/common/ldap/core.py", line 698, in get_all
    return super(EnabledEmuMixIn, self).get_all(filter)
  File "/opt/stack/keystone/keystone/common/ldap/core.py", line 368, in get_all
    for x in self._ldap_get_all(filter)]
  File "/opt/stack/keystone/keystone/identity/backends/ldap.py", line 240, in _ldap_res_to_model
    obj['enabled_nomask'] = obj['enabled']
KeyError: 'enabled'

When work around can't get a token, can't create user.

(keystone.common.wsgi): 2013-08-08 09:57:20,717 ERROR wsgi __call__ unsupported operand type(s) for &: 'str' and 'int'
Traceback (most recent call last):
  File "/opt/stack/keystone/keystone/common/wsgi.py", line 240, in __call__
    result = method(context, **params)
  File "/opt/stack/keystone/keystone/identity/controllers.py", line 206, in create_user
    new_user_ref = self.identity_api.create_user(user_id, user_ref)
  File "/opt/stack/keystone/keystone/identity/core.py", line 72, in create_user
    return self.driver.create_user(user_id, user)
  File "/opt/stack/keystone/keystone/identity/backends/ldap.py", line 93, in create_user
    user_ref = self.user.create(user)
  File "/opt/stack/keystone/keystone/identity/backends/ldap.py", line 260, in create
    self.mask_enabled_attribute(values)
  File "/opt/stack/keystone/keystone/identity/backends/ldap.py", line 250, in mask_enabled_attribute
    if value != ((values['enabled_nomask'] & self.enabled_mask) !=
TypeError: unsupported operand type(s) for &: 'str' and 'int'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/40893

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/40986

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/40987

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/40988

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/40989

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/40996

[Brant Knudson (blk-u)]

The problem was that the user_enabled_default is a string (defined in keystone.common.config), but then the code was using it like an int (with a bitwise-and operator).

Also, the unit tests weren't actually testing the user_enabled_mask case; the didn't reload the backend so was running with the default config. In addition, the test for user_enabled_mask set user_enabled_default to an int and not a string as it would be if it was read from the config file.

Then for some reason when running with

[ldap]
user_enabled_attribute = employeeType
user_enabled_mask = 2
user_enabled_default = 512

the response from OpenLdap wouldn't include the enabled. I didn't really figure this out but it was easy to recreate. I couldn't figure out a way to test it cleanly because the test is loading the data twice with different configs.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/40986
Committed: http://github.com/openstack/keystone/commit/54a4c0696e3817307b8e9e50a2ffa5b5013e1f2e
Submitter: Jenkins
Branch: master

commit 54a4c0696e3817307b8e9e50a2ffa5b5013e1f2e
Author: Brant Knudson <email address hidden>
Date: Thu Aug 8 15:36:20 2013 -0500

    Skip test_create_unicode_user_name in _ldap_livetest

    Live LDAP tests were not passing because this test doesn't work.
    This is being addressed with a different bug.

    Change-Id: Ic01aa505d867c1de30e2a1ed7c79ff1478e213ef
    Related-Bug: #1172106
    Related-Bug: #1210175

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/40987
Committed: http://github.com/openstack/keystone/commit/68f38a65c60485c34474e490d649b328421e10f5
Submitter: Jenkins
Branch: master

commit 68f38a65c60485c34474e490d649b328421e10f5
Author: Brant Knudson <email address hidden>
Date: Thu Aug 8 15:37:16 2013 -0500

    Do not skip test_user_enable_attribute_mask in _ldap_livetest

    This test is usable with OpenLDAP and is useful for validation.

    Change-Id: Ie4da746a17d2ca545eb1125c1e7249620f0efbc0
    Related-Bug: #1210175

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/40988
Committed: http://github.com/openstack/keystone/commit/116897786dbb8473154ec85a01b019af8106a1f4
Submitter: Jenkins
Branch: master

commit 116897786dbb8473154ec85a01b019af8106a1f4
Author: Brant Knudson <email address hidden>
Date: Thu Aug 8 16:36:52 2013 -0500

    Fix test_user_enable_attribute_mask so it actually tests

    test_user_enable_attribute_mask wasn't actually testing
    user_enable_attribute_mask because it didn't reload the backend
    after changing the config value.

    Change-Id: I9fa6bebe0c4b3d2afc1eb53867cf217b046b0210
    Related-Bug: #1210175

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/40989
Committed: http://github.com/openstack/keystone/commit/54178b735dea4dfee4578caa95cb3ae704afef07
Submitter: Jenkins
Branch: master

commit 54178b735dea4dfee4578caa95cb3ae704afef07
Author: Brant Knudson <email address hidden>
Date: Fri Aug 16 12:42:17 2013 -0500

    More validation in test_user_enable_attribute_mask

    Validate the enabled attribute returned by create_user, update_user.
    Also, validate that the enabled attribute in the LDAP server is
    set.

    Change-Id: I78d194528ad4fd67fc35ca4d124f2e031d02d9cc
    Related-Bug: #1210175

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/40893
Committed: http://github.com/openstack/keystone/commit/87ababb38506a1a51b9e38fc343dd3b46c828a80
Submitter: Jenkins
Branch: master

commit 87ababb38506a1a51b9e38fc343dd3b46c828a80
Author: Brant Knudson <email address hidden>
Date: Thu Aug 8 17:41:10 2013 -0500

    Fix LDAP Identity with non-zero user_enabled_default

    The LDAP Identity backend was not properly using the
    user_enabled_default option as a string. This caused
    operations to fail with

     TypeError: unsupported operand type(s) for &: 'str' and 'int'

    Partial-Bug: #1210175

    Change-Id: I54931e669186871d18dea088870945b9de40d573

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/40996
Committed: http://github.com/openstack/keystone/commit/781c65b72b78bd1e2b1d93db029d6b0c6fbc2050
Submitter: Jenkins
Branch: master

commit 781c65b72b78bd1e2b1d93db029d6b0c6fbc2050
Author: Brant Knudson <email address hidden>
Date: Thu Aug 8 18:38:23 2013 -0500

    Fix LDAP Identity get user with user_enabled_mask

    Fetching users using the LDAP Identity backend would fail with

     KeyError: 'enabled'

    from _ldap_res_to_model when user_enabled_mask is not 0.

    Change-Id: I39a9606ba7210f0fea064abd6adad379218b432b
    Closes-Bug: #1210175

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/43524

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/43524
Committed: http://github.com/openstack/keystone/commit/801a81506e688bca800bd4164b4210fdcc5fee05
Submitter: Jenkins
Branch: stable/grizzly

commit 801a81506e688bca800bd4164b4210fdcc5fee05
Author: Brant Knudson <email address hidden>
Date: Thu Aug 8 15:36:20 2013 -0500

    Fixes for Active Directory

    The LDAP Identity backend was not properly using the
    user_enabled_default option as a string. This caused
    operations to fail with

     TypeError: unsupported operand type(s) for &: 'str' and 'int'

    Also, fetching users using the LDAP Identity backend would fail with

     KeyError: 'enabled'

    from _ldap_res_to_model when user_enabled_mask is not 0.

    Closes-Bug: #1210175
    (cherry picked from commit 68f38a65c60485c34474e490d649b328421e10f5)
    (cherry picked from commit 116897786dbb8473154ec85a01b019af8106a1f4)
    (cherry picked from commit 54178b735dea4dfee4578caa95cb3ae704afef07)
    (cherry picked from commit 87ababb38506a1a51b9e38fc343dd3b46c828a80)
    (cherry picked from commit 781c65b72b78bd1e2b1d93db029d6b0c6fbc2050)

    Change-Id: Ic01aa505d867c1de30e2a1ed7c79ff1478e213ef