Need support for OpenDirectory in LDAP driver

[Andrey Grebennikov]

It is necessary to support Apple OpenDirectory as the backend for Keystone Identity.

OpenDirectory uses a concept of POSIX groups, when the entities of users in the groups are represented as UIDs, not full DNs:

dn: cn=group1, cn=groups,dc=domain,dc=com
....
memberUid: user1
memberUid: user2
....

while in the driver of LDAP it is hardcoded that the entities could be only full DNs, like:

dn: cn=group1, cn=groups,dc=domain,dc=com
....
memberUid: uid=user1,cn=users,dc=domain,dc=com
memberUid: uid=user2,cn=users,dc=domain,dc=com

Because of this reason it is impossible to use groups in Keystone and we cannot assign the roles to the Keystone groups - Keystone doesn't recognize any user to be a part of any group. When it checks the roles, it searches for the direct user's assignments, and then for any groups which the user can be a member of. So by default the search returns nothing.

We have to have an additional parameter in the config where we specify the type of the entity in the groups - whether is it currently a dn or an id.

[Steve Martinelli (stevemar)]

sounds like you created a work around for this? is that true? if so, can you post your patch here so others may look at the code and get a better feel for the problem.

overall, it sounds like it's solvable.

[Andrey Grebennikov (agrebennikov)]

Sure, here is the patch. The review hopefully will be submitted today.

[Guang Yee (guang-yee)]

Looks like a dup of this one

https://bugs.launchpad.net/keystone/+bug/1489105

Thanks for fixing it. I look forward to reviewing it.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/258528

[Steve Martinelli (stevemar)]

not gating, no tests, moving to mitaka-3

[Alexandre Carnal (yoba-f-deactivatedaccount)]

I have posted a bugs report #1546136 that seems to be related to that one.
I've applied the changes proposed by Andrey and it seems to be working but not at all the times.
The openstack command "openstack user list --group Cloudmembers --domain <openldapdomainID> returns sometimes "Could not find group: 10005 (HTTP 404) (Request-ID: req-6670562c-2ab3-4e3d-abfa-f15610511db9)" or "No group with a name or ID of 'Cloudmembers' exists"
To have a result I have to run the openstack command several times.

What I discover in the keystone logs is that at each non positive return, there's no ldap search.

[Andrey Grebennikov (agrebennikov)]

Alexandre,
The most confusing part here is "not at all times". I have an installation with the fix proposed and it always works just fine.
If you see the output in the log where ldapsearch shows the dn instead of uid sometimes, I'd suggest not all keystone instances were restarted after the patch (in case you have HA installation). Could you please clarify? I'd also like to know if you see different queries on the same keystone node when you are requesting the same user, like:
LDAP search: base=ou=Group,dc=gvadc,dc=localdomain scope=2 filterstr=(&(memberUid=cn=<first_name last_name>,ou=People,dc=<domain>,dc=localdomain)(objectClass=posixGroup)(cn=*)) attrs=['cn', 'description'] attrsonly=0
next time:
LDAP search: base=ou=Group,dc=gvadc,dc=localdomain scope=2 filterstr=(&(memberUid=<last_name>)(objectClass=posixGroup)(cn=*)) attrs=['cn', 'description'] attrsonly=0
Thanks.

[Alexandre Carnal (yoba-f-deactivatedaccount)]

Andrey,
After rebooting the controller which host keystone, the command "openstack user list --group Cloudmembers --domain d02cc542b9c741999bc7addda943c701" works fine.
The log shows "LDAP search: base=cn=Cloudmembers,ou=Group,dc=gvadc,dc=localdomain scope=0 filterstr=(objectClass=posixGroup) attrs=['memberUid'] attrsonly=0
........
LDAP search: base=ou=People,dc=gvadc,dc=localdomain scope=2 filterstr=(&(uid=acarnal)(objectClass=posixAccount)) attrs=['mail', 'userPassword', 'enabled', 'uid'] attrsonly=0"

For the openstack command "openstack group list --user acarnal --user-domain gvadc" the parsed query is not correct: the cn is parsed in place of the uid
The logs shows "LDAP search: base=ou=Group,dc=gvadc,dc=localdomain scope=2 filterstr=(&(memberUid=cn=Alexandre Carnal,ou=People,dc=gvadc,dc=localdomain)(objectClass=posixGroup)(memberUid=*)) attrs=['memberUid', 'description', 'cn'] attrsonly=0
If memberUid=acarnal, it would works.
Is it possible to add an option for that?

Thanks.

[Alexandre Carnal (yoba-f-deactivatedaccount)]

Okay,
I found a work around.
in keystone/identity/backends/ldap.py line 161
replace
 "return self.group.list_user_groups_filtered(user_dn, hints)"
with
 "return self.group.list_user_groups_filtered(user_id, hints)"

Now I can retrieve the group membership of an openldap user.

openstack group list --user acarnal --user-domain gvadc
+------------------------------------------------------------------+--------------+
| ID | Name |
+------------------------------------------------------------------+--------------+
| c7b50b3cdbbcd9803b3e09b37fc69c10550bed1a93d6403a1694194e2f3442fc | IMM |
| 41c11a9c445f3284deb2e96fea7c7492d4d463e878820731cf43f701c49ee5f0 | Cloudmembers |
+------------------------------------------------------------------+--------------+

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/258528
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=449f1f2bdee5fe8026239667838cf2ab976806fd
Submitter: Jenkins
Branch: master

commit 449f1f2bdee5fe8026239667838cf2ab976806fd
Author: Alexander Makarov <email address hidden>
Date: Wed Dec 16 17:11:36 2015 +0300

    Enable support for posixGroups in LDAP

    Support LDAP backends using POSIX goups

    Change-Id: Iaaf022bfdcbd26b3a29c84ff60a033f65a60302b
    Closes-Bug: 1526462

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/keystone 9.0.0.0b3 development milestone.

[Andrey Grebennikov (agrebennikov)]

It looks like the issue was not fixed.
With the patch applied, it is possible to get users as group members ("openstack group contains user"), but at the same time assignments don't work correctly.
When I assign a role in a project to the group instead of the user, I'm expecting the role would be inherited by the user. Nevertheless it comes to:

    def list_groups_for_user(self, user_id, hints):
        user_ref = self._get_user(user_id)
        user_dn = user_ref['dn']
        return self.group.list_user_groups_filtered(user_dn, hints)

where we still have "dn" hardcoded.
Here we should have something similar to:

    def list_users_in_group(self, group_id, hints):
        users = []
        for user_key in self.group.list_group_users(group_id):
            if self.conf.ldap.group_members_are_ids:
                user_id = user_key
            else:
                user_id = self.user._dn_to_id(user_key)

[Steve Martinelli (stevemar)]

Patch for issue in comment #12: https://review.openstack.org/#/c/291497/1

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/291497
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=5fa308439ffa3161652718fda4e0607ff0e25126
Submitter: Jenkins
Branch: master

commit 5fa308439ffa3161652718fda4e0607ff0e25126
Author: Andrey <email address hidden>
Date: Thu Mar 10 16:04:04 2016 -0800

    Fixed user in group participance

    When a group has an assignment instead of user,
    Keystone should use proper attribute while searching in
    groups DN in LDAP. Originaly DN is hardcoded.

    Change-Id: I57bf78fffbd5df0458f3c633108a625a93ed33a6
    Closes-bug: #1526462

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/376627

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/376675

[OpenStack Infra (hudson-openstack)]

Change abandoned by Steve Martinelli (<email address hidden>) on branch: stable/liberty
Review: https://review.openstack.org/376627
Reason: stable team -1'ed so i'm abandoning

[OpenStack Infra (hudson-openstack)]

Change abandoned by Steve Martinelli (<email address hidden>) on branch: stable/liberty
Review: https://review.openstack.org/376675
Reason: stable team -1'ed so i'm abandoning