LDAP backend fails when connecting to Active Directory root DN

Bug #1233365 reported by Allison Grimes
30
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
High
Dolph Mathews
Havana
Won't Fix
High
Unassigned

Bug Description

When using the LDAP backend and connecting to Active Directory, trying to use the root DN (dc=example,dc=com) as the user_tree_dn (or tenant/role_tree_dn) fails with "Authorization Failed: Unable to communicate with identity service: {"error": {"message": "An unexpected error prevented the server from fulfilling your request. {'info': '000004DC: LdapErr: DSID-0C0906E8, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v1db1', 'desc': 'Operations error'}", "code": 500, "title": "Internal Server Error"}}. (HTTP 500)".

This is because python-ldap chases all referrals with anonymous access, which is disabled by default in AD for security reasons. Adding a line in core.py under ldap.initialize to not chase referrals (self.conn.set_option(ldap.OPT_REFERRALS, 0)) gets around this error, but then we get "AttributeError: 'list' object has no attribute 'iteritems'" in search_s. This is because while the referrals aren't chased, they still show up in the results list. The keystone code can't seem to handle the format the referrals come in. I was able to work around this by adding an if statement before o.append to ignore the referral results (if type(dn) is not NoneType). I also added "from types import *" in the beginning of core.py.

I'm sure this isn't the best workaround for everybody, but in general I think there should be an option in keystone.conf to enable or disable chasing of referrals. If it is disabled, then the previous ldap option should be set and something should be done to remove the referrals from the results list.

Edit: I'm using the Grizzly packages from the Ubuntu Cloud Archive on Ubuntu 12.04.

Tags: ldap
Adam Young (ayoung)
Changed in keystone:
assignee: nobody → Adam Young (ayoung)
description: updated
Dolph Mathews (dolph)
tags: added: grizzly-backport-potential havana-backport-potential
Thierry Carrez (ttx)
tags: added: havana-rc-potential
removed: havana-backport-potential
Thierry Carrez (ttx)
tags: added: havana-backport-potential
removed: havana-rc-potential
Adam Young (ayoung)
Changed in keystone:
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Craig Jellick (craig-jellick) wrote :

We've observed this same behavior.

You can turn off chasing referrals in /etc/openldap/ldap.conf by adding:
REFERRALS off

However, you still need to add the
if dn is not None:
line.

Does it make more sense to configure the referral chasing in OpenStack or to leave it to the configuration of the underlying openldap client configuration?

Revision history for this message
David Bingham (wwriverrat) wrote :

Humble question: Could the "if dn is not None" line be added regardless of whether config is added (at least in the short term)? I'm having trouble seeing a case (AD, no AD, or any other config) of where a None dn could be used. Hoping others out there know more of the big picture. :-)

Revision history for this message
Allison Grimes (alliegintha703) wrote :

I agree with David. As far as the referral chasing config, I think it should be an option in OpenStack, because there are use cases for referrals outside of OpenStack, it just so happens that it is a somewhat broken concept with LDAP clients because they only bind anonymously to the referrals. The referrals work fine with Windows AD clients because they know the right credentials to use. Leaving the config at the AD/LDAP client level would require making a change (solely for the purpose of OpenStack) that could affect the rest of an enterprise's infrastructure.

Revision history for this message
Craig Jellick (craig-jellick) wrote :

Tweaking /etc/openldap/ldap.conf as I suggested changes the setting of the openldap _client_ that Keystone uses to connect to the AD server. It wouldn't impact an enterprise's infrastructure.

That said, I've changed my mind and agree with you that it should be just another config option inside of keystone.conf. I think that I just temporarily fell in love with our sweet workaround.

Revision history for this message
Allison Grimes (alliegintha703) wrote :

Ah ok, I see what you're saying. I didn't have openldap installed, so I didn't have that file. When I installed it and made that change, even though I wasn't using it for anything, it did work.

On another note, I have a separate Havana setup and I made these same changes that worked in Grizzly, but the fixes don't seem to work anymore. I keep getting "Could not find user, OpenStackAdmin. (HTTP 401)". This seems to be related to the referrals again, because when I change the user_tree_dn to a different OU it works fine.

Revision history for this message
Allison Grimes (alliegintha703) wrote :

Oops, I guess I had been staring at this for too long and made a dumb mistake - forgot to change the query scope to sub. I got it working in Havana now, I just had to add the line to ignore None type DNs. I did not need to set the option to not chase referrals, so I guess that part got fixed at some point between Grizzly and Havana.

Revision history for this message
Dolph Mathews (dolph) wrote :

Any update on this?

Changed in keystone:
milestone: none → icehouse-3
Revision history for this message
Dolph Mathews (dolph) wrote :
Revision history for this message
Craig Jellick (craig-jellick) wrote :

We have a patch but it isn't quite upstream-ready. Let me see if I can get bandwidth next week to get it cleaned up and submittable.

Revision history for this message
Dolph Mathews (dolph) wrote :

Thanks Craig!

Dolph Mathews (dolph)
Changed in keystone:
milestone: icehouse-3 → icehouse-rc1
Revision history for this message
Dolph Mathews (dolph) wrote :

Craig: any word on that patch? I'd love to have this fixed before icehouse's release

Revision history for this message
Craig Jellick (craig-jellick) wrote :

Yes, will have a patch submitted later today or tomorrow.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/78521

Changed in keystone:
assignee: Adam Young (ayoung) → Craig Jellick (craig-jellick)
status: Confirmed → In Progress
Revision history for this message
Dolph Mathews (dolph) wrote :

Craig: thanks!

Changed in keystone:
assignee: Craig Jellick (craig-jellick) → Dolph Mathews (dolph)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystone (master)

Reviewed: https://review.openstack.org/78521
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9c15b73f8361ce8606a531b5765c94b3927d99c4
Submitter: Jenkins
Branch: master

commit 9c15b73f8361ce8606a531b5765c94b3927d99c4
Author: Craig Jellick <email address hidden>
Date: Wed Mar 5 18:58:57 2014 -0700

    Ability to turn off ldap referral chasing

    When getting an ldap connection, check a config property to see if
    referral chasing should be turned off.

    Co-Authored-By: Dolph Mathews <email address hidden>
    Change-Id: I3bb5cc11b3c5c9108cfa656d5f5de1eaeb103fe5
    Closes-Bug: 1233365

Changed in keystone:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to keystone (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/82090

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to keystone (master)

Reviewed: https://review.openstack.org/82090
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=1632e0f5c1da95bc1d5f010bed82146ae06f3d04
Submitter: Jenkins
Branch: master

commit 1632e0f5c1da95bc1d5f010bed82146ae06f3d04
Author: Dolph Mathews <email address hidden>
Date: Fri Mar 21 08:23:44 2014 -0500

    update sample conf

    Change-Id: Ic80e79ff412a0156a94b2992efb9ec9487b41a2c
    Partial-Bug: 1293781
    Related-Bug: 1233365

Thierry Carrez (ttx)
Changed in keystone:
status: Fix Committed → Fix Released
Alan Pevec (apevec)
tags: removed: grizzly-backport-potential
Thierry Carrez (ttx)
Changed in keystone:
milestone: icehouse-rc1 → 2014.1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (stable/icehouse)

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/93045

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/93060

Dolph Mathews (dolph)
tags: added: ldap
removed: activedirectory havana-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on keystone (stable/havana)

Change abandoned by Dolph Mathews (<email address hidden>) on branch: stable/havana
Review: https://review.openstack.org/93060
Reason: stable/havana is only security-supported at this point :(

Revision history for this message
Morgan Fainberg (mdrnstm) wrote :

At this time, it is outside the window to add this to Havana.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.