Using Microsoft AD's objectGUID attribute as user_id_attribute breaks

[Lance Bragstad]

Microsoft AD has a default attribute in its schema for users and groups called objectGUID [0]. The attribute is assigned when new users and groups are created.

If we attempt to use this attribute as a user's ID by setting user_id_attribute=objectGUID, keystone throws an HTTP 404 listing users.

I was able to recreate this with using a 2016 AD server with the following keystone LDAP configuration file:

[root@overcloud-controller-0 /]# cat /etc/keystone/domains/keystone.windows.conf
[ldap]
url=ldap://192.168.1.209
user=CN=Administrator,CN=Users,DC=mycompany,DC=local
password=p@ssw0rd1
suffix=DC=mycompany,DC=local
user_tree_dn=DC=mycompany,DC=local
user_objectclass=user
user_id_attribute=objectGUID
query_scope=sub
user_name_attribute=sAMAccountName

[identity]
driver=ldap
[stack@undercloud ~]$ openstack --os-cloud overcloud user list --domain windows
ID attribute objectGUID not found in LDAP object CN=Administrator,CN=Users,DC=mycompany,DC=local (HTTP 404) (Request-ID: req-cdc056fd-2ebc-4a24-81d9-8a0948fc56bf)

The root of the issue is that keystone isn't properly decoding the value, which you can see from the logs as the ldap backend processes results from AD.

/var/log/containers/keystone/keystone.log:2020-07-31 18:58:36.319 19 WARNING keystone.common.wsgi [req-ae63fcc8-64a1-48f5-96c9-c107df9c4be0 c4c7be68b7304ba884a0d15f3f882df4 b970c795de7749af821f1d53b51fa0d5 - default default] ID attribute objectGUID not found in LDAP object CN=Administrator,CN=Users,DC=mycompany,DC=local: NotFound: ID attribute objectGUID not found in LDAP object CN=Administrator,CN=Users,DC=mycompany,DC=local

Relevant code:

https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L1328-L1330
https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L950
https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L144-L183
https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L174
https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L141
https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L81-L97

We might need to consider handling this value similar what the python-ldap community suggests:

https://mail.python.org/pipermail/python-ldap/2014q3/003410.html

[0] https://docs.microsoft.com/en-us/windows/win32/adschema/a-objectguid

[Lance Bragstad (lbragstad)]

The workaround for this issue is to not use objectGUID as the user or group ID. However, that workaround might not be applicable in all situations. For example, the default value for user_id_attribute is 'cn', but if that value spans more than 64 characters, keystone can't work with it.

(overcloud) [heat-admin@overcloud-controller-0 ~]$ openstack user list --domain tripleo
String length exceeded. The length of string '5e2bd347ac6e4e22ac75a440800d1d179d0f3d153b3d466b96c7ac50301512539a4367314f0d49c3ad33e85c6e1cafe1 5e2bd347ac6e4e22ac75a440800d1d' exceeds the limit of column local_id(CHAR(64)). (HTTP 400) (Request-ID: req-6e5082d5-b768-4376-8990-f3004eae4617)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/744251

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/744251
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8bf222ac5d390e25d306d35f69bd958b18bee4d8
Submitter: Zuul
Branch: master

commit 8bf222ac5d390e25d306d35f69bd958b18bee4d8
Author: Lance Bragstad <email address hidden>
Date: Fri Jul 31 15:28:15 2020 -0500

    Properly handle octet (byte) strings when converting LDAP responses

    If LDAP returns a UUID as an octet string the LDAP driver will fail to
    convert it to something meaningful. The error usually looks something
    like:

      ID attribute objectGUID not found in LDAP object

    Microsoft AD's `objectGUID` parameter is stored and transmitted as an
    octet string [0]. If you attempt to use the `objectGUID` to generate
    user or group IDs, you'll get an HTTP 404 because keystone can't decode
    it properly. This is unfortunate because `objectGUID` are a fixed
    length, UUID format, and ideal for generating IDs in keystone. As
    opposed to using the object's CN, which is variable length, and can
    generate hashes that are larger than keystone's database table limit for
    user IDs.

    [0] https://docs.microsoft.com/en-us/windows/win32/ad/reading-an-objectampaposs-objectguid-and-creating-a-string-representation-of-the-guid

    Change-Id: Id80b17bdff015e10340e636102576b7435bd564f
    Closes-Bug: 1889936

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/747924

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/747927

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/747928

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/747929

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/747931

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/747924
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=d5870f69c12c034dd97b164345e85e6259ee7abe
Submitter: Zuul
Branch: stable/ussuri

commit d5870f69c12c034dd97b164345e85e6259ee7abe
Author: Lance Bragstad <email address hidden>
Date: Fri Jul 31 15:28:15 2020 -0500

    Properly handle octet (byte) strings when converting LDAP responses

    If LDAP returns a UUID as an octet string the LDAP driver will fail to
    convert it to something meaningful. The error usually looks something
    like:

      ID attribute objectGUID not found in LDAP object

    Microsoft AD's `objectGUID` parameter is stored and transmitted as an
    octet string [0]. If you attempt to use the `objectGUID` to generate
    user or group IDs, you'll get an HTTP 404 because keystone can't decode
    it properly. This is unfortunate because `objectGUID` are a fixed
    length, UUID format, and ideal for generating IDs in keystone. As
    opposed to using the object's CN, which is variable length, and can
    generate hashes that are larger than keystone's database table limit for
    user IDs.

    [0] https://docs.microsoft.com/en-us/windows/win32/ad/reading-an-objectampaposs-objectguid-and-creating-a-string-representation-of-the-guid

    Change-Id: Id80b17bdff015e10340e636102576b7435bd564f
    Closes-Bug: 1889936
    (cherry picked from commit 8bf222ac5d390e25d306d35f69bd958b18bee4d8)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/747927
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=9beb3a58011596af5c704e3930a0f01d67f5c845
Submitter: Zuul
Branch: stable/train

commit 9beb3a58011596af5c704e3930a0f01d67f5c845
Author: Lance Bragstad <email address hidden>
Date: Fri Jul 31 15:28:15 2020 -0500

    Properly handle octet (byte) strings when converting LDAP responses

    If LDAP returns a UUID as an octet string the LDAP driver will fail to
    convert it to something meaningful. The error usually looks something
    like:

      ID attribute objectGUID not found in LDAP object

    Microsoft AD's `objectGUID` parameter is stored and transmitted as an
    octet string [0]. If you attempt to use the `objectGUID` to generate
    user or group IDs, you'll get an HTTP 404 because keystone can't decode
    it properly. This is unfortunate because `objectGUID` are a fixed
    length, UUID format, and ideal for generating IDs in keystone. As
    opposed to using the object's CN, which is variable length, and can
    generate hashes that are larger than keystone's database table limit for
    user IDs.

    [0] https://docs.microsoft.com/en-us/windows/win32/ad/reading-an-objectampaposs-objectguid-and-creating-a-string-representation-of-the-guid

    Conflicts:
          keystone/identity/backends/ldap/common.py
          Due to python3 string detection differences between ussuri and
          train.

    Change-Id: Id80b17bdff015e10340e636102576b7435bd564f
    Closes-Bug: 1889936
    (cherry picked from commit 8bf222ac5d390e25d306d35f69bd958b18bee4d8)
    (cherry picked from commit d5870f69c12c034dd97b164345e85e6259ee7abe)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 16.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 17.0.1 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Douglas Mendizábal <email address hidden>" on branch: stable/rocky
Review: https://review.opendev.org/c/openstack/keystone/+/747929
Reason: Abandoning unmerged stable/rocky changes.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Douglas Mendizábal <email address hidden>" on branch: stable/queens
Review: https://review.opendev.org/c/openstack/keystone/+/747931
Reason: Abandoning unmerged stable/queens changes.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone stein-eol release.