Problems with AD nested groups

[Jorge Merlino]

[Impact]
There are some issues with the implementation of AD nested groups from LP #1638603

It works fine when listing the groups a user belongs to, but fails when listing all members of a group. This function of listing all members is also used to check if a user belongs to a group which also fails.

[Test Plan]
Test plan consists of setting up two VMs (Windows AD Domain Controller and Ubuntu server) on a KVM host. We need a functional keystone installation on the ubuntu server and we will use regress-stack for that. The ubuntu server version and installed packages have to be adjusted to test each affected keystone version.

The low level procedure is as follows:

1. Install virt-manager on your host

sudo apt update && sudo apt install virt-manager virt-viewer qemu-kvm libvirt-daemon-system libvirt-clients

2. Download Windows Server 2022 iso image: https://www.microsoft.com/en-us/evalcenter/download-windows-server-2022

3. Start the Windows VM:

virt-install --name winserver \
  --virt-type kvm --memory 4096 --vcpus 4 \
  --disk size=60 \
  --cdrom /path/to/windows/iso/SERVER_EVAL_x64FRE_en-us.iso \
  --network network:default \
  --osinfo detect=on,require=off \
  --noautoconsole \
  --graphics spice

4. Use either the virt-manager or the remote-viewer to connect to the VMs console. The following is a sample command for the remote-viewer. You can get the VMs spice port by running:

virsh dumpxml winserver | grep graphics

Connect to VMs graphical interface (change port if needed)

remote-viewer spice://127.0.0.1:5900

5. Follow the installation in the VM. I picked the Windows Server 2022 Standard Evaluation, then Custom Install. During the installation the VM will shutdown so you will need to start it with:

virsh start winserver

Installation completes with the SConfig menu with multiple options. Use the menu items to configure the computer name, IP address, default gateway, and time/timezone. Remember the Administrator user password.

6. Install the spice-guest-tools to enable copy/paste between the Windows guest and host

Invoke-WebRequest -Uri https://www.spice-space.org/download/windows/spice-guest-tools/spice-guest-tools-latest.exe -OutFile sgtl.exe
.\sgtl.exe

An then reboot the server (run SConfig and then option 13)

7. Install and configure the Domain Controller

Install-WindowsFeature -Name AD-Domain-Services -IncludeManagementTools
Install-ADDSForest `
  -DomainName "test.local" `
  -InstallDns `
  -DomainMode WinThreshold `
  -ForestMode WinThreshold `
  -SafeModeAdministratorPassword (ConvertTo-SecureString "ao4Ahnuzah$d!eeg" -AsPlainText -Force) `
  -NoRebootOnCompletion:$false `
  -Force

VM will restart to apply changes

8. Verification

Get-WindowsFeature -Name AD-Domain-Services
nslookup test.local

9. Add groups for testing

New-ADOrganizationalUnit `
  -Name "groups" `
  -Path "DC=test,DC=local"
New-ADGroup `
  -Name "GroupA" `
  -SamAccountName GroupA `
  -GroupCategory Security `
  -GroupScope Global `
  -Path "OU=Groups,DC=test,DC=local" `
  -Description "Group A"
New-ADGroup `
  -Name "GroupB" `
  -SamAccountName GroupB `
  -GroupCategory Security `
  -GroupScope Global `
  -Path "OU=Groups,DC=test,DC=local" `
  -Description "Group B"

10. Add users for testing

New-ADUser `
  -SamAccountName "user1" `
  -UserPrincipalName "<email address hidden>" `
  -Name "User1" `
  -GivenName "User" `
  -Surname "One" `
  -Enabled $true `
  -AccountPassword (ConvertTo-SecureString "pAsswd&987!" -AsPlainText -Force) `
  -ChangePasswordAtLogon $true `
  -Path "CN=Users,DC=test,DC=local"
New-ADUser `
  -SamAccountName "user2" `
  -UserPrincipalName "<email address hidden>" `
  -Name "User2" `
  -GivenName "User" `
  -Surname "Two" `
  -Enabled $true `
  -AccountPassword (ConvertTo-SecureString "pAsswd&987!" -AsPlainText -Force) `
  -ChangePasswordAtLogon $true `
  -Path "CN=Users,DC=test,DC=local"

11. Add users to groups, and nest groups

Add-ADGroupMember `
  -Identity GroupA `
  -Members user1, GroupB
Add-ADGroupMember `
  -Identity GroupB `
  -Members user2

12. Verify

Get-ADUser -Identity "user1"
Get-ADUser -Identity "user2"
Get-ADGroup -Identity "groupA"
Get-ADGroup -Identity "groupB"

13. Download the appropriate ubuntu server version from https://ubuntu.com/download/server

14. Configure Ubuntu VM:

virt-install --name ubuntu \
  --virt-type kvm --memory 4096 --vcpus 4 \
  --disk size=50 \
  --cdrom /path/to/ubuntu/iso/ubuntu<version>.iso \
  --network network:default \
  --osinfo ubuntu<version>

Use default values and enable the SSH server.
Login via ssh to the server after installation.

15. Install packages and regress-stack

If testing UCA packages that repo should be enabled first:

sudo add-apt-repository cloud-archive:<version>

sudo snap install openstackclients
git clone https://github.com/canonical/regress-stack.git
cd regress-stack
sudo snap install astral-uv --classic
uvx pre-commit install
sudo apt install -y dpkg-dev python3-dev python-apt-dev python3-openstackclient keystone apache2 libapache2-mod-wsgi-py3 mysql-server crudini python3-ldappool
uv sync
sudo uv run regress-stack setup
sudo cp /root/auth.rc ~
sudo chown $(id -u):$(id -g) ~/auth.rc
sudo crudini --set /etc/keystone/keystone.conf identity domain_specific_drivers_enabled true

16. Create file keystone.windows.lan.conf in /etc/keystone/domains with these contents and set the windows server IP address and Administrator password

[ldap]
url = ldap://<windows_server_ip>
user = CN=Administrator,CN=Users,DC=test,DC=local
password = <windows_admin_password>
suffix = DC=test,DC=local

user_allow_create = False
user_allow_update = False
user_allow_delete = False

group_allow_create = False
group_allow_update = False
group_allow_delete = False

query_scope = sub
user_tree_dn = CN=Users,DC=test,DC=local
user_objectclass = person
user_id_attribute = cn
user_name_attribute = sAMAccountName
user_enabled_attribute = userAccountcontrol
user_enabled_invert = False
user_enabled_mask = 2
user_enabled_default = 512
group_tree_dn = OU=groups,DC=test,DC=local
group_objectclass = group
group_id_attribute = cn
group_name_attribute = sAMAccountName
group_member_attribute = member
group_members_are_ids = False

group_ad_nesting = True
[identity]
driver = ldap

17. Finish configuration
openstack domain create windows.lan
sudo systemctl restart apache2

18. Test before patch:
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupA user1
user1 not in group groupA
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupA user2
user2 not in group groupA
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupB user1
user1 in group groupB
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupB user2
user2 not in group groupB

19. Apply patch and retest:
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupA user1
user1 in group groupA
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupA user2
user2 in group groupA
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupB user1
user1 not in group groupB
$ openstack group contains user --group-domain windows.lan --user-domain windows.lan groupB user2
user2 in group groupB

[Where problems could occur]
Applications relying on the previous erroneous behavior could experience errors as the users permissions might change reflecting the actual user assigned groups.
Moreover, the nested groups being functional now could add new permissions to users as they are considered to belong to the parent groups. This can be disabled by setting group_ad_nesting to false in the keystone-ldap charm config.

[Other Info]
Packages in Questing and Resolute already have the patch. Same for flamingo in UCA

[David Wilde (dave-wilde)]

We're going to be working with AD and LDAP and will attempt to reproduce this bug at that point. Should we find an issue we will dedicate engineering time to fix it.

/Dave

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/keystone/+/951792

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/keystone/+/951792
Committed: https://opendev.org/openstack/keystone/commit/f8338be43073f23f3db64fa4ba658c3e1f554aa7
Submitter: "Zuul (22348)"
Branch: master

commit f8338be43073f23f3db64fa4ba658c3e1f554aa7
Author: Jorge Merlino <email address hidden>
Date: Wed Jun 4 13:58:17 2025 -0300

    Fix AD nested groups issues

    The implementation of AD nested groups searches works fine when
    listing the groups a user belongs to, but fails when listing all
    members of a group. This function of listing all members is also
    used to check if a user belongs to a group which also fails.
    This patch fixes the query for getting all users in a group.

    Closes-Bug: #2112477

    Depends-on: https://review.opendev.org/c/openstack/devstack/+/953569
    Depends-on: https://review.opendev.org/c/openstack/devstack/+/954914

    Change-Id: I9707e1a9bc4a334902933d6251888144f8c3bc19
    Signed-off-by: Jorge Merlino <email address hidden>

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2025.1
Review: https://review.opendev.org/c/openstack/keystone/+/960420

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 28.0.0.0rc1 Flamingo release candidate.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.2
Review: https://review.opendev.org/c/openstack/keystone/+/961413

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/keystone/+/962850

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/keystone/+/962850
Reason: stable/2024.1 branch of openstack/keystone is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/2024.1 if you want to further work on this patch.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/keystone/+/960420
Committed: https://opendev.org/openstack/keystone/commit/ba210d3e788eb7e6880ddbdcd1898349f43f9bfb
Submitter: "Zuul (22348)"
Branch: stable/2025.1

commit ba210d3e788eb7e6880ddbdcd1898349f43f9bfb
Author: Jorge Merlino <email address hidden>
Date: Wed Jun 4 13:58:17 2025 -0300

    Fix AD nested groups issues

    The implementation of AD nested groups searches works fine when
    listing the groups a user belongs to, but fails when listing all
    members of a group. This function of listing all members is also
    used to check if a user belongs to a group which also fails.
    This patch fixes the query for getting all users in a group.

    Closes-Bug: #2112477

    Depends-on: https://review.opendev.org/c/openstack/devstack/+/960683
    Depends-on: https://review.opendev.org/c/openstack/devstack/+/960684

    Change-Id: I9707e1a9bc4a334902933d6251888144f8c3bc19
    Signed-off-by: Jorge Merlino <email address hidden>
    (cherry picked from commit f8338be43073f23f3db64fa4ba658c3e1f554aa7)

[Jorge Merlino (jorge-merlino)]

debdiff for plucky

[Jorge Merlino (jorge-merlino)]

debdiff for UCA epoxy

[Jorge Merlino (jorge-merlino)]

Fixed debdiff for plucky

[Jorge Merlino (jorge-merlino)]

Fixed debdiff for UCA epoxy

[Jorge Merlino (jorge-merlino)]

debdiff for UCA dalmatian

[Jorge Merlino (jorge-merlino)]

debdiff for Noble

[Jorge Merlino (jorge-merlino)]

debdiff for UCA caracal

[Dariusz Gadomski (dgadomski)]

Thanks Jorge. I've sponsored your patch for Noble.

[Timo Aaltonen (tjaalton)]

please close the devel task if it's already fixed there

[Timo Aaltonen (tjaalton)]

Hello Jorge, or anyone else affected,

Accepted keystone into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/keystone/2:25.0.0-0ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.