sssd can't parse GPT.INI if it contains non-ASCII characters

[Sergio Durigan Junior]

[ Impact ]

When the Active Directory Domain Controller uses GPO, and if sssd is configured to enforce GPO permissions, and if the GPT.INI file (which is part of the server's GPO policies, and served to the client when an authentication happens) contains non-ASCII characters, then the user can experience a crash on sssd and the authentication will fail.

[ Test Plan ]

The test case for this bug is a bit complex, but I will do my best to describe it in detail.

We need to setup one VM running a Samba Active Directory Domain Controller (AD DC), which will be our main server. This needs only to be configured once. We then need to setup another VM which will join our AD DC realm using realmd and sssd.

== Virtual Network setup ==

The first step is to create a dedicated virtual network for our tests. This is not strictly mandatory, but it will simplify things. The best way to create this network is via virt-manager. Install it if needed, open the program and select the "QEMU/KVM" line. Go to Edit > Connection Details > Virtual Networks, click on the "+" icon (bottom left), give this network a name (I will use "sssdad"), make sure that "Enable IPv4" and "Enable DHCPv4" are selected (under "IPv4 configuration"). Go to "DNS domain name" and select "Custom". For the domain name, type "test.lan". Click on "Finish".

Take note of the "Device" name that shows up after you create the network. We will use it when creating the VMs. For this test plan, let's assume the device name is "virbr1".

== Samba AD DC VM setup ==

We need to setup a Samba AD DC server. It doesn't matter which Ubuntu release we use for it. Note that we have to use "-n virbr1" when creating the VM, otherwise it won't use our virtual network.

$ lxc launch ubuntu-daily:jammy sambadc -n virbr1 --vm
$ lxc shell sambadc
server# apt update
server# ip a

Make sure to grab this VM's IP address.

server# cat >> /etc/hosts << _EOF_
IP_ADDRESS_HERE sambadc sambadc.test.lan
_EOF_
server# reboot
$ lxc shell sambadc
server# apt install -y samba winbind

server# mv /etc/samba/smb.conf /etc/samba/smb.conf.bkp
server# samba-tool domain provision --use-rfc2307 --realm TEST.LAN --domain TEST --server-role dc --dns-backend SAMBA_INTERNAL --adminpass MyPassword1

server# systemctl mask smbd.service nmbd.service winbind.service
server# systemctl disable --now smbd.service nmbd.service winbind.service
server# systemctl unmask samba-ad-dc.service
server# systemctl enable --now samba-ad-dc.service

We now have to adjust the DNS server settings of the server. We are going to disable systemd-resolved.service and use samba as our DNS service.

You will notice that the samba-tool command issued above has added 127.0.0.53 as the "dns forwarder" in /etc/samba/smb.conf. Edit the file and set the forwarder to be the virtual network's DNS resolver -- it should be the same as IP_ADDRESS_HERE, but ending in .1.

server# systemctl disable --now systemd-resolved.service
server# unlink /etc/resolv.conf
server# cat > /etc/resolv.conf << _EOF_
nameserver IP_ADDRESS_HERE
search test.lan
_EOF_
server# reboot

This should be enough to configure Samba as an AD DC. While at it, create a test user that will later be used to trigger the bug.

server# samba-tool user create testuser MyUserPassword1

We also need to insert non-ASCII characters into the GPT.INI file, and make sure that we also create a GptTmpl.inf file that will allow us to login as the user.

server# cd /var/lib/samba/sysvol/test.lan/Policies/\{31B2F340-016D-11D2-945F-00C04FB984F9\}/
server# echo -n -e "\ndisplayName=crásher\n" | iconv -f UTF-8 -t CP850 >> GPT.INI
server# mkdir -p MACHINE/Microsoft/Windows\ NT/SecEdit/
server# cat > MACHINE/Microsoft/Windows\ NT/SecEdit/GptTmpl.inf << _EOF_
[Unicode]
Unicode=yes
[Version]
signature="\$CHICAGO\$"
Revision=1
_EOF_

== VM AD client setup ==

Let's configure a VM to act as an AD client.

$ lxc launch ubuntu-daily:jammy sssdclient -n virbr1 --vm
$ lxc shell sssdclient

Before anything else, let's configure this VM to use the Samba AD DC VM as its DNS resolver. In the excerpt below, IP_ADDRESS_HERE refers to the IP address of the Samba AD DC VM (configured in the last section).

client# systemctl disable --now systemd-resolved.service
client# unlink /etc/resolv.conf
client# cat > /etc/resolv.conf << _EOF_
nameserver IP_ADDRESS_HERE
search test.lan
_EOF_

client# apt update
client# apt install -y sssd-ad sssd-tools realmd adcli sssd-dbus
client# pam-auth-update --enable mkhomedir

We can now check if our container can detect the AD DC:

client# realm -v discover sambadc.test.lan
 * Resolving: _ldap._tcp.sambadc.test.lan
 * Resolving: sambadc.test.lan
 * Performing LDAP DSE lookup on: 192.168.101.142
 * Successfully discovered: test.lan
test.lan
  type: kerberos
  realm-name: TEST.LAN
  domain-name: test.lan
  configured: no
  server-software: active-directory
  client-software: sssd
  required-package: sssd-tools
  required-package: sssd
  required-package: libnss-sss
  required-package: libpam-sss
  required-package: adcli
  required-package: samba-common-bin

You can also confirm that the "realm -v discover" works on the domain name:

client# realm -v discover test.lan
 * Resolving: _ldap._tcp.test.lan
 * Performing LDAP DSE lookup on: 192.168.101.142
 * Successfully discovered: test.lan
test.lan
  type: kerberos
  realm-name: TEST.LAN
  domain-name: test.lan
  configured: no
  server-software: active-directory
  client-software: sssd
  required-package: sssd-tools
  required-package: sssd
  required-package: libnss-sss
  required-package: libpam-sss
  required-package: adcli
  required-package: samba-common-bin

And we can join the realm:

client# realm -v join test.lan

You will be prompted the password for the Administrator user. It is MyPassword1. If everything worked OK, you should now be able to list the information from the Administrator user:

client# getent passwd <email address hidden>
<email address hidden>:*:1522000500:1522000513:Administrator:/<email address hidden>:/bin/bash

== Reproducing the bug ==

Make sure you can obtain the passwd information for the test user we've created in the Samba AD DC server.

client# getent passwd <email address hidden>
<email address hidden>:*:1522001104:1522000513:testuser:/<email address hidden>:/bin/bash

Now, try to login as the user:

client# login <email address hidden>
Password: MyUserPassword1

System error

As can be seen, the user will get a system error when trying to login in a default AD DC setup. In order to confirm that the problem is indeed caused by the non-ASCII characters present in the GPT.INI file, you can inspect the /var/log/sssd/gpo_child.log file. You should find the messages saying "Invalid or incomplete multibyte or wide character". If you would like to check that the login works without the non-ASCII characters, simply edit the GPT.INI (on the server side) and remove its last line (starting with "displayName=").

Once you upgrade the sssd package to the fixed version, you should be able to confirm that the login works even when GPT.INI contains non-ASCII characters. Upon inspecting /var/log/sssd/gpo_child.log, this time you will see a message like this:

(2022-06-22 14:14:12): [gpo_child[9778]] [gpo_sanitize_buffer_content] (0x1f7c0): [RID#7] Value for key 'displayName' contains non-ascii symbol. Replacing with '?'

If you're testing this on Focal, you will also need to edit /etc/sssd/sssd.conf and, inside the "[domain...]" section, add the following line:

debug = 2

Otherwise the /var/log/sssd/gpo_child.log file will be empty.

Be aware of the client-side caching when testing sssd. Sometimes the authentication can work when it's not expected to, or vice-versa. It's recommended to completely remove sssd's cache before testing a new scenario:

client# sssctl cache-remove -o -p -s

[ Where problems could occur ]

Although the problem seems somewhat simple to overcome, the patch is not entirely trivial. It involves parsing the GPT.INI file, determining the location of the non-ASCII characters, and finally ignoring them. It is not likely, but also not impossible that this new parser doesn't handle some obscure corner case very well, and we end up discarding more than we'd like from GPT.INI.

[Sergio Durigan Junior (sergiodj)]

This has been fixed upstream by https://github.com/SSSD/sssd/commit/d241b55291419753ce3e961a1b201d62f7851513.

The fix is part of sssd 2.7.1-2, which is currently sitting on kinetic-proposed.

[Steve Langasek (vorlon)]

Hello Sergio, or anyone else affected,

Accepted sssd into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sssd/2.6.3-1ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

why has impish been skipped for this SRU?

[Andreas Hasenack (ahasenack)]

Jammy verification

with the jammy packages, I get the system error upon login:

root@j-client:~# login <email address hidden>
Password:

System error

And the logs confirm it's an encoding error:
   * (2022-07-01 23:53:45): [gpo_child[4860]] [ad_gpo_parse_ini_file] (0x0020): [RID#7] ini_config_file_open failed [84][Invalid or incomplete multibyte or wide character]

I then upgrade to the packages in proposed:

root@j-client:~# apt-cache policy sssd
sssd:
  Installed: 2.6.3-1ubuntu3.1
  Candidate: 2.6.3-1ubuntu3.1
  Version table:
 *** 2.6.3-1ubuntu3.1 500
        500 http://archive.ubuntu.com/ubuntu jammy-proposed/main amd64 Packages
(...)

And login this time succeeds:

root@j-client:~# login <email address hidden>
Password:
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-39-generic x86_64)
(...)
Creating directory '/<email address hidden>'.
<email address hidden>@j-client:~$

Log shows the non-ascii sanitization that the code is doing now:
(2022-07-01 23:58:32): [gpo_child[6305]] [gpo_sanitize_buffer_content] (0x1f7c0): [RID#5] Value for key 'displayName' contains non-ascii symbol. Replacing with '?'

Jammy verification succeeded.

[Chris Halse Rogers (raof)]

An upload of sssd to focal-proposed has been rejected from the upload queue for the following reason: "Patch lp1934997-authentication-fails-gpo-non-existent.patch appears to have unrelated change to Makefile.am; I'm unsure if this makes any functional change to the build, but would appear to be a bit of a landmine for potential future uuploads".

[Chris Halse Rogers (raof)]

Hello Sergio, or anyone else affected,

Accepted sssd into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sssd/2.2.3-3ubuntu0.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Chris Halse Rogers (raof)]

Impish will reach EOL before this SRU would be eligible for release; approving without an impish upload.

[Andreas Hasenack (ahasenack)]

Focal verification

Reproducing the bug with the non-proposed focal packages:

root@sssdclient:~# apt-cache policy sssd
sssd:
  Installed: 2.2.3-3ubuntu0.8
  Candidate: 2.2.3-3ubuntu0.8
  Version table:
 *** 2.2.3-3ubuntu0.8 500
        500 http://br.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        500 http://br.archive.ubuntu.com/ubuntu focal-security/main amd64 Packages
        100 /var/lib/dpkg/status

Login fails:

root@sssdclient:~# login <email address hidden>
Password:

System error

And the logs confirm it's the non-ascii character in GPT.INI (after bumping sssd log level to 2):
(Wed Jul 6 13:56:44 2022) [gpo_child[3382]] [ad_gpo_parse_ini_file] (0x0020): ini_config_file_open failed [84][Invalid or incomplete multibyte or wide character]
(Wed Jul 6 13:56:44 2022) [gpo_child[3382]] [ad_gpo_parse_ini_file] (0x0020): Error encountered: 84.
(Wed Jul 6 13:56:44 2022) [gpo_child[3382]] [perform_smb_operations] (0x0020): Cannot parse ini file: [84][Invalid or incomplete multibyte or wide character]
(Wed Jul 6 13:56:44 2022) [gpo_child[3382]] [main] (0x0020): perform_smb_operations failed.[84][Invalid or incomplete multibyte or wide character].
(Wed Jul 6 13:56:44 2022) [gpo_child[3382]] [main] (0x0020): gpo_child failed!

I then updated the sssd packages to the ones from focal-proposed:
root@sssdclient:~# apt-cache policy sssd
sssd:
  Installed: 2.2.3-3ubuntu0.9
  Candidate: 2.2.3-3ubuntu0.9
  Version table:
 *** 2.2.3-3ubuntu0.9 500
        500 http://br.archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages
        100 /var/lib/dpkg/status

Retry the login, which works this time:
root@sssdclient:~# login <email address hidden>
Password:
Welcome to Ubuntu 20.04.4 LTS (GNU/Linux 5.4.0-1070-kvm x86_64)
(...)
Creating directory '/<email address hidden>'.

<email address hidden>@sssdclient:~$ id
uid=906401103(<email address hidden>) gid=906400513(domain <email address hidden>) groups=906400513(domain <email address hidden>)

And the log only has this entry this time:
(Wed Jul 6 14:00:07 2022) [gpo_child[4779]] [gpo_sanitize_buffer_content] (0x0040): Value for key 'displayName' contains non-ascii symbol. Replacing with '?'

focal verification succeeded.

[Launchpad Janitor (janitor)]

This bug was fixed in the package sssd - 2.6.3-1ubuntu3.1

---------------
sssd (2.6.3-1ubuntu3.1) jammy; urgency=medium

  * d/p/lp1934997-authentication-fails-gpo-non-existent.patch:
    Fix authentication failure when GPO is enabled and
    SecEdit/GptTmpl.inf is missing (LP: #1934997).
  * d/p/lp1979350-GPO-ignore-non-ascii-symbols-in-GPT.INI.patch:
    Ignore non-ASCII characters in GPT.INI. (LP: #1979350)
  * Fix "sssctl analyze" (LP: #1979453)
    [ Timo Aaltonen ]
    - d/{python3-libipa-hbac.install,python3-sss.install,
      sssd-tools.install,rules}: Fix Python installation directory
      path.
    [ Sergio Durigan Junior ]
    - d/p/lp1979453-fix-shebang-on-sss_analyze.patch: Fix shebang on
      sss_analyze.

 -- Sergio Durigan Junior <email address hidden> Wed, 22 Jun 2022 12:31:22 -0400

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for sssd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package sssd - 2.2.3-3ubuntu0.9

---------------
sssd (2.2.3-3ubuntu0.9) focal; urgency=medium

  * d/p/lp1934997-authentication-fails-gpo-non-existent.patch:
    Fix authentication failure when GPO is enabled and
    SecEdit/GptTmpl.inf is missing (LP: #1934997).
  * d/p/lp1979350-GPO-ignore-non-ascii-symbols-in-GPT.INI.patch:
    Ignore non-ASCII characters in GPT.INI. (LP: #1979350)

 -- Sergio Durigan Junior <email address hidden> Tue, 21 Jun 2022 14:29:52 -0400