gdm-smartcard not passing successful authentication to desktop at system logon

[Neil Webster]

[ Impact ]

gdm-smartcard returns a Permission denied when logging in with an user name:

+ pamtester -v gdm-smartcard ubuntu authenticate
pamtester: invoking pam_start(gdm-smartcard, ubuntu, ...)
pamtester: performing operation - authenticate
PIN for Test Organization Root Tr Token:
pamtester: Permission denied

Using an empty user name works instead.

[ Test case ]

1. Use a smartcard to login in gdm

This can also be simulated via:

# Must be ran as user
sudo apt install pamtester
pamtester -v gdm-smartcard $USER authenticate

Expected output is
+ pamtester -v gdm-smartcard ubuntu authenticate
pamtester: invoking pam_start(gdm-smartcard, ubuntu, ...)
pamtester: performing operation - authenticate
PIN for Test Organization Sub Int Token:
pamtester: successfully authenticated

---

Alternatively, if no smartcard or hardware is available, this can be tested and simulated using these scripts (they will reset the system setup at each run, but it's suggested to run them in a VM, lxd container or in a test installation):
 https://gist.github.com/3v1n0/287d02ca8e03936f1c7bba992173d47a

- sudo apt install gdm3 pamtester softhsm2 openssl wget sssd gnutls-bin && \
  sudo apt-mark auto gdm3 pamtester softhsm2 openssl wget sssd gnutls-bin
- wget https://gist.github.com/3v1n0/287d02ca8e03936f1c7bba992173d47a/raw/sssd-gdm-smartcard-pam-auth-tester.sh
- sudo sssd-gdm-smartcard-pam-auth-tester.sh

The script will generate some fake CA authority, issue some certificates, will install them in some software-based smartcards (using softhsm2) and test that they work properly to login with gdm-smartcard.

Using `WAIT` environment variable set (to any value) will make it to restart gdm at each iteration so that an user can try to access, using the username that launched the script and the pin of 123456.

[ Regression Potential ]

A root user could access to pam_sss, however it's the responsibility of such module to block such access.

---

For information I've repeated this entire process on RHEL8 and it works there, it also was working upon last test on Ubuntu 20.04

Releases: 22.04 LTS and 22.10
Package Version (for reporting purposes): 43.0-1ubuntu1

Background:

System has been configured with sssd, krb5 and pkinit. All of these packages confirm a successful connection to the Active Directory Domain Controller. I have a YubiKey which has a CA generated certificate on it (with all required uses/capabilities including sign) and this is working fine on other systems.

Expected Behavior:

Insert YubiKey before boot. At the logon window press enter on the Username field. Select the certificate, enter PIN when prompted. Authenticate to desktop.

What is happening:

Insert YubiKey before boot. At the logon window press enter on the Username field. Select the certificate, enter PIN when prompted. Returns to Username field and does not log in.

Other:

This is a clean install of 22.10 updated to 16 Dec 2022. I also tried the same thing with 22.04 LTS just in case.

I have enabled level 6 logging on SSSD and can confirm that side of the entire process is fine. I can also log on with a password and do a kinit <username> and get a valid kerberos ticket.

With some systematic tests, I managed to pinpoint the login is failing after gdm-smartcard reports a successful login:

Dec 16 10:25:43 ubu-vm-2022 gdm-smartcard]: gkr-pam: stashed password to try later in open session
Dec 16 10:26:22 ubu-vm-2022 gdm-smartcard]: pam_sss(gdm-smartcard:auth): authentication success; logname= uid=0 euid=0 tty=/dev/tty1 ruser= rhost= <email address hidden>

I did not have this problem on 20.04.

ProblemType: BugDistroRelease: Ubuntu 22.10
Package: gdm3 43.0-1ubuntu1
ProcVersionSignature: Ubuntu 5.19.0-26.27-generic 5.19.7
Uname: Linux 5.19.0-26-generic x86_64
ApportVersion: 2.23.1-0ubuntu3
Architecture: amd64
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Fri Dec 16 11:43:25 2022
InstallationDate: Installed on 2022-12-16 (0 days ago)
InstallationMedia: Ubuntu 22.10 "Kinetic Kudu" - Release amd64 (20221020)SourcePackage: gdm3
UpgradeStatus: No upgrade log present (probably fresh install)

[Neil Webster (neilw-nmw)]

Some additional information.

Authenticating with a password, then running kinit -X in a privileged terminal window DOES work, everything matches and I get a kerberos ticket.

Deleting everything from the YubiKey and issuing a brand new certificate (instead of having 2 on the key) also doesn't work.

sssd_authenticate.me.uk.log doesn't show any errors and does not update when using tail -f during authentication

sssd_pam.log doesn't show any errors and does not update when using tail -f during authentication

p11_child.log shows informational events when selecting the certificate, no errors shown.

krb5_child.log (which I have attached) shows no errors until I cancel the password prompt.

auth.log contains only two lines and this occurs after I type the PIN and press enter:

Dec 16 13:47:42 ubu2210 gdm-smartcard]: pam_sss(gdm-smartcard:auth): received for user <email address hidden>: 7 (Authentication failure)
Dec 16 13:47:42 ubu2210 gdm-smartcard]: gkr-pam: no password is available for user

kern.log shows no errors (apparmor ALLOWED on all requests)

syslog shows no errors I can spot. This will be attached on the next comment.

[Marco Trevisan (Treviño) (3v1n0)]

I feel this is more a SSSD issue, so let's move it there.

[Marco Trevisan (Treviño) (3v1n0)]

Can you try to see what you get with pamtester, by using

  pamtester -v gdm-smartcard $USER authenticate

Also try with:

  pamtester -v gdm-smartcard "" authenticate

[Marco Trevisan (Treviño) (3v1n0)]

Ah, please also provide your auth.log as it may include some more infos

[Neil Webster (neilw-nmw)]

Thanks for the updates. I'll get this extra testing completed and come back to you.

[Neil Webster (neilw-nmw)]

auth.log attached which contains everything from the moment the YubiKey is inserted and PIN entered. I then removed it and typed the password and logged in.

[Neil Webster (neilw-nmw)]

pamtester results attached for both variations on the command ($USER and "")

The result is permission denied as shown in your testing.

[Marco Trevisan (Treviño) (3v1n0)]

I think we're talking of a different bug here though, because in your log here pam_sss via gdm-smartcard always returns an authentication failure:

Jan 31 22:22:25 lnx-ubu-2110 gdm-smartcard]: pam_sss(gdm-smartcard:auth): authentication failure; logname= uid=0
euid=0 tty=/dev/tty1 ruser= rhost= <email address hidden>
Jan 31 22:22:25 lnx-ubu-2110 gdm-smartcard]: pam_sss(gdm-smartcard:auth): received for user
<email address hidden>: 15 (Authentication service cannot retrieve user credentials)

While, in the initial description of this bug the problem was that pam_sss returned "success", but still `gdm-smartcard` was returning a failure:

Dec 16 10:26:22 ubu-vm-2022 gdm-smartcard]: pam_sss(gdm-smartcard:auth): authentication success; logname= uid=0 euid=0 tty=/dev/tty1 ruser= rhost= <email address hidden>

And this is the kind of error we're fixing here, so please in case the fix that is coming won't fix your case re-open a new bug against SSSD, because the bug we're fixing here is when:

1. pam_sss exits with success
2. gdm-smartcard still gives a permission error

[Launchpad Janitor (janitor)]

This bug was fixed in the package gdm3 - 43.0-3ubuntu1

---------------
gdm3 (43.0-3ubuntu1) lunar; urgency=medium

  [ Simon McVittie ]
  * d/tests: Don't reset root password.
    Even if the root password is blank, we want to assert that
    authentication still doesn't succeed, because we explicitly don't allow
    smart card authentication as root.
  * d/tests: Explicitly use blank input when checking for blank password.
    Otherwise we could block indefinitely when running tests that have an
    interactive console available.

  [ Marco Trevisan (Treviño) ]
  * debian/tests/control: Add explicit dependency on libpam-sss.
    Even though it could be an implicit one it's still what we're testing
  * debian/tests/sssd-gdm-smartcard-pam-auth-tester.sh: Some minor cleanups
  * debian/tests/control,
    debian/tests/sssd-gdm-smartcard-pam-auth-tester-env.sh Manually use sudo
    as ubuntu autopkgtest does not support needs-sudo yet
  * debian/gdm3.install: Do not list config files, just install all gdm3 ones
    That's used as is in ubuntu (where we install more data and we use the
    upstream `custom.conf` name for config file), so we don't have to diverge.
  * Merge with debian, remaining changes:
    + readme.debian: update for correct paths in ubuntu
    + control.in:
      - don't recommend desktop-base
      - depend on bash for config_error_dialog.patch
      - update vcs field
    + rules:
      - don't override default user/group
      - -dgdm-xsession=true to install upstream xsession script
      - override dh_installinit with --no-start to avoid session being killed
    + rules, readme.debian, gdm3.8.pod:
      use upstream custom.conf instead of daemon.conf
    + gdm3.{postinst,postrm}: rename user and group back to gdm
    + debian/tests/control:
      - Use gdm user name
      - Use needs-root instead of needs-sudo (to remove when ubuntu autopkgtest
        will be updated to include such feature)
    + debian/tests/sssd-gdm-smartcard-pam-auth-tester-env.sh:
      - Added to use needs-root autopkgtest instead of needs-sudo
    + gdm3.*.pam: make pam_env read ~/.pam_environment, as we use in g-c-c
      settings
    + gdm3.install:
      - don't install debian/xsession
    + add run_xsession.d.patch
    + add xresources_is_a_dir.patch
      - fix loading from /etc/x11/xresources/*
    + add nvidia_prime.patch:
      - add hook to run prime-offload (as root) and prime-switch if
        nvidia-prime is installed
    + add revert_override_lang_with_accountservices.patch:
      - on ubuntu accountservices only stores the language and not the
        full locale as needed by lang.
    + add dont_set_language_env.patch:
      - don't run the set_up_session_language() function, since it
        overrides variable values set by ~/.pam_environment
    + add config_error_dialog.patch:
      - show warning dialog in case of error in ~/.profile etc. and
        don't let a syntax error make the login fail
    + add debian/patches/revert_nvidia_wayland_blacklist.patch:
      - don't blacklist nvidia for wayland
    + add gdm3.service-wait-for-drm-device-before-trying-to-start-i.patch:
      - wait for the first valid gdm device on pre-start...

[Neil Webster (neilw-nmw)]

Hi, this seems to have fixed the issue. The different issue you saw was relating to how I'd issued a test certificate to the YubiKey. I resolved this and can confirm the fix works great (although I have been having issues with pcscd.socket not triggering pcscd to run).

Will this fix be released for 22.04/22.10?

[Andreas Hasenack (ahasenack)]

Marco, why have you kept the code that may download a test script from a github gist? I can more or less understand having that there while testing/troubleshooting, but now? I know it's gated on a) OFFLINE_MODE (although lunar has some changes in that area that are not in kinetic); and b) the test script NOT being in d/t (and it is), but it still feels odd to have this potentially-execute-code-from-internet case in there

[Andreas Hasenack (ahasenack)]

We discussed this a bit in the SRU team, and the external dependency is "fine" because it's disabled by default. I would still prefer it's not there, or at least that the default for OFFLINE_MODE was already 1 and you wouldn't have to specify it when running the test, but this is not a blocker for the SRU.

[Andreas Hasenack (ahasenack)]

Hello Neil, or anyone else affected,

Accepted gdm3 into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gdm3/43.0-1ubuntu1.22.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Andreas Hasenack (ahasenack)]

Hello Neil, or anyone else affected,

Accepted gdm3 into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gdm3/42.0-1ubuntu7.22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted gdm3 (42.0-1ubuntu7.22.04.1) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

systemd/249.11-0ubuntu3.6 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#gdm3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Nathan Teodosio (nteodosio)]

Test summary
------------

Sorry but the test case is unclear to me. I don't consider it has succeeded, I think some steps may be missing.

Test description
----------------

Started Jammy virtual machine, logged in with normal user, installed gdm from jammy/proposed (42.0-1ubuntu7.22.04.1), opened a terminal:

--->
q1@q1-Standard-PC-i440FX-PIIX-1996:~$ pamtester -v gdm-smartcard q1 authenticate
pamtester: invoking pam_start(gdm-smartcard, q1, ...)
pamtester: performing operation - authenticate
Please insert smart card
PIN for Smartcard: [123456 is what I arbitrarily inserted]
pamtester: Authentication service cannot retrieve authentication info
<---

Seeing that this did not match the expected output, I tried the alternative path.

  sudo bash sssd-gdm-smartcard-pam-auth-tester.sh

outputs a bunch during some 20 s. I saw some prompts for password for short durations but I guess those weren't meant for the user because I typed nothing and it nonetheless "finishes successfully". However, I fail to see how this affects GDM. I did not get any log in prompt in GDM, I didn't see GDM.

So I tried

  WAIT=1 sudo bash sssd-gdm-smartcard-pam-auth-tester.sh

This killed the GUI and after some time of black screen GDM popped up normally. I could only log in with my normal user credentials.

[Jimmy Graham (jimmyg20974)]

Is there any update on this particular issue?

Rocky and RedHat version 8 systems can successfully use PIV authentication in conjunction with SSSD and can be used on Government systems. Ubuntu can not as of now.

Jim Graham

[Orion-cora (orion-cora)]

The gdm3 update in jammy-proposed fixed the smartcard login issue for us. Thanks.

[Marco Trevisan (Treviño) (3v1n0)]

Nathan, when using `sssd-gdm-smartcard-pam-auth-tester.sh` it does all the checks that were failing before of this fix.

It doens't use the gdm UI directly, because there's no need for it, but it indeed uses the gdm-smartcard PAM configuration that was the buggy one. So if the script works for you, it's a good sign :)

Using the WAIT mode also requires a smartcard to be properly configured or to use the script from another machine so that you can insert the credentials it expects from the virtual smartcard.

However, marking it as resolved in jammy as per Orion report.

[Chris Halse Rogers (raof)]

Can anyone do the testing for the kinetic update? Releasing to jammy-updates is blocked on verifying the kinetic fix.

[Neil Webster (neilw-nmw)]

Hi Chris,

To confirm, the fix is working fine for me when using a YubiKey with a smartcard certificate loaded onto it.

  * Neil
From: <email address hidden> <email address hidden> on behalf of Chris Halse Rogers <email address hidden>
Date: Wednesday, 29 March 2023 at 01:55
To: Neil Webster <email address hidden>
Subject: [Bug 1999884] Re: gdm-smartcard not passing successful authentication to desktop at system logon
Can anyone do the testing for the kinetic update? Releasing to jammy-
updates is blocked on verifying the kinetic fix.

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1999884

Title:
  gdm-smartcard not passing successful authentication to desktop at
  system logon

Status in gdm3 package in Ubuntu:
  Fix Released
Status in sssd package in Ubuntu:
  Incomplete
Status in gdm3 source package in Jammy:
  Fix Committed
Status in gdm3 source package in Kinetic:
  Fix Committed

Bug description:
  [ Impact ]

  gdm-smartcard returns a Permission denied when logging in with an user
  name:

  + pamtester -v gdm-smartcard ubuntu authenticate
  pamtester: invoking pam_start(gdm-smartcard, ubuntu, ...)
  pamtester: performing operation - authenticate
  PIN for Test Organization Root Tr Token:
  pamtester: Permission denied

  Using an empty user name works instead.

  [ Test case ]

  1. Use a smartcard to login in gdm

  This can also be simulated via:

  # Must be ran as user
  sudo apt install pamtester
  pamtester -v gdm-smartcard $USER authenticate

  Expected output is
  + pamtester -v gdm-smartcard ubuntu authenticate
  pamtester: invoking pam_start(gdm-smartcard, ubuntu, ...)
  pamtester: performing operation - authenticate
  PIN for Test Organization Sub Int Token:
  pamtester: successfully authenticated

  ---

  Alternatively, if no smartcard or hardware is available, this can be tested and simulated using these scripts (they will reset the system setup at each run, but it's suggested to run them in a VM, lxd container or in a test installation):
   https://gist.github.com/3v1n0/287d02ca8e03936f1c7bba992173d47a

  - sudo apt install gdm3 pamtester softhsm2 openssl wget sssd gnutls-bin && \
    sudo apt-mark auto gdm3 pamtester softhsm2 openssl wget sssd gnutls-bin
  - wget https://gist.github.com/3v1n0/287d02ca8e03936f1c7bba992173d47a/raw/sssd-gdm-smartcard-pam-auth-tester.sh
  - sudo sssd-gdm-smartcard-pam-auth-tester.sh

  The script will generate some fake CA authority, issue some
  certificates, will install them in some software-based smartcards
  (using softhsm2) and test that they work properly to login with gdm-
  smartcard.

  Using `WAIT` environment variable set (to any value) will make it to
  restart gdm at each iteration so that an user can try to access, using
  the username that launched the script and the pin of 123456.

  [ Regression Potential ]

  A root user could access to pam_sss, however it's the responsibility
  of such module to block such access.

  ---

  For information I've repeated this entire process on RHEL8 and it
  works there, it also was working upon last test on Ubuntu 20.04

  Releases: 22.04 LTS and 22.10
  Package Ve...

[Andreas Hasenack (ahasenack)]

Neil didn't clarify which ubuntu release he tested in his comment above, but given that he is the one who filed this bug originally on kinetic (22.10), and also confirmed in comment #12 that the issue was fixed, I'll take this as verification done for kinetic, assuming it was the test case number one: "login with a smartcard".

[Andreas Hasenack (ahasenack)]

I verified the test results and am satisfied that they show the executed planned test case, and that the results are correct. Since the test case was a script, I also ran it successfully on a kinetic LXD.

The package built correctly in all architectures and Ubuntu releases it was meant for.

There are no DEP8 regressions.

There is no SRU freeze ongoing at the moment.

There is no halted phasing on the previous update.

[Launchpad Janitor (janitor)]

This bug was fixed in the package gdm3 - 43.0-1ubuntu1.22.10.1

---------------
gdm3 (43.0-1ubuntu1.22.10.1) kinetic; urgency=medium

  * debian: Update vcs references to ubuntu/kinetic branch
  * debian/gdm3-gdm-smartcard*: Do not fail if pam_succeed_if suceeded.
    We were not handling the success case in pam_succeed_if.so, and so even
    if other modules were successful, gdm-smartcard was failing with a
    permission denied error, because the pam_succeed_if default was bad, and
    this was applied to the success case too.
    Alternatively we could even just use success=ignore here, but it's
    better to be consistent with other usages. (LP: #1999884)
  * debian/tests: Add autopkg tests testing gdm smartcard authentication.
    Create fake certificates from fake CA's and verify they can be used with
    from a virtual smartcard.

 -- Marco Trevisan (Treviño) <email address hidden> Tue, 31 Jan 2023 05:25:15 +0100

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for gdm3 has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package gdm3 - 42.0-1ubuntu7.22.04.1

---------------
gdm3 (42.0-1ubuntu7.22.04.1) jammy; urgency=medium

  * debian: Update vcs references to ubuntu/jammy branch
  * debian/gdm3-gdm-smartcard*: Do not fail if pam_succeed_if suceeded.
    We were not handling the success case in pam_succeed_if.so, and so even
    if other modules were successful, gdm-smartcard was failing with a
    permission denied error, because the pam_succeed_if default was bad, and
    this was applied to the success case too.
    Alternatively we could even just use success=ignore here, but it's
    better to be consistent with other usages. (LP: #1999884)
  * debian/tests: Add autopkg tests testing gdm smartcard authentication.
    Create fake certificates from fake CA's and verify they can be used with
    from a virtual smartcard.

 -- Marco Trevisan (Treviño) <email address hidden> Tue, 31 Jan 2023 05:24:48 +0100