pam_sss_gss fails to work when KRB5CCNAME is not set

[Karl Grindley]

[Impact]

If you don't set KRB5CCNAME for the current user on login, pam_sss_gss fails to
authenticate:

$ unset KRB5CCNAME

$ klist
Ticket cache: FILE:/run/user/1234567890/krb5cc
Default principal: NETWORKUSER@REALM
[valid creds listed here]

$ sudo -i
pam_sss_gss: sss_cli_getenv() call failed [2]: No such file or directory
pam_sss_gss: User not found
Please insert smart card
Please (re)insert (different) Smartcard
Please (re)insert (different) Smartcard
sudo: a password is required

A workaround is to set KRB5CCNAME before you call pam_sss_gss.

[Testcase]

You need a full AD Domain Controller set up, that uses kerberos and ldap.
From there, join the domain on the client.

Setup pam_sss_gss:

$ sudo vim /etc/sssd/sssd.conf
...
[domain/example.com]
...
pam_gssapi_services = sudo, sudo-i

auth sufficient pam_sss_gss.so needs to be the first line in the following:

$ sudo vim /etc/pam.d/sudo
auth sufficient pam_sss_gss.so
...
@include common-auth
...
$ sudo vim /etc/pam.d/sudo-i
auth sufficient pam_sss_gss.so
...
@include common-auth
...

Reboot, log back in.

You can either configure to not set KRB5CCNAME on login, or just unset it:

$ kinit <email address hidden>
$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: <email address hidden>
Valid starting Expires Service principal
03/20/25 22:49:12 03/21/25 08:49:12 <email address hidden>
 renew until 03/21/25 22:49:09

$ unset KRB5CCNAME
$ sudo -i
pam_sss_gss: sss_cli_getenv() call failed [2]: No such file or directory
pam_sss_gss: User not found
Please insert smart card
Please (re)insert (different) Smartcard
Please (re)insert (different) Smartcard
sudo: a password is required

Test packages are available in the following ppa:

https://launchpad.net/~mruffell/+archive/ubuntu/sf408269-test

If you install them, you should be able to authenticate as expected.

[Where problems can occur]

This change only affects users of the pam_sss_gss module. If users are not using
this module, there is no change in functionality.

If a regression were to occur, it would affect users trying to authenticate with
pam_sss_gss. Users might be able to workaround and use a different method / pam
module to authenticate if there was a regression.

[Other info]

Upstream issue: https://github.com/SSSD/sssd/issues/6180

This was fixed in 2.8.0 by the commit:

commit 9aad30711a5928f0e8a3627305b6449291de507f
From: Pavel Březina <email address hidden>
Date: Mon, 23 May 2022 11:05:01 +0200
Subject: pam_sss_gss: KRB5CCNAME may be NULL
Link: https://github.com/SSSD/sssd/commit/9aad30711a5928f0e8a3627305b6449291de507f

Only jammy needs this patch.

[Matthew Ruffell (mruffell)]

Attached is a debdiff for jammy which solves this issue.

[Karl Grindley (karlg100)]

ppa packages tested and now fixed! thanks for the fast turnaround!

[Matthew Ruffell (mruffell)]

This has been sponsored to jammy.

[Andreas Hasenack (ahasenack)]

We have to be specially careful to avoid sssd regressions. I see that pam_sss_gss is not used by default in the packaging, and has to be enabled manually, so that's a plus side. Too bad we don't have a dep8 test for it, but we do for the other cases, and can then rely on their results to catch regressions in other areas of sssd.

About the test plan, it seems to be showing the failure case? Can you show what the good case should look like?

[Karl Grindley (karlg100)]

Sure can! I can provide Pam-with-update config, test instructions and debug output. Some of that is above, but please give me a day or so to generate.

[Karl Grindley (karlg100)]

Note the failure mode is shown above. the extra logging in the description of the template is the failure mode.

Test setup:

# bind system to KDC (ie. AD)
# ensure the system for kerberos (make sure you can kinit as your user)
# add the following to the pam-auth-update templates (/usr/share/pam-configs/sss-gssapi) (note I added debug to increase verbosity for this test, this is not needed for general use)

Name: SSS GSSAPI authentication
Default: yes
Priority: 900

Auth-Type: Primary
Auth:
 [success=end default=ignore] pam_sss_gss.so debug
Auth-Initial:
 [success=end default=ignore] pam_sss_gss.so debug

# run pam-auth-update to enable the module
# add to /etc/sssd/sssd.conf under the kerberos realm/domain section, restart sssd
pam_gssapi_services = sudo, sudo-i

# add yourself to the /etc/sudoers.d/user_[username]
user1 ALL=(ALL) ALL

Testing that pam_sss_gss.so is properly authing:
# login as your user, view (or gain) kerberos credentials (klist to verify valid ccache, kinit to gain creds if needed)
# run sudo -i or sudo [command]. this should execute the sudo without issue

eg:
user1@working-host1:~$ sudo -i
pam_sss_gss: Initializing GSSAPI authentication with SSSD
pam_sss_gss: Switching euid from 0 to 1234556
pam_sss_gss: Trying to establish security context
pam_sss_gss: SSSD User name: <email address hidden>
pam_sss_gss: User domain: example.com
pam_sss_gss: User principal:
pam_sss_gss: Target name: <email address hidden>
pam_sss_gss: Using ccache: default
pam_sss_gss: Acquiring credentials, principal name will be derived
pam_sss_gss: Switching euid from 1234556 to 0
pam_sss_gss: Authentication successful
root@host1:~#

# exit back to user session, view krb ccache. a valid service ticket for host/working-host1.example.com will be seen.

test to verify blocking access without valid kerberos creds: (intentional failure test)
# login as your user, destroy all krb creds: kdestroy -A
# verify creds are destroyped/ccache is empty with klist -A
# run sudo -i or sudo [command]. this should execute sudo as usual via the pam stack, asking for alternative credentials based on the pam-auth-update configuration

user1@working-host1:~$ sudo -i
pam_sss_gss: Initializing GSSAPI authentication with SSSD
pam_sss_gss: Switching euid from 0 to 1234556
pam_sss_gss: Trying to establish security context
pam_sss_gss: SSSD User name: <email address hidden>
pam_sss_gss: User domain: example.com
pam_sss_gss: User principal:
pam_sss_gss: Target name: <email address hidden>
pam_sss_gss: Using ccache: default
pam_sss_gss: Acquiring credentials, principal name will be derived
pam_sss_gss: Unable to read credentials from [default] [maj:0x70000, min:0x96c73a8d]
pam_sss_gss: GSSAPI: No credentials were supplied, or the credentials were unavailable or inaccessible
pam_sss_gss: GSSAPI: No Kerberos credentials available (default cache: FILE:/run/user/1234556/krb5cc)
pam_sss_gss: Switching euid from 1234556 to 0
pam_sss_gss: System error [5]: Input/output error
Please insert smart card

access is blocked.

please let me know if anything else is needed!

As a side note, I have a completely rebuilt krb5 configuration and optimizations applied to all the support...

[Andreas Hasenack (ahasenack)]

In this test:

> Testing that pam_sss_gss.so is properly authing:
> # login as your user, view (or gain) kerberos credentials (klist to verify valid ccache, kinit to gain creds if needed)
> # run sudo -i or sudo [command]. this should execute the sudo without issue
>
> eg:
> user1@working-host1:~$ sudo -i
> pam_sss_gss: Initializing GSSAPI authentication with SSSD
> pam_sss_gss: Switching euid from 0 to 1234556
> pam_sss_gss: Trying to establish security context
> pam_sss_gss: SSSD User name: <email address hidden>
> pam_sss_gss: User domain: example.com
> pam_sss_gss: User principal:
> pam_sss_gss: Target name: <email address hidden>
> pam_sss_gss: Using ccache: default
> pam_sss_gss: Acquiring credentials, principal name will be derived
> pam_sss_gss: Switching euid from 1234556 to 0
> pam_sss_gss: Authentication successful
> root@host1:~#

Did you also unset KRB5CCNAME? That's what we are trying to fix, right? That it works without that variable set.

[Karl Grindley (karlg100)]

yes, sorry.

add to /etc/sssd/sssd.conf in the [pam] seciton:

pam_response_filter = ENV:KRB5CCNAME

[Karl Grindley (karlg100)]

or unset the var. either way.

[Andreas Hasenack (ahasenack)]

Thanks. All I need is the good test outcome, making clear the KRB5CCNAME variable is unset, added to the [ test plan ] part of the description of this bug. I'm wary of copy & pasting your comments into that section, because I haven't run this myself, and don't want to "put words in someone else's mouth", so to speak.

And thanks for also checking the no-creds case, showing that sudo then does not work. I would preface that test with explicits "kdestroy; klist" calls to show there are no credentials, and also an echo $KRB5CCNAME to show none is set pointing elsewhere.

[Karl Grindley (karlg100)]

Test setup:

# bind system to KDC (ie. AD)
# ensure the system for kerberos (make sure you can kinit as your user)
# add the following to the pam-auth-update templates (/usr/share/pam-configs/sss-gssapi) (note I added debug to increase verbosity for this test, this is not needed for general use)

Name: SSS GSSAPI authentication
Default: yes
Priority: 900

Auth-Type: Primary
Auth:
 [success=end default=ignore] pam_sss_gss.so debug
Auth-Initial:
 [success=end default=ignore] pam_sss_gss.so debug

# run pam-auth-update to enable the module
# add to /etc/sssd/sssd.conf under the kerberos realm/domain section, restart sssd
[domain/realm section]
pam_gssapi_services = sudo, sudo-i

[pam]
pam_response_filter = ENV:KRB5CCNAME

# add yourself to the /etc/sudoers.d/user_[username]
user1 ALL=(ALL) ALL

Testing that pam_sss_gss.so is properly authing:
# login as your user, view (or gain) kerberos credentials (klist to verify valid ccache, kinit to gain creds if needed)
# run sudo -i or sudo [command]. this should execute the sudo without issue

eg:
user1@working-host1:~$ echo $KRB5CCNAME

user1@working-host1:~$ kinit <email address hidden>

user1@working-host1:~$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: <email address hidden>
Valid starting Expires Service principal
03/20/25 22:49:12 03/21/25 08:49:12 <email address hidden>
 renew until 03/21/25 22:49:09

user1@working-host1:~$ sudo -i
pam_sss_gss: Initializing GSSAPI authentication with SSSD
pam_sss_gss: Switching euid from 0 to 1234556
pam_sss_gss: Trying to establish security context
pam_sss_gss: SSSD User name: <email address hidden>
pam_sss_gss: User domain: example.com
pam_sss_gss: User principal:
pam_sss_gss: Target name: <email address hidden>
pam_sss_gss: Using ccache: default
pam_sss_gss: Acquiring credentials, principal name will be derived
pam_sss_gss: Switching euid from 1234556 to 0
pam_sss_gss: Authentication successful
root@host1:~#

# exit back to user session, view krb ccache. a valid service ticket for host/working-host1.example.com will be seen.

test to verify blocking access without valid kerberos creds: (intentional failure test)
# login as your user, destroy all krb creds: kdestroy -A
# verify creds are destroyped/ccache is empty with klist -A
# run sudo -i or sudo [command]. this should execute sudo as usual via the pam stack, asking for alternative credentials based on the pam-auth-update configuration

user1@working-host1:~$ echo $KRB5CCNAME

user1@working-host1:~$ kdestroy -A

user1@working-host1:~$ klist
klist: No credentials cache found (filename: /run/user/1234556/krb5cc)

user1@working-host1:~$ sudo -i
pam_sss_gss: Initializing GSSAPI authentication with SSSD
pam_sss_gss: Switching euid from 0 to 1234556
pam_sss_gss: Trying to establish security context
pam_sss_gss: SSSD User name: <email address hidden>
pam_sss_gss: User domain: example.com
pam_sss_gss: User principal:
pam_sss_gss: Target name: <email address hidden>
pam_sss_gss: Using ccache: default
pam_sss_gss: Acquiring credentials, principal name will be derived
pam_sss_gss: Unable to read credentials from [default] [maj:0x70000, min:0x96c73a8d]
pam_sss_gss: GSSAPI: No credentials were supplie...

[Andreas Hasenack (ahasenack)]

Hello Karl, or anyone else affected,

Accepted sssd into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/sssd/2.6.3-1ubuntu3.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted sssd (2.6.3-1ubuntu3.5) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

adsys/0.14.3~22.04ubuntu0.1 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#sssd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Karl Grindley (karlg100)]

tested and working from proposed!

[Karl Grindley (karlg100)]

libpam-sss/jammy-proposed,now 2.6.3-1ubuntu3.5 amd64 [installed]

ran tests using sudo without KRB5CCNAME set, and now functions as expected.

[Matthew Ruffell (mruffell)]

Performing verification for Jammy.

I set up a Samba Active Directory KDC on a Focal Server by:

You will need a 20.04 server instance, and a 20.04 Desktop instance.

1) Create a fresh 20.04 server instance
2) sudo apt update
3) sudo apt upgrade
4) sudo hostnamectl set-hostname samba-dc
5) sudo vim /etc/hosts
Add an entry with its IP address, e.g.:
192.168.122.199 samba-dc samba-dc.example.com
6) sudo apt install -y samba smbclient winbind libpam-winbind libnss-winbind krb5-kdc libpam-krb5
Note: skip config of kerberos KDC.
7) sudo rm /etc/krb5.conf
8) sudo rm /etc/samba/smb.conf
9) sudo samba-tool domain provision --server-role=dc --use-rfc2307 --dns-backend=SAMBA_INTERNAL --realm=samba-dc.EXAMPLE.COM --domain=SAMBA --adminpass=Password1
10) sudo cp /var/lib/samba/private/krb5.conf /etc/krb5.conf
11) sudo systemctl mask smbd nmbd winbind
12) sudo systemctl disable smbd nmbd winbind
13) sudo systemctl stop smbd nmbd winbind
14) sudo systemctl unmask samba-ad-dc
15) sudo systemctl start samba-ad-dc
16) sudo systemctl enable samba-ad-dc
17) sudo reboot
18) sudo systemctl stop systemd-resolved
19) sudo systemctl disable systemd-resolved
20) cat << EOF >> /etc/resolv.conf
nameserver 192.168.122.199
search SAMBA
EOF
21) sudo reboot
22) host -t SRV _ldap._tcp.samba-dc.example.com
_ldap._tcp.samba-dc.example.com has SRV record 0 100 389 samba-dc.samba-dc.example.com.
23) $ smbclient -L localhost -N
Anonymous login successful

 Sharename Type Comment
 --------- ---- -------
 sysvol Disk
 netlogon Disk
 IPC$ IPC IPC Service (Samba 4.13.17-Ubuntu)
SMB1 disabled -- no workgroup available
24) smbclient //localhost/netlogon -UAdministrator -c 'ls'
Password for [SAMBA\Administrator]:
  . D 0 Tue Apr 8 03:17:16 2025
  .. D 0 Tue Apr 8 03:17:18 2025

  9974088 blocks of size 1024. 7847840 blocks available
25) kinit administrator
Password for <email address hidden>:
Warning: Your password will expire in 41 days on Mon Apr 08 13:20:04 2025
26) klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: <email address hidden>

Valid starting Expires Service principal
04/08/25 03:20:04 04/08/25 13:20:04 <email address hidden>
 renew until 04/09/25 03:20:00
27)

Create a share:
28) sudo mkdir -p /srv/samba/Demo/
29) sudo vim /etc/samba/smb.conf
[Demo]
 path = /srv/samba/Demo/
 read only = no
30) sudo chmod 0770 /srv/samba/Demo/
31) $ sudo samba-tool user create user1 --unix-home /home/user1 --login-shell /bin/bash --uid-number 10000 --gid-number 10000 --nis-domain samba-dc
New Password:
Retype Password:
User 'user1' added successfully

I then made a 22.04 Client and ran:

$ sudo apt install realmd smbclient
$ sudo vim /etc/hosts
Add an entry with its IP address, e.g.:
192.168.122.199 samba-dc samba-dc.example.com
$ sudo realm join --user=Administrator SAMBA-DC.EXAMPLE.COM
$ getent passwd <email address hidden>
<email address hidden>:*:577001104:577000513:user1:/home/user1:/bin/bash
$ groups <email address hidden>
<email address hidden> : domain <email address hidden>

Okay, we currently hav...