Comment 18 for bug 2020834

Hi everyone,

The term "regression" is a slight overstatement for Kinetic, as login fails with the version in -updates as well, due to it not supporting the "-" in the domain name.

With 0.9.2 in kinetic -updates:

ERRORgithub.com/ubuntu/adsys/cmd/adsysd/main.go:50 main.run() Error from server: error while updating policy: can't get policies for "ec2amaz-hg2r0q8.fabio-rg.com": failed to retrieve offline state from SSSD: dbus: invalid message: invalid path name

and then login also fails. So, the package in -proposed does not break anything further, but it doesn't fix everything either.

Now, here is what I found with the package in -proposed:

Jul 20 05:21:35 ip-172-31-55-219 adsysctl[2564546]: level=error msg="Error from server: error while updating policy: can't get policies for \"ip-172-31-55-219\": failed to retrieve the list of GPO (exited with 1): exit status 1
Failed to bind - LDAP client internal error: NT_STATUS_INVALID_PARAMETER
Failed to connect to 'ldap://ec2amaz-hg2r0q8.fabio-rg.com' with backend 'ldap': LDAP client internal error: NT_STATUS_INVALID_PARAMETER
Failed to open session: (1, 'LDAP client internal error: NT_STATUS_INVALID_PARAMETER')
"

> Failed to connect to 'ldap://ec2amaz-hg2r0q8.fabio-rg.com'
> ith backend 'ldap': LDAP client internal error: NT_STATUS_INVALID_PARAMETER

So this is a ldap based error, but we know this already.

I then tried to use other ldap based tools, like ldbsearch:

$ sudo apt install ldb-tools samba-dsdb-modules
$ cd /tmp
$ ll
krb5cc_1930801111_Xu8aKP
$ sudo ldbsearch -H ldap://ec2amaz-hg2r0q8.fabio-rg.com --use-krb5-ccache=/tmp/krb5cc_1930801111_Xu8aKP --debug-stdout --debuglevel 20
...
resolve_lmhosts: Attempting lmhosts lookup for name ec2amaz-hg2r0q8.fabio-rg.com<0x20>
startlmhosts: Can't open lmhosts file /etc/samba/lmhosts. Error was No such file or directory
Starting GENSEC mechanism spnego
Starting GENSEC submechanism gssapi_krb5
cli_credentials(WORKGROUP/root) without realm, cannot use kerberos for this connection ldap/ec2amaz-hg2r0q8.fabio-rg.com
Failed to start GENSEC client mech gssapi_krb5: NT_STATUS_INVALID_PARAMETER
gensec_spnego_create_negTokenInit_step: Failed to setup SPNEGO negTokenInit request
gensec_update_send: spnego[0x5556c039ff10]: subreq: 0x5556c03a0450
gensec_update_done: spnego[0x5556c039ff10]: NT_STATUS_INVALID_PARAMETER tevent_req[0x5556c03a0450/../../auth/gensec/spnego.c:1631]: state[3] error[-7963671676338569203 (0x917B5ACDC000000D)] state[struct gensec_spnego_update_state (0x5556c03a0610)] timer[(nil)] finish[../../auth/gensec/spnego.c:1947]
Failed to bind - LDAP client internal error: NT_STATUS_INVALID_PARAMETER
Failed to connect to 'ldap://ec2amaz-hg2r0q8.fabio-rg.com' with backend 'ldap': LDAP client internal error: NT_STATUS_INVALID_PARAMETER
Failed to connect to ldap://ec2amaz-hg2r0q8.fabio-rg.com - LDAP client internal error: NT_STATUS_INVALID_PARAMETER

Now, if we do this on Jammy, it works fine.

I was trying out different parameters to ldbsearch on kinetic, as you can see the user is incorrect:

WORKGROUP/root

The clue came from:

cannot use kerberos for this connection ldap/ec2amaz-hg2r0q8.fabio-rg.com

I started looking at kerberos credential caches.

I found that if I did a fresh kinit:

$ sudo kinit <email address hidden>
Password for <email address hidden>:

$ ll
krb5cc_0
krb5cc_1930801111_Xu8aKP

Now, if we use this fresh one:

$ sudo ldbsearch -H ldap://ec2amaz-hg2r0q8.fabio-rg.com --use-krb5-ccache=/tmp/krb5cc_0 --debug-stdout --debuglevel 20

Everything works fine on Kinetic.

So, maybe the kerberos keytab / credential cache is broken.

If we compare:

$ sudo klist /tmp/krb5cc_0
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: <email address hidden>

Valid starting Expires Service principal
07/20/23 05:28:49 07/20/23 15:28:49 <email address hidden>
 renew until 07/21/23 05:28:38
07/20/23 05:30:18 07/20/23 15:28:49 <email address hidden>
 renew until 07/21/23 05:28:38

$ sudo klist /tmp/krb5cc_1930801111_Xu8aKP
Ticket cache: FILE:/tmp/krb5cc_1930801111_Xu8aKP
Default principal: <email address hidden>

Valid starting Expires Service principal
07/20/23 03:19:25 07/20/23 13:19:25 <email address hidden>
 renew until 07/21/23 03:19:25

If we compare to Jammy:

$ sudo klist /tmp/krb5cc_1930801111_5HVmi5
Ticket cache: FILE:/tmp/krb5cc_1930801111_5HVmi5
Default principal: <email address hidden>

Valid starting Expires Service principal
07/20/23 01:19:29 07/20/23 11:19:29 <email address hidden>
 renew until 07/21/23 01:19:29
07/20/23 01:19:30 07/20/23 11:19:29 <email address hidden>
07/20/23 01:19:30 07/20/23 11:19:29 <email address hidden>

The kerberos credential cache obtained by sssd is missing the service principal for <email address hidden>.

I haven't figured out why sssd is not adding ldap/ to the credential cache, or how to add one manually to test, but I think this is what is broken on Kinetic.

Thanks,
Matthew