JAAS Krb5LoginModule authenticates wrong principal

[Andreas Ufert]

Up until recently we've used a JAAS configuration to allow Kafka to authenticate against a Kerberos secured Zookeeper server using this JAAS config:

Client {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    keyTab="/etc/security/keytabs/kafka.service.keytab"
    storeKey=true
    useTicketCache=false
    serviceName="zookeeper"
    <email address hidden>";
};

With OpenJDK 8 Update 242 (in Ubuntu Bionic: 8u242-b08-0ubuntu3~18.04) this configuration fails to work in our environment under the following conditions:

- In the JAAS config, the service principal is referenced to using its AD servicePrincipalName kafka/fqdn.redacted.
- The sAMAccountName differs, it is actually $48M300-TK9QSTNPCT80.
- Kafka gets itself a Kerberos ticket using the JAAS config above.

Turns out the principal is authenticated as $<email address hidden> instead of <email address hidden> like it was before Update 242. Krb5LoginModule seemed to pick "client_alias" before and "client" after the upgrade (see debug output in section "Actual Result" of this bug report.

However, the problem does *not* appear when JAAS is configured to use a Kerberos ticket in the ticket cache for the same principal using this JAAS config:

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=false
useTicketCache=true;
};

Here, the principal is still authenticated correctly as <email address hidden>.

#####

Steps to reproduce:

- In AD there is a service principal with servicePrincipalName=kafka/fqdn.redacted and sAMAccountName=$48M300-TK9QSTNPCT80
- JAAS config is as follows:

Client {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    keyTab="/etc/security/keytabs/kafka.service.keytab"
    storeKey=true
    useTicketCache=false
    serviceName="zookeeper"
    <email address hidden>";
};

- Zookeeper client connects to Zookeeper the JAAS config above: KAFKA_OPTS="-Djava.security.auth.login.config=/opt/kafka/kafka/config/kafka_jaas.conf -Dsun.security.krb5.debug=true" bin/zookeeper-shell.sh fqdn.redacted:2181

#####

Expected Result:

Zookeeper authenticates <email address hidden> as it was before the OpenJDK 8 Update 242 upgrade.

#####

Actual Result:

Zookeeper authenticates $<email address hidden>

See this log:

Connecting to fqdn.redacted:2181
Welcome to ZooKeeper!
JLine support is disabled
>>> KeyTabInputStream, readName(): EXAMPLE.COM
>>> KeyTabInputStream, readName(): kafka
>>> KeyTabInputStream, readName(): fqdn.redacted
>>> KeyTab: load() entry length: 99; type: 18
Looking for keys for: <email address hidden>
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
Added key: 18version: 1
>>> KdcAccessibility: reset
Looking for keys for: <email address hidden>
Added key: 18version: 1
default etypes for default_tkt_enctypes: 18 17 16 23 18 17 16 23.
>>> KrbAsReq creating message
>>> KrbKdcReq send: kdc=example.com UDP:88, timeout=30000, number of retries =3, #bytes=197
>>> KDCCommunication: kdc=example.com UDP:88, timeout=30000,Attempt =1, #bytes=197
>>> KrbKdcReq send: #bytes read=195
>>>Pre-Authentication Data:
         PA-DATA type = 19
         PA-ETYPE-INFO2 etype = 18, salt = EXAMPLE.COMkafkafqdn.redacted, s2kparams = null

>>>Pre-Authentication Data:
         PA-DATA type = 2
         PA-ENC-TIMESTAMP
>>>Pre-Authentication Data:
         PA-DATA type = 16

>>>Pre-Authentication Data:
         PA-DATA type = 15

>>> KdcAccessibility: remove example.com
>>> KDCRep: init() encoding tag is 126 req type is 11
>>>KRBError:
         sTime is Fri Jan 31 11:03:30 CET 2020 1580465010000
         suSec is 347467
         error code is 25
         error Message is Additional pre-authentication required
         sname is <email address hidden>
         eData provided.
         msgType is 30
>>>Pre-Authentication Data:
         PA-DATA type = 19
         PA-ETYPE-INFO2 etype = 18, salt = EXAMPLE.COMkafkafqdn.redacted, s2kparams = null

>>>Pre-Authentication Data:
         PA-DATA type = 2
         PA-ENC-TIMESTAMP
>>>Pre-Authentication Data:
         PA-DATA type = 16

>>>Pre-Authentication Data:
         PA-DATA type = 15

KrbAsReqBuilder: PREAUTH FAILED/REQ, re-send AS-REQ
default etypes for default_tkt_enctypes: 18 17 16 23 18 17 16 23.
Looking for keys for: <email address hidden>
Added key: 18version: 1
Looking for keys for: <email address hidden>
Added key: 18version: 1
default etypes for default_tkt_enctypes: 18 17 16 23 18 17 16 23.
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbAsReq creating message
>>> KrbKdcReq send: kdc=example.com UDP:88, timeout=30000, number of retries =3, #bytes=282
>>> KDCCommunication: kdc=example.com UDP:88, timeout=30000,Attempt =1, #bytes=282
>>> KrbKdcReq send: #bytes read=88
>>> KrbKdcReq send: kdc=example.com TCP:88, timeout=30000, number of retries =3, #bytes=282
>>> KDCCommunication: kdc=example.com TCP:88, timeout=30000,Attempt =1, #bytes=282
>>>DEBUG: TCPClient reading 1617 bytes
>>> KrbKdcReq send: #bytes read=1617
>>> KdcAccessibility: remove example.com
Looking for keys for: <email address hidden>
Added key: 18version: 1
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> CksumType: sun.security.krb5.internal.crypto.HmacSha1Aes256CksumType
>>> KrbAsRep cons in KrbAsReq.getReply kafka/fqdn.redacted

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
Found ticket for $<email address hidden> to go to <email address hidden> expiring on Fri Jan 31 21:03:30 CET 2020
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for $<email address hidden> to go to <email address hidden> expiring on Fri Jan 31 21:03:30 CET 2020
Service ticket not found in the subject
>>> Credentials serviceCredsSingle: same realm
default etypes for default_tgs_enctypes: 18 17 16 23 18 17 16 23.
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> CksumType: sun.security.krb5.internal.crypto.HmacSha1Aes256CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbKdcReq send: kdc=example.com TCP:88, timeout=30000, number of retries =3, #bytes=1494
>>> KDCCommunication: kdc=example.com TCP:88, timeout=30000,Attempt =1, #bytes=1494
>>>DEBUG: TCPClient reading 1496 bytes
>>> KrbKdcReq send: #bytes read=1496
>>> KdcAccessibility: remove example.com
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> TGS credentials serviceCredsSingle:
>>> DEBUG: ----Credentials----
        client: $<email address hidden>
        client alias: <email address hidden>
        server: <email address hidden>
        ticket: sname: <email address hidden>
        startTime: 1580465010000
        endTime: 1580501010000
        ----Credentials end----
>>> KrbApReq: APOptions are 00000000 00000000 00000000 00000000
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
Krb5Context setting mySeqNumber to: 94571730
Krb5Context setting peerSeqNumber to: 94571730

#####

Workaround:

There are two workarounds:

1. Changing the JAAS config to use a ticket in the ticket cache created upfront using kinit. However this workaround can't be used in automated environments where clients authenticate themselves using keytabs.

2. Downgrade from OpenJDK 8 Update 242 to Update 232. Using the same configuration, the principal is authenticated as <email address hidden>. However, downgrading a package can only be a temporary solution.

#####

[Andreas Ufert (andreas.ufert)]

Problem can be reproduced using a minimal Example as follows:

# This is what we have:

user@host:~/work$ ls -al
total 20
drwxrwx--- 2 user user 4096 Feb 14 17:33 .
drwxr-xr-x 5 user user 4096 Feb 14 17:29 ..
-rw-rw---- 1 user user 942 Feb 14 17:29 KerberosDemo.java
-rw-rw---- 1 user user 101 Feb 13 13:13 jaas_cached.conf
-rw-rw---- 1 user user 276 Feb 13 13:24 jaas_keytab.conf

# it's a minimal example

user@host:~/work$ cat KerberosDemo.java
import javax.security.auth.login.*;
import java.util.Iterator;
import java.util.Set;
import javax.security.auth.Subject;
import javax.security.auth.kerberos.KerberosTicket;

public class KerberosDemo {
        public static void main (String[] args) {
                LoginContext loginContext = null;
                try {
                        loginContext = new LoginContext("Demo");
                }
                catch (LoginException e) {
                        System.err.println("login context creation failed: "+e.getMessage());
                        System.exit(1);
                }
                try {
                        loginContext.login();
                }
                catch (LoginException e) {
                        System.out.println("authentication failed");
                        System.exit(1);
                }
                Subject subject = loginContext.getSubject();
                System.out.println("Authenticated principal: " + subject.getPrincipals());
                Set credentials = subject.getPrivateCredentials();
                Iterator iterator = credentials.iterator();
                KerberosTicket kt = (KerberosTicket) iterator.next();
                System.out.println("Client name: " + kt.getClient());
        }
}

# let's compile it

user@host:~/work$ javac KerberosDemo.java

# and use it either with a keytab (JAAS is getting the ticket) ...

user@host:~/work$ cat jaas_keytab.conf # use keytab!
Demo {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    keyTab="/etc/security/keytabs/myprincipal.service.keytab"
    storeKey=true
    useTicketCache=false
    serviceName="serviceprincipal"
    <email address hidden>";
};

# ... or with a ticket gotten earlier by MIT Kerberos client (kinit)

user@host:~/work$ cat jaas_cached.conf # use cached!
Demo {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=false
    useTicketCache=true;
};

# this is how the ticket was placed in the cache

user@host:~/work$ kinit -kt /etc/security/keytabs/myprincipal.service.keytab <email address hidden>

# now, this is what happens with OpenJDK 1.8.0_232
# principal name and client name all refer to <email address hidden> (in AD, this is the servicePrincipalName):

user@host:~/work$ java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
user@host:~/work$ java -Djava.security.auth.login.config=jaas_keytab.conf KerberosDemo
Authenticated principal: [<email address hidden>]
Client name: myprincipal/fqdn.example.com@EXA...

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Andreas Ufert (andreas.ufert)]

Bug is tracked here: https://bugs.openjdk.java.net/browse/JDK-8239385