krb5 prefers the reverse pointer no matter what for locating service tickets.

[Jesper Krogh]

I'm trying to upgrade workstations to lucid an fails to access our kerberos enabled websites. It reveals that the krb5 implementation in lucid now tries to resolve the "reverse dns" and aquire a tikket for <service>/<reverse dns> instead of <service>/<what the user typed in the first place>.

The latter behavior is what the MS environment does and is what Ubuntu has done (i think) until Lucid. A diff of the sourcecode from hardy revealse that we now hint the getaddrinfo with AI_CANONNAME which it didnt before.

Applying below patch enables the old behaviour.

--- krb5-1.8.1+dfsg/src/lib/krb5/os/sn2princ.c.orig 2010-04-29 09:04:11.401567914 +0200
+++ krb5-1.8.1+dfsg/src/lib/krb5/os/sn2princ.c 2010-04-29 09:04:21.762191834 +0200
@@ -112,7 +112,7 @@

             memset(&hints, 0, sizeof(hints));
             hints.ai_family = AF_INET;
- hints.ai_flags = AI_CANONNAME;
+// hints.ai_flags = AI_CANONNAME;
         try_getaddrinfo_again:
             err = getaddrinfo(hostname, 0, &hints, &ai);
             if (err) {

[Jesper Krogh (jesper)]

Since the problem is in the clientside kerberos libraries it affects all kerberos enabled stuff.

[Sam Hartman (hartmans)]

Try setting rdns=false in the libdefaults section of krb5.conf.

[Jesper Krogh (jesper)]

Tried.. had that before.. but doesn't work any more. (and isn't documented in man krb5.conf either).

[Sam Hartman (hartmans)]

The Kerberos Consortium has a paper on integrating Kerberos into an
application; see http://www.kerberos.org/software/appskerberos.pdf .

I believe that the lucid behavior is correct according to MIT's
documentation: what should be happening is that

* with rdns=true (default), both forward and reverse resolution is
  performed and the reverse name is used

* With rdns=false, forward resolution is performed including alias
  resolution--that is cnames turn into the pointed-to value not the
  entered value.

That behavior seems consistent with the code. If you believe that
things aren't working that way, then I can attempt to reproduce.

As I understand your patch, it would (on some platforms including all
Ubuntu platforms) cause the rdns=false behavior to actually skip
resolution and just use the entered name not resolving cnames.

It's possible there was a bug in previous releases of MIT Kerberos and
this was the behavior.

I also understand that the behavior surrounding Kerberos and DNS is kind
of complicated and not entirely desirable. The paper I pointed you at
includes discussions of problems with the current behavior and eventual
goals. It also recommends ways applications can avoid forward/reverse
DNS resolution if they wish to do so.

[Jesper Krogh (jesper)]

Hi Sam.

I agree.. the current behaviors seems to be excactly what is in the code and in the documentation.

Never the less it is a change from earlier versions of Ubuntu and a change that makes Ubuntu + Firefox work in a different way than MS Windows + MSIE (negoiating different tickets), thus breaking Single Signon in typical Kerberos enabled environments.. our is a corporate one with Active Directory as Kerbereos and both MS IIS and Ubuntu Apache + mod_auth_kerb on the serverside.

Used to work.. lucid breaks it..

As far as I can tell, the change snug in between MIT kerberos 1.6 and 1.8 .

Jesper

[Russ Allbery (rra-debian)]

Jesper Krogh <email address hidden> writes:

> Never the less it is a change from earlier versions of Ubuntu and a
> change that makes Ubuntu + Firefox work in a different way than MS
> Windows + MSIE (negoiating different tickets), thus breaking Single
> Signon in typical Kerberos enabled environments.. our is a corporate one
> with Active Directory as Kerbereos and both MS IIS and Ubuntu Apache +
> mod_auth_kerb on the serverside.

> Used to work.. lucid breaks it..

I'm confused why you're seeing a change, since in my experience it's been
this way for quite some time. Firefox used the final hostname, whereas IE
always used the URL name. When we deployed Negotiate-Auth with
mod_auth_kerb, we had to add both principals to the server keytab. Many
other people had the same issue, as discussed on the mod_auth_kerb mailing
list, which is why mod_auth_kerb added an option to use any principal in
its keytab. This all happened back in 2007 for us.

--
Russ Allbery (<email address hidden>) <http://www.eyrie.org/~eagle/>

[Sam Hartman (hartmans)]

Well, everything should work fine if you make your DNS consistent.

Honestly if I was going to make a behavior change here I'd have Firefox
call gss_import_name with a name type that does not involve resolution.

--Sam

[Russ Allbery (rra-debian)]

Sam Hartman <email address hidden> writes:

> Well, everything should work fine if you make your DNS consistent.

> Honestly if I was going to make a behavior change here I'd have Firefox
> call gss_import_name with a name type that does not involve resolution.

The main place where you cannot make DNS consistent is if you have a web
service that uses DNS-based load-balancing. That's where we ran into that
issue. The public name is a CNAME that points to the least-loaded host
(which is dynamically discovered by the DNS server).

--
Russ Allbery (<email address hidden>) <http://www.eyrie.org/~eagle/>

[Jesper Krogh (jesper)]

Hi Russ.

I cannot say anything about what other are experiencing.. but in our setup we haven't put the final hostname in the keytab (and neither did the guys that only cared about MSIE+IIS). And all Ubuntu releases (from around Dapper) and up and until Lucid has worked for us.

So seen from an Ubuntu perspective, this is a regression.. wether or not it is desired to fix it I dont know?

But I guess until Windows changes behaviour (thus all the configuration of the Windows servers), I'll have to maintain this patch locally for our installation. An it is a fair assumption to say that our setup is fairly "standard".

Would a patch that makes the behaviour configurable be acceptable?

Jesper

[Sam Hartman (hartmans)]

>>>>> "Jesper" == Jesper Krogh <email address hidden> writes:

    Jesper> Hi Russ. I cannot say anything about what other are
    Jesper> Would a patch that makes the behaviour configurable be
    Jesper> acceptable?

I think that this patch should be accepted only if upstream is
interested in the patch. Given that upstream accepted rdns (something I
thought was kind of dubious at the time), a patch to completely disable
dns processing seems reasonable.

Apple's Kerberos maintainer argues that this behavior really needs to be
configured on a per-realm basis. Unfortunately, because of the way
krb5_sname_to_principal interacts with referrals makes this kind of
tricky. If I were upstream I'd require the design of the patch to be
forward-compatible to an eventual model where it was
configured/auto-detected on a per-realm basis and the behavior of any
configuration knobs you add to be documented well enough so that people
would understand how they will behave in the future, but beyond that
would accept the patch.
So, if upstream agrees with me here, you'd have to do somewhat more
design work up front, but the actual patch would be simple.

I'm certainly happy to accept such a patch into Debian as soon as
upstream accepts it and to encourage Ubuntu to accept it.

I don't have the time facilitate the discussion between you and
upstream; I wish I did. my recommendation for interacting with upstream
is to bring up the issue on <email address hidden> and to include the URI of
this bug report.

Kerberos DNS behavior is complicated enough that having Ubuntu or Debian
diverge from upstream seems undesirable, so I think involving upstream
in the discussion is important.

--Sam

[Sam Hartman (hartmans)]

In terms of work arounds, if your KDC is an AD KDc, you can add the
final hostnames as ServicePrincipalName attributes on AD for the account
in question. That should make things work either for a Windows server
or for a 1.7+ MIT server.

If your KDC is Unix you can add principals for the final hostnames. If
your eventual server is Windows you'll need to make sure the key and
salt is the same for all these principals. If your server is Unix,
simply add all the keys to the keytab.

--Sam

[Jesper Krogh (jesper)]

I agree that it is a partial workaround.. it fixes the Ubuntu/Firefox + apache combination.
But without changing the same thing for all the IIS servers it would still render my Ubuntu/Firefox + IIS SSO broken.

Since I only administrate the Linux stuff, and the "other side" genereally are very reluctant to do change to "only fit linux", then patching it locally is much more doable in my environment.

Anyway, now the bug is at least here to document it for other people hitting the same wall.

Jesper

[Taylor Yu (tlyu)]

Recent discussion on #krbdev suggests that this is actually (in part) a glibc bug in getaddinfo(). With ai_flags=AI_CANONNAME and ai_family=AF_INET, getaddrinfo() performs a PTR lookup anyway (contrary to POSIX), effectively preventing a setting of rdns=false from being effective. We have a workaround for the glibc bug upstream at http://krbdev.mit.edu/rt/Ticket/Display.html?id=6922

[Taylor Yu (tlyu)]

Our fix in #6922 appears to itself have a bug; we believe that http://krbdev.mit.edu/rt/Ticket/Display.html?id=7124 resolves it. If you need a back port, http://krbdev.mit.edu/rt/Ticket/Display.html?id=7164 is for krb5-1.9.x, and http://krbdev.mit.edu/rt/Ticket/Display.html?id=7184 is for krb5-1.8.x. (given that the initial report was against 1.8.1)

[Mark Pröhl (mark-mproehl)]

Hi,

we are seeing the same problems with msktutil (http://code.google.com/p/msktutil/issues/detail?id=11)

I seems to me that this issue is already fixed in the source packages. I did a rebuild of libkrb5-3_1.10+dfsg~beta1-2ubuntu0.3 with these sources:

  http://archive.ubuntu.com/ubuntu/pool/main/k/krb5/krb5_1.10+dfsg~beta1-2ubuntu0.3.dsc
  http://archive.ubuntu.com/ubuntu/pool/main/k/krb5/krb5_1.10+dfsg~beta1.orig.tar.gz
  http://archive.ubuntu.com/ubuntu/pool/main/k/krb5/krb5_1.10+dfsg~beta1-2ubuntu0.3.debian.tar.gz

With this rebuild package no reverse lookups are done for service principal canonicalization while the binary version from ubuntu repositories still seems to have this bug

Can anyone tell me when this will be officially fixed in Ubuntu 12.04.1

Cheers,

Mark Pröhl

[Taylor Yu (tlyu)]

I would strongly recommend SRUs for all supported releases, because this is a high-impact bug for people who are deploying krb5 in environments where they do not have tight control over their reverse DNS information. Experience has shown that this type of hard-to-debug DNS interaction leads to a lot of frustration and wasted time.

[Robie Basak (racb)]

To answer questions about getting an update into 12.04, we need (from https://wiki.ubuntu.com/StableReleaseUpdates):

An impact statement which explains who this bug affects (use cases), why this is a problem and why we need an update in 12.04 for it.
A test case with exact steps to reproduce the problem, so that we can verify any backported fix.
Confirmation that this is fixed in the development release (Raring).
A patch to fix this issue in Precise. Ideally this would be a pointer to the upstream commit and apply cleanly.
Discussion of possible regressions to existing users, particularly any change in behaviour that an existing user not affected by this bug might get angry about, and areas where a regression is likely to be found if one does exist so that the SRU verification team can try and find them.
All of this information gathered together in one place for the SRU team to review.

I am familiar with Kerberos and have spent my share of time debugging DNS-related Kerberos issues, but I'm read through this bug and although I have some idea I don't feel that I'm completely clear on answers to these questions. If somebody can help with this "paperwork", point out the upstream commit to cherry-pick and there is consensus on all of this, I think there's enough here to warrant an update to 12.04.

[William (kc-cobradevil)]

Hi Robie,

I'm also affected with this bug.
When rebuilding the source on quantal as described in comment: https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/571572/comments/15 the sso to the problematic site disappears when setting rdns=false in krb5.conf.
But this is not the case for precise, there it only works when patching the source from comment 15 with the original post.

Precise fix:
What i did was getting the source package for precise and patched it with:
https://github.com/krb5/krb5/commit/57738b357e8b03bcb7af2f147c97cb84d0ce96e2
install package libkrb5-3 libgssapi
After adding the rdns=false i can now authenticate sso to iis sites that were previously failing.
when commenting this option out (which is default) default behaviour is restored and i still can authenticate to servers that were previously working with e.g. mod_auth_kerb on apache but failed on iis sites.

I will try to setup raring desktop to test if the bug does not exist there.
Will try also patched version for quantal and explain my findings inclusive tickets in my ticket cache and cname/ptr/a records to those servers which were failing but working with the above patch.

William van de Velde.

[William (kc-cobradevil)]

Ok i have done some testing with rdns=false or commented out
I have replaced our internal domain with testdomain and our kerberos realm with EXAMPLE.COM

DNS:

dig searchsite.testdomain
searchsite.testdomain. 2264 IN A 10.0.0.10
dig sharepointsite.testdomain
sharepointsite.testdomain. 1325 IN A 10.0.0.10

dig -x 10.0.0.10
10.0.0.10.in-addr.arpa. 27924 IN PTR 2010searchsite.testdomain.
10.0.0.10.in-addr.arpa. 27924 IN PTR sharepointsite.testdomain.

(I know not my configuration)

[William (kc-cobradevil)]

Raring:
kinit testuser
klist
==========================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 08:28 27/02/2013 18:28 <email address hidden>
    renew until 28/02/2013 08:28
==========================================================================================================
requesting sharepointsite.testdomain with firefox with the following option set in about:config
network.negotiate-auth.trusted-uris "https://, http://"
No tickets without option rdns=false
popup window for authentication
klist
==========================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 08:28 27/02/2013 18:28 <email address hidden>
    renew until 28/02/2013 08:28
==========================================================================================================

option rdns=false
requesting sharepointsite.testdomain
klist
==========================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 07:23 27/02/2013 17:23 <email address hidden>
    renew until 28/02/2013 07:23
27/02/2013 07:24 27/02/2013 17:23 HTTP/sharepointsite.testdomain@
    renew until 28/02/2013 07:23
27/02/2013 07:24 27/02/2013 17:23 <email address hidden>
    renew until 28/02/2013 07:23
==========================================================================================================

So adding option rdns=false works for default raring install

[William (kc-cobradevil)]

Quantal
requesting sharepointsite.testdomain with firefox with the following option set in about:config
network.negotiate-auth.trusted-uris "https://, http://"
klist
====================================================================================================Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 08:35 27/02/2013 18:35 <email address hidden>
        renew until 28/02/2013 08:35
====================================================================================================

option rdns=false
klist
====================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 08:35 27/02/2013 18:35 <email address hidden>
        renew until 28/02/2013 08:35
27/02/2013 08:37 27/02/2013 18:35 HTTP/searchsite.testdomain@
        renew until 28/02/2013 08:35
27/02/2013 08:37 27/02/2013 18:35 <email address hidden>
        renew until 28/02/2013 08:35
====================================================================================================
This results in a request for a ticket for the wrong name and no sso.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Rebuilding kerberos for quantal
apt-get build-dep libkrb5-3
apt-get source libkrb5-3
edit src/lib/krb5/os/sn2princ.c
            //hints.ai_flags = AI_CANONNAME | AI_ADDRCONFIG;
            hints.ai_flags = AI_CANONNAME;

rebuild:
fakeroot debian/rules binary
dpkg -i ../libkrb5-3.........deb

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
retest Quantal
option rdns not set
requesting sharepointsite.testdomain with firefox with the following option set in about:config
network.negotiate-auth.trusted-uris "https://, http://"
klist
====================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 08:53 27/02/2013 18:53 <email address hidden>
        renew until 28/02/2013 08:53
27/02/2013 08:54 27/02/2013 18:53 HTTP/searchsite.testdomain@
        renew until 28/02/2013 08:53
27/02/2013 08:54 27/02/2013 18:53 <email address hidden>
        renew until 28/02/2013 08:53

====================================================================================================

option rdns=false
klist
====================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 08:59 27/02/2013 18:59 <email address hidden>
        renew until 28/02/2013 08:59
27/02/2013 09:00 27/02/2013 18:59 HTTP/sharepointsite.testdomain@
        renew until 28/02/2013 08:59
27/02/2013 09:00 27/02/2013 18:59 <email address hidden>
        renew until 28/02/2013 08:59
====================================================================================================

Now the setting rdns=false causes sso to ...

[William (kc-cobradevil)]

Precise

option rdns not set
requesting sharepointsite.testdomain with firefox with the following option set in about:config
network.negotiate-auth.trusted-uris "https://, http://"
klist
==========================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 09:09 27/02/2013 19:09 <email address hidden>
        renew until 28/02/2013 09:09
27/02/2013 09:10 27/02/2013 19:09 HTTP/searchsite.testdomain@
        renew until 28/02/2013 09:09
27/02/2013 09:10 27/02/2013 19:09 <email address hidden>
        renew until 28/02/2013 09:09
==========================================================================================================

option rdns=false
klist
==========================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 09:18 27/02/2013 19:18 <email address hidden>
        renew until 28/02/2013 09:18
27/02/2013 09:19 27/02/2013 19:18 HTTP/searchsite.testdomain@
        renew until 28/02/2013 09:18
27/02/2013 09:19 27/02/2013 19:18 <email address hidden>
        renew until 28/02/2013 09:18
==========================================================================================================
no sso
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Rebuilding kerberos for precise
apt-get build-dep libkrb5-3
apt-get source libkrb5-3
edit src/lib/krb5/os/sn2princ.c
            //hints.ai_flags = AI_CANONNAME | AI_ADDRCONFIG;
            hints.ai_flags = AI_CANONNAME;

rebuild:
fakeroot debian/rules binary
dpkg -i ../libkrb5-3.........deb

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
retest precise

option rdns not set
requesting sharepointsite.testdomain with firefox with the following option set in about:config
network.negotiate-auth.trusted-uris "https://, http://"
klist
==========================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 09:30 27/02/2013 19:30 <email address hidden>
        renew until 28/02/2013 09:30
27/02/2013 09:30 27/02/2013 19:30 HTTP/searchsite.testdomain@
        renew until 28/02/2013 09:30
27/02/2013 09:30 27/02/2013 19:30 <email address hidden>
        renew until 28/02/2013 09:30
==========================================================================================================

option rdns=false
klist
==========================================================================================================
Default principal: <email address hidden>

Valid starting Expires Service principal
27/02/2013 09:34 27/02/2013 19:35 <email address hidden>
        renew until 28/02/2013 09:34
27/02/2013 09:35 27/02/2013 19:35 HTTP/sharepointsite.testdomain@
 ...

[Taylor Yu (tlyu)]

I can see no obvious source code changes to the krb5 packages between Quantal and Raring that would result in the observed behavior of rdns=false functioning on stock Raring libkrb5-3 but not on Quantal. It's possible that the underlying bug in glibc got fixed in the meanwhile. I haven't confirmed the Raring result personally yet, but I do confirm that Precise is broken. I'll try to set up a public-facing test fixture soon.

[Taylor Yu (tlyu)]

Additional experimentation indicates that Raring has a partial fix to glibc that results in the observed libkrb5 behavior of rdns=false working as intended. SRUs are still a good idea for earlier Ubuntu releases. See also bug 1057526 for the underlying glibc bug.

[Benjamin Kaduk (kaduk-launchpad)]

This bug is fixed in Debian's krb5-1.10.1+dfsg-5.

[Nathan Rosenblum (nater-n)]

[Replying from a duplicating issue:]

This affects any system using MIT's Kerberos in the 1.10 series prior to 1.10.2-final. To the best of my knowledge, no 1.11 series releases were affected by this issue, and 1.9 remains affected. The upstream patch [1] applies cleanly against the Ubuntu 12.04 krb5-1.10+dfsg~beta1 source package, with which I've successfully built and deployed my own packages.

I believe that all Ubuntu versions from Precise through Saucy are affected, though maybe some of the later variants (I have only looked into Precise) have a glibc that fixes the underlying issue. There is no harm in applying both the workaround here and the glibc fix.

[1] https://github.com/krb5/krb5/commit/57738b357e8b03bcb7af2f147c97cb84d0ce96e2