[snap] kerberos GSSAPI no longer works after deb->snap transition

[st0ve]

I configure AuthServerWhitelist as documented:

https://www.chromium.org/developers/design-documents/http-authentication

and can see my whitelisted domains in chrome://policy/

but websites that used to work with SPNEGO/GSSAPI/kerberos no longer work. I'm guessing the snap needs some sort of permission to use the kerberos ticket cache (or the plumbing to do so doesn't exist...).

I can confirm that Chrome has the desired behavior.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Olivier Tilloy (osomon)]

Thanks for the report.
Can you check for apparmor denials in the system journal when reproducing the problem? Run the following command in a terminal before launching chromium:

    journalctl -f | grep DEN

[st0ve (st0ve)]

The /etc/gss/mech.d/ and /etc/krb5.conf.d/ denials may be relevant. Both directories are empty in my case, but lack of access may be killing some logic that relies on checking them.

[Olivier Tilloy (osomon)]

Thanks, that's useful.

I'm not familiar with SPNEGO/GSSAPI/kerberos, could you maybe come up with easy steps to reproduce the problem on a clean system? That would allow me to dig further into the problem.

[Vitezslav Kotrla (vitezslav-kotrla)]

The same problem here, after upgrading to 'snapped' chromium 79 we lost Single Sign-On, all our Kerberos security based intranet web servers started asking for username and password.

Kerberos ticket cache is file /tmp/krb5cc_<uid>:
<pre>
johndoe@computer:~$ klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: <email address hidden>

Valid starting Expires Service principal
3.2.2020 12:39:10 3.2.2020 22:39:10 <email address hidden>
 renew until 4.2.2020 12:39:10
3.2.2020 12:42:39 3.2.2020 22:39:10 <email address hidden>
</pre>
<pre>
johndoe@computer:~$ ls -la /tmp/krb5cc_1000
-rw------- 1 johndoe johndoe 3125 úno 3 12:42 /tmp/krb5cc_1000
</pre>

[Stefan Priebe (s-priebe)]

Sorry this sucks guys - when is it going to get fixed? Ubuntu 20.04 also stopped working for me since the transition to snap happend - all kerberos and gssapi authentications no longer work

[Alexey Tsitsin (cicin)]

This problem still persist and SPNEGO won't work even with new policies:

https://cloud.google.com/docs/chrome-enterprise/policies/?policy=AuthServerAllowlist
https://cloud.google.com/docs/chrome-enterprise/policies/?policy=AuthNegotiateDelegateAllowlist

The policies are loaded successfully but kerboros negotiation won't work

[Sebastien Bacher (seb128)]

Why do you think it's a security issue?

[Alexey Tsitsin (cicin)]

Sorry, I changed the issue's type accidently.

[Olivier Tilloy (osomon)]

The snap should have the required libraries to support kerberos authentication, but it's likely that confinement is getting in the way. Does kerberos allow verbose logging on the server end, to inspect where authentication is failing?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Sebastien Bacher (seb128)]

Unsure why thunderbird is listed there, it's not mentioned in the description nor posts, could you give some details on what isn't working and how?

[Marian+mozilla (marian+mozilla)]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.99 Safari/537.36

Steps to reproduce:

This issue concerns the Firefox snap package. I have configured Firefox to use SPNEGO authentication against my authentication server using the policy `Authentication/SPNEGO` (as documented at https://github.com/mozilla/policy-templates/blob/master/README.md#authentication). Firefox shows the policy in `about:policies` and the corresponding setting `network.negotiate-auth.trusted-uris` in `about:config`, so the policy is found and applied correctly.

Actual results:

Even though the policy is active, Firefox does not attempt Kerberos authentication against my authentication server. The exact same policy DOES work with the regular deb-based version of Firefox, so the issue has to be in the snap package.

I guess that the snap version does not have access to the required files and/or environment variables. I logged which files and directories the deb-based Firefox accesses that seem to have to do with Kerberos/SPNEGO using `strace -f -t -e trace=file firefox` on my system running Ubuntu 21.10 beta:
```
/lib/x86_64-linux-gnu/libgssapi_krb5.so.2
/lib/x86_64-linux-gnu/libkrb5.so.3
/lib/x86_64-linux-gnu/libk5crypto.so.3
/lib/x86_64-linux-gnu/libkrb5support.so.0
/etc/gss/mech
/etc/gss/mech.d
/etc/krb5.conf
/etc/krb5/user/10017/client.keytab
/usr/share/locale/*/LC_MESSAGES/mit-krb5.mo
/usr/share/locale-langpack/*/LC_MESSAGES/mit-krb5.mo
/tmp/krb5cc_10017_QfHqc3
```
`10017` is the user ID of the user running firefox. The last file `/tmp/krb5cc_10017_QfHqc3` is the user's Kerberos ticket cache, which is given by the environment variable `KRB5CCNAME`.

So the first step would be to allow the snap to access the listed files and directories, as well as to the environment variable `KRB5CCNAME`. Of course, the list is just generated by looking at the deb-based Firefox on my system and might not be complete.

In any case, I'd be happy to test an updated snap.

Expected results:

Kerberos/SPNEGO authentication should work the same as in the deb-based Firefox.

[Martin Petersen (martux69)]

Same problem here, chromium is not able to use kerberos ticket. I think, it is time to get back chromium as a deb package until snap is really working.

[Marian Rainer-Harbach (marianrh)]

Maybe the information I collected here https://bugzilla.mozilla.org/show_bug.cgi?id=1734791 for the Firefox snap, which suffers from the same problem, is helpful in order to fix the problem for the Chromium snap as well.

[Andrei-purice (andrei-purice)]

Setting a component for this issue in order to get the dev team involved.
If you feel it's an incorrect one please feel free to change it to a more appropriate one.

[Daniel Calcoen (daniel-calcoen)]

Today my brand new Ubuntu 21.10 Impish has forced the change of Firefox as Snap, so I'm suffering Kerberos not working from inside the Firefox snap.
Kerberos works fine at Linux level.KInit, KList, etc... shows that the tickets are assigned and handle correctly when requested.
Some closed door seems to isolate the snap from the real world...

[Olivier Tilloy (osomon)]

Triagers, please update the component to "Release Automation: Snap" and add a blocks reference to bug 1665641. Thanks!

[Daniel Calcoen (daniel-calcoen)]

Which is different in our case that for normal people is that the use of Kerberos requires to set in firefox the preference "network.negotiate-auth.trusted-uris" which by default is not set.
In my case it is set as network.negotiate-auth.trusted-uris=cern.ch

I have everything setup correctly, linux and firefox snap, and the problem present so if somebody could guide me a bit I can output any log file/debug you wish to spot where the magic disappears.

[Daniel Calcoen (daniel-calcoen)]

There are several references across the internet about similar problems.

https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1849346

Firefox with Kerberos/SPENGO works fine in the non snap version and Firefox snap version don't.
Seems some door is closed in the snap.

[Daniel Calcoen (daniel-calcoen)]

There is a similar bug follow up at https://bugzilla.mozilla.org/show_bug.cgi?id=1734791

[Daniel Calcoen (daniel-calcoen)]

In my brand new Ubuntu 21.10 Impish has forced the change of Firefox as Snap, so I'm suffering Kerberos not working from inside the Firefox snap.
Kerberos works fine at Linux level.KInit, KList, etc... shows that the tickets are assigned and handle correctly when requested.

Which is different in our case that for normal people is that the use of Kerberos requires to set in firefox the preference "network.negotiate-auth.trusted-uris" which by default is not set.
In my case it is set as network.negotiate-auth.trusted-uris=cern.ch

I have everything setup correctly, linux and firefox snap, and the problem present so if somebody could guide me a bit I can output any log file/debug you wish to spot where the magic disappears.

[frigo (rigault-francois)]

if the goal is to have a single snap making use of the kerberos ticket, as a workaround you can put something like this in /etc/krb5.conf

[libdefaults]
        default_ccache_name = DIR:/home/%{username}/snap/firefox/common/.cache/.k5_ccache

the default connections for the firefox snap prevent it to read hidden files under $HOME, so we help it a little. Seems to work so far

* not ideal for security though as Firefox snap has direct access to the TGT.

[Luke Faraone (lfaraone)]

Unless the snap move was intended to provide isolation from Kerberos, setting to medium because this breaks many enterprise usecases for Firefox.

[Marian Rainer-Harbach (marianrh)]

The same applies to Chromium, please set it to medium as well, Luke. Thanks!

[Daniel Calcoen (daniel-calcoen)]

@frigo Thanks for the tip.
Unfortunately my systems requires pam_krb5 which has precedence over default_ccache_name and sets KRB5CCNAME directly.
I tried to set ccache_dir in the pam section of krb5.conf but I didn't manage.
While waiting for this to be fixed I'll continue with the not-snap version of firefox ...

[Georges Toth (georges-c)]

Being on Ubuntu 22.04, the .deb package has been removed and one is forced to use the snap version.

kerberos does not work with the latest version and this is quite a blocker for me as there is no alternative (other than switching browsers).

[Mantas Mikulėnas (grawity)]

The magic disappears in Firefox's AppArmor profile, which doesn't allow it to access `/tmp/krb5cc_*`. As an easy workaround until the Snap configuration is fixed, edit `/etc/krb5.conf` to relocate your Kerberos ticket cache somewhere Firefox *can* access it:

```
[libdefaults]
 default_ccache_name = FILE:/home/%{username}/krb5cc
```

(Don't forget to re-`kinit`.)

---

In addition to the AppArmor problems, the snap is also missing the `krb5/plugins/tls/k5tls.so` module that's required to access KDCs via MS-KKDCP (aka KdcProxy). Now _most_ realms should work fine without the k5tls plugin, but in some cases it might be necessary to manually specify non-proxied KDC hostnames in krb5.conf `[realms]`. (If you're using Azure AD Kerberos, you're out of luck.)

The magic environment variables to reveal such problems are `KRB5_TRACE=/dev/stderr NSPR_LOG_MODULES=negotiateauth:5`.

[Mantas Mikulėnas (grawity)]

(In reply to Mantas M. (grawity) from comment #6)
> The magic disappears in Firefox's AppArmor profile, which doesn't allow it to access `/tmp/krb5cc_*`. As an easy workaround until the Snap configuration is fixed, edit `/etc/krb5.conf` to relocate your Kerberos ticket cache somewhere Firefox *can* access it:
>
> ```
> [libdefaults]
> default_ccache_name = FILE:/home/%{username}/krb5cc
> ```

I just realized this won't work if you're also using Kerberos for NFS or AFS. :(

[Release-mgmt-account-bot (release-mgmt-account-bot)]

The severity field is not set for this bug.
:gerard-majax, could you have a look please?

For more information, please visit [auto_nag documentation](https://wiki.mozilla.org/Release_Management/autonag#workflow.2Fno_severity.py).

[Bugzilla-8 (bugzilla-8)]

My workplace was hit with this earlier this week (Upgrading from Ubuntu 20.04 -> 22.04.1). Just like to point out that KRB5CCNAME can refer to several different cache storage [alternatives](https://web.mit.edu/kerberos/krb5-1.12/doc/basic/ccache_def.html#ccache-types), for example I use `KEYRING:persistent:uidnumber` so an updated apparmor profile granting it access to `/tmp/krb5cc_*` would not solve it for me.

[andlla (andlla)]

Hallo,
I can confirm this is a problem also for us,
with the latest firefox and chromium from snap store,

including the limited applicability of the workaround regarding ccache_name location

thanks

[Götz Waschk (goetz-waschk)]

This should have been a show-stopper for Firefox Snap.

[Rigoberto Sanchez (sanch1)]

This is a problem for snap versions of Chromium and Firefox, they are unable to access the kerberos ticket and also (not sure why) /etc/gss/mech.d/

[Magnus M (magnusm)]

I've also experienced this bug and it's a show-stopper for either upgrading or requires changing distro, and makes Ubuntu unusable for many enterprises. We don't accept user/password combination so a fallback to that from Kerberos isn't possible and moving the ccache isn't possible or supported either.

Also, since the snap:ification, the browser doesn't use system certificates store anymore, which makes certificates break for all websites when for example intermediate certs are rotated in the system store, which is a security issue since users will be taught to accept expired certificates.

[Lissyx+mozillians (lissyx+mozillians)]

Olivier, is there any progress on that matter?

[Olivier Tilloy (osomon)]

No, this isn't being worked on.

[Vegard Søbstad Alsli (alslinet)]

I hope this gets some attention very soon. Its a huge issue for enterprise users.

[Vegardalsli (vegardalsli)]

I can confirm this is still a huge issue i have hundreds of users with this problem in my environment.
This is both a big security and productivity issue for my users.

How can we get this fixed?

[Pferrell (pferrell)]

I'll second that. We've got a lot of users with this issue in our environment as well.

[Daniel Calcoen (daniel-calcoen)]

I'll second that too.

[Lissyx+mozillians (lissyx+mozillians)]

(In reply to vegardalsli from comment #12)
> I can confirm this is still a huge issue i have hundreds of users with this problem in my environment.
> This is both a big security and productivity issue for my users.
>
> How can we get this fixed?

Unfortunately, this is an upstream issue, and I have no time to help there.

[Vegardalsli (vegardalsli)]

Im not sure what this beeing an upstream issue means. Is there a bug or a point of contact upstream that we can get in contact with?
Thank you for increasing the priority of this case btw.

[supremesyntax (supremesyntax)]

If you force people into a snap package at least there should be no loss in functionality.
I have linked the snapd project so maybe someone from there could have a look.

[Lissyx+mozillians (lissyx+mozillians)]

(In reply to vegardalsli from comment #16)
> Im not sure what this beeing an upstream issue means. Is there a bug or a point of contact upstream that we can get in contact with?
> Thank you for increasing the priority of this case btw.

It means that the issue is not in our codebase but it's a ubuntu-level issue. Please look at `See also` for https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1849346

[Lissyx+mozillians (lissyx+mozillians)]

Amin, is this on any roadmap?

[Amin Bandali (bandali)]

I think we're hoping to tackle this for the next (Ubuntu) cycle.

[Lissyx+mozillians (lissyx+mozillians)]

*** Bug 1831722 has been marked as a duplicate of this bug. ***

[Vegard Søbstad Alsli (alslinet)]

Any updates on this?