chromium fails to start with non-standard home directory

[Wouter Depuydt]

CURRENT WORKAROUND
------------------

snap disconnect chromium:dot-local-share-applications
snap disconnect chromium:dot-local-share-icons

Those are just "quality of life" interfaces (c.f. LP:1732482) that most users probably do not even use, disconnecting them will cause no breakage. (Of course, a proper diagnosis and fix is due.)

ORIGINAL REPORT
---------------

cannot update snap namespace: cannot expand mount entry (none $HOME/.local/share none x-snapd.kind=ensure-dir,x-snapd.must-exist-dir=$HOME 0 0): cannot use invalid home directory "/home/department/user_name": permission denied

Fails to start on Ubuntu 22.04 for users with 'non-standard' home directory /home/department/user_name
Used to work fine up to last week.

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: chromium-browser 1:85.0.4183.83-0ubuntu2.22.04.1
ProcVersionSignature: Ubuntu 6.5.0-27.28~22.04.1-generic 6.5.13
Uname: Linux 6.5.0-27-generic x86_64
NonfreeKernelModules: zfs nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: XFCE
Date: Wed Apr 17 10:32:18 2024
InstallationDate: Installed on 2017-03-24 (2580 days ago)
InstallationMedia: Xubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1)
Snap.Changes:
 ID Status Spawn Ready Summary
 206 Error 2024-04-16T19:08:43+02:00 2024-04-16T19:16:51+02:00 Remove "chromium" snap
 207 Done 2024-04-16T19:17:27+02:00 2024-04-16T19:18:44+02:00 Remove "chromium" snap
 208 Done 2024-04-16T19:18:46+02:00 2024-04-16T19:19:47+02:00 Install "chromium" snap
 209 Done 2024-04-17T10:31:58+02:00 2024-04-17T10:32:02+02:00 Connect chromium:password-manager-service to snapd:password-manager-service
SourcePackage: chromium-browser
UpgradeStatus: No upgrade log present (probably fresh install)

[Wouter Depuydt (wouterd)]

Not a duplicate
It's specific to the latest chromium snap update: it worked fine last week. (using chromium only for specific sites. The firefox-esr PPA is my daily driver).

Other snaps don't have this problem. They're still working as they did last week.

And I'm not the only one, See also:
https://forum.snapcraft.io/t/chromium-cannot-update-snap-namespace-cannot-expand-mount-entry-none-home-local-share-none-x-snapd-kind-ensure-dir-x-snapd-must-exist-dir-home-0-0-cannot-use-invalid-home-directory/39764

[Wouter Depuydt (wouterd)]

As far as I can tell, snap and apparmor home directories are set correctly:

user_name@computer:~$ sudo snap get system homedirs
[sudo] password for user_name:
/home/department/

user_name@computer:/home$ sudo grep -R department /etc/apparmor.d/
[sudo] password for user_name:
/etc/apparmor.d/tunables/home:@{HOMEDIRS}=/home/ /home/department/
/etc/apparmor.d/tunables/home.d/snapd:@{HOMEDIRS}+="/home/department/"
/etc/apparmor.d/tunables/home.d/ubuntu:@{HOMEDIRS}+=/home/department/

[Nathan Teodosio (nteodosio)]

Thanks, I'll look into the issue, for now you can use `snap revert chromium` to use the previous version.

[Nathan Teodosio (nteodosio)]

What is 'snap connections chromium | grep personal-files'? If you disconnect the dot-local-share ones, does Chromium start?

[AP (enaira05)]

I saw in another discussion that what we (people having this issue) could have in common is an encrypted home directory. Mine is encrypted and I have the issue.
Btw, I tried 'snap revert chromium' but it reverted to a version that also doesn't start.

$ snap connections chromium | grep personal-files
personal-files chromium:chromium-config :personal-files -
personal-files chromium:dot-local-share-applications :personal-files -
personal-files chromium:dot-local-share-icons :personal-files -

How do we "disconnect the dot-local-share ones" ?

[Wouter Depuydt (wouterd)]

user_name@computer:~$ snap connections chromium | grep personal-files
personal-files chromium:chromium-config :personal-files -
personal-files chromium:dot-local-share-applications :personal-files -
personal-files chromium:dot-local-share-icons :personal-files -

disconnecting "chromium:dot-local-share-applications" and "chromium:dot-local-share-icons"
seems to fix the problem.

Thanks!

W;-)

[Wouter Depuydt (wouterd)]

PS: Can't revert as I uninstalled and reinstalled chromium yesterday when I first encountered the problem.

user_name@computer:~$ sudo snap revert chromium
error: cannot revert "chromium": no revision to revert to

[AP (enaira05)]

$ snap disconnect chromium:dot-local-share-applications
$ snap disconnect chromium:dot-local-share-icons
also helped in my case to be able to start chromium :) But for how long ...?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Nathan Teodosio (nteodosio)]

Thank you for the additional information.

enaira05, you can disconnect them with

  snap disconnect chromium:dot-local-share-applications
  snap disconnect chromium:dot-local-share-icons

wouterd, glad to hear that solves your problem.

The interface has been present for a long time. The Chromium update I released yesterday was just a new upstream release and as such introduces nothing that would trigger such an error.

On my side I can see in 'snap changes'

  1618 Done today at 12:40 CEST today at 12:43 CEST Auto-refresh snap "snapd"

So I'm placing my bets on that change.

Are you able to 'snap revert snapd' for a test? If yes, please remember to reconnect the interfaces for the test.

[AP (enaira05)]

$ snap connect chromium:dot-local-share-applications
$ snap connect chromium:dot-local-share-icons
$ snap revert snapd
2024-04-17T14:16:18+02:00 INFO Waiting for automatic snapd restart...
snapd revenu à 2.61.2
$ chromium &

also works, so snapd update seems to be the cause :)

[Wouter Depuydt (wouterd)]

I don't have an encrypted home directory (though it is on a ZFS filesystem, but that one is also NOT encrypted).

W.

[Nathan Teodosio (nteodosio)]

Thank you very much for testing that, having the input of the directly affected user is always highly valuable for debugging.

[Wouter Depuydt (wouterd)]

Reverting snapd fixes it for me too on multiple computers.

[Nathan Teodosio (nteodosio)]

Thanks!

As a note for self, I diffed snapd 2.61.2 and 2.62 and found in cmd/snap-update-ns/user.go:

--->
+func isPlausibleHome(path string) error {
+ if path == "" {
+ return fmt.Errorf("cannot allow empty path")
+ }
+ if path != filepath.Clean(path) {
+ return fmt.Errorf("cannot allow unclean path")
+ }
+ if !filepath.IsAbs(path) {
+ return fmt.Errorf("cannot allow relative path")
+ }
+ const openFlags = syscall.O_NOFOLLOW | syscall.O_CLOEXEC | syscall.O_DIRECTORY
+ fd, err := sysOpen(path, openFlags, 0)
+ if err != nil {
+ return err
+ }
+ sysClose(fd)
+ return nil
+}
<---

--->
+ if err := isPlausibleHome(realHome); err != nil {
+ realHomeError = fmt.Errorf("cannot use invalid home directory %q: %v", realHome, err)
<---

It remains to be ascertained whether this lack of permission always existed but didn't cause the launch of the snap to fail.

[Ernest Lotter (ernestl)]

As Nathan correctly suggest, this behaviour relates to a snapd 2.62 new ability introduced for the personal files interface, to create missing parent directories of write paths/files indicated in the plug declaration. Release notes: https://forum.snapcraft.io/t/the-snapd-roadmap/1973

isPlausibleHome() is an early check (does not result in termination itself) to determine if the calling user have access to its supposed home directory, as a basic way to verify that unintended user cannot exploit the mechanism.

In the reported case the personal-files interface connection results in a special type of mount entry
that instructs creation of missing parent directories between $HOME and $HOME/.local/share.

none $HOME/.local/share none x-snapd.kind=ensure-dir,x-snapd.must-exist-dir=$HOME 0 0

Because this entry exists (when the interface is connected), the result from isPlausibleHome() informs if directory creation within $HOME should be allowed or not. This is way disconnecting the interface would solve the problem.

(1) Please `stat /home/department/user_name`
(2) Does the "permission denied" result in any AppArmor denials?

[Ernest Lotter (ernestl)]

I will do some tests myself for non-standard home directory.

[Wouter Depuydt (wouterd)]

1)
user_name@computer:~$ stat /home/department/user_name/
  File: /home/department/user_name/
  Size: 267 Blocks: 289 IO Block: 16384 directory
Device: 27h/39d Inode: 9542 Links: 146
Access: (0755/drwxr-xr-x) Uid: (XXXXXXXXX/user_name) Gid: (YYYYYYYY/default_group)
Access: 2024-03-20 14:50:40.921533172 +0100
Modify: 2024-04-17 18:20:30.872510781 +0200
Change: 2024-04-17 18:20:30.872510781 +0200
 Birth: 2019-06-03 16:14:43.032386913 +0200

2) as far as I know, no apparmor specific denials.
The non-standard home directory is added to the apparmor ${HOMEDIRS} paramater
(see https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/2061981/comments/3)

The only error message I get is:

user_name@computer:~$ chromium
cannot update snap namespace: cannot expand mount entry (none $HOME/.local/share none x-snapd.kind=ensure-dir,x-snapd.must-exist-dir=$HOME 0 0): cannot use invalid home directory "/home/department/user_name": permission denied
snap-update-ns failed with code 1

[Zygmunt Krynicki (zyga)]

We've debugged this issue and the problem is caused by apparmor re-execution and configuration files being loaded from read-only snapd squashfs, thus making HOMEDIRS customization invisible.

[Zygmunt Krynicki (zyga)]

We are still debugging it, it's not this. It's related to HOMEDIRS tunable and variable expansion, somehow.

[Zygmunt Krynicki (zyga)]

This bug is now well-understood and a fix is proposed in: https://github.com/snapcore/snapd/pull/13853

[Ernest Lotter (ernestl)]

Root cause was identified:

In the case of non-deb install (re-exec), Apparmor tunables from host /etc/apparmor.d/tunables/home.d/...
was not included in the profile for snap-update-ns. The a snapd 2.60 PR that aimed to fix home directories outside of /home/. missed adding it for the case of snap-update-ns template.

(1) fixed
https://github.com/snapcore/snapd/pull/13853

(2) and similar issue prevented in the future
https://github.com/snapcore/snapd/pull/13854

Please re-test on snapd edge tomorrow (after the nightly build that will include the fix).
It will be available in 2.63 which we are building today and releasing in approx 3 weeks.