when /home is somewhere else, snaps don't work

[Marcin]

Problem:
$ hello-world
cannot bind mount /home to /tmp/snap.rootfs_vAgIND/home. errmsg: Permission denied

When /home is a symlink snaps don't work.
When /home is a real directory snaps work, see output below

Output:
marcin@ubuntu:~$ snap list
Name Version Rev Developer Notes
hello-test 0.01 1 suncheul-kim -
hello-world 6.3 27 canonical -
ubuntu-core 16.04.1 352 canonical -
marcin@ubuntu:~$ hello-world
cannot bind mount /home to /tmp/snap.rootfs_eILRgd/home. errmsg: Permission denied
marcin@ubuntu:~$ hello-test.hello
cannot bind mount /home to /tmp/snap.rootfs_A9iYc5/home. errmsg: Permission denied
marcin@ubuntu:~$ sudo rm -R /home && sudo mkdir -p /home/${whoami} && sudo chmod ugo+rwx /home/$whoami
marcin@ubuntu:~$ hello-test.hello
Hello, world!
marcin@ubuntu:~$ hello-world
Hello World!
marcin@ubuntu:~$ sudo rm -R /home && sudo ln -s /media/Dane/.home/ /home
marcin@ubuntu:~$ hello-world
cannot bind mount /home to /tmp/snap.rootfs_vAgIND/home. errmsg: Permission denied
marcin@ubuntu:~$

My configuration:

$ ll /home
lrwxrwxrwx 1 root root 18 Sep 6 20:02 /home -> /media/Dane/.home//

As /media/Dane is encrypted with LUKS+ext4 fs. I tried to use hardlink but they are not allowed for directories:(

[Marcin (mwoz123)]

It also doesn't work when user home is a symlink (and /home "normal" directory) :

marcin@ubuntu:~$ sudo rm -R /home && sudo mkdir -p /home/
marcin@ubuntu:~$ sudo ln -s /media/Dane/.home/marcin/ /home/marcin
marcin@ubuntu:~$ hello-world
failed to create user data directory. errmsg: Too many levels of symbolic links
marcin@ubuntu:~$

[Marcin (mwoz123)]

@Sergio any reasons/explanations why you changed bug status to "invalid"?
Do you need more information?

[Gustavo Niemeyer (niemeyer)]

Hi Marcin,

Sergio just moved over the issue from snapcraft to snapd.

That said, I'm not sure there's much we can do about this. We have strict permissions enforced via apparmor which depend on things being on the correct place.

Have you tried to bind-mount your home instead of symlinking it?

[Sergio Schvezov (sergiusens)]

Right, I marked it as Invalid from the point of view of snapcraft; I did not dismiss the problem out of hand as I added the snappy project as well.

[Zygmunt Krynicki (zyga)]

Current design of snap-confine makes using symbolic links as the home directory problematic. Snap-confine itself is heavily confined to prevent abuse (as it is a seguid-root executable) and snap-confine carefully avoids traversing symbolic links implicitly. Even when we do the apparmor profile for snap-confine prevents it from performing certain operations, this includes arbitrary source and destination of various mount operations it performs internally.

As a quick workaround, I suspect that you can bind mount custom home directory back to your regular directory (see mount --bind /source /destination) and this should allow you to have the desired data layout and the ability to execute snaps.

[Jamie Strandboge (jdstrand)]

Actually you don't have to do bind mounts. One of the reasons we use apparmor variables for HOME is so that an administrator can adjust what HOME expands to for situations like this.

Feel free to do one of:
$ sudo dpkg-reconfigure apparmor

Or drop a file into /etc/apparmor.d/tunables/home.d that has:
@{HOMEDIRS}+=/media/Dane/.home/

[Jamie Strandboge (jdstrand)]

I forgot to mention you'll need to reload the policy after doing this. So after adjust the HOMEDIRS apparmor variable as stated in comment #6, I suggest:

$ sudo rm -f /etc/apparmor.d/cache/* /var/cache/apparmor/snap.*
$ sudo reboot

(there are other ways to do this without a reboot, but this should work regardless of what apparmor policy you have already generated).

[Marcin (mwoz123)]

Solved (or rather workaround) :
by using as @zyga suggested mount --bind option.
I added to /etc/fstab entry:
/media/Dane/.home/ /home none bind

[Terry L. Triplett (terry-triplett)]

I seem to have encountered a slight variant of this issue. snaps don't work when the homedir is a real directory, but not in /home.

Scenario:

A user's home is defined in /etc/passwd as "/nonstandard/home/rebel".

$> ECHO $HOME

/nonstandard/home/rebel

$>hello-world

cannot create user data directory: /nonstandard/home/rebel/snap/hello-world/27: Read-only file system

That's interesting, because that directory is in fact created when running hello-world for the first time:

$tree ./snap

snap
└── hello-world
    ├── 27
    └── common

On the same system, hello-world works fine for a user under /home:

$> ECHO $HOME

/home/wellbehaved

$> hello-world

Hello World!

I will try the workaround in comments #6-#7.

[Terry L. Triplett (terry-triplett)]

Update: Putting this into /etc/apparmor.d/tunables/home.d/nonstandard_homedirs:

@{HOMEDIRS}+=/nonstandard/home/

Did not work. hello-world still fails claiming read-only filesystem (after writing to it successfully).

Exploring other workarounds. Can't bind mount to /home, because valid homedirs exist there already.

[Jamie Strandboge (jdstrand)]

@Terry,

Did you reload the profiles? In particular, you need to reload /etc/apparmor.d/usr.lib.snapd.snap-confine and /var/lib/snap/apparmor/profiles. Eg:

sudo rm -f /etc/apparmor.d/cache/usr.lib.snapd.snap-confine /var/cache/apparmor/snap.*
sudo apparmor_parser -r /etc/apparmor.d/usr.lib.snapd.snap-confine /var/lib/snapd/apparmor/profiles/*

(or reboot after removing the cache files).

[Terry L. Triplett (terry-triplett)]

@Jamie,

Yes. The reboot was the reason for the break between the "I will try ..." and "Update:" follow up message. Just to be sure. :-)

I tried reloading the profiles first, then rebooted to drive the point home in case the computer was being stubborn. Same result both times.

I did a full experiment to eliminate the possibility that there was something in my existing accounts that was causing the difference.

$ cat /etc/apparmor.d/tunables/home.d/snaptest

# The following is a space-separated list of where additional user home
# directories are stored, each must have a trailing '/'. Directories added
# here are appended to @{HOMEDIRS}. See tunables/home for details.
@{HOMEDIRS}+=/nonstandard/home/

At this point I did both the profile reload you suggested, and also tried deleting the cache files + reboot. In either case the following was the result. (Cleaned up in between, I created the test accounts from scratch after in-place profile reload and after delete+reboot).

** General info **

Test system is KDE Neon. Created by doing a basic install from standard Ubuntu 16.04 install media and then adding the Neon repos afterward. From all outward appearances, this system behaves the same as all my Ubuntu 16.04 systems, which include several Ubuntu Server, Ubuntu MATE and Neon installs. The underlying base system seems to be identical on all of them - they differ only in the desktop components.

** Non standard home location **
/nonstandard is a separate LVM partition on my system, for what it's worth.

mount shows:

/dev/mapper/VG-NONSTANDARD on /nonstandard type ext4 (rw,relatime,data=ordered)

ls -l /nonstandard

drwxr-xr-x 6 root root 4096 Feb 21 14:34 home
drwx------ 2 root users 16384 Apr 4 2004 lost+found

** The test **

$ sudo adduser snaptest-goodfella

* standard adduser output here *

$ sudo adduser snaptest-rebel --home /nonstandard/home/snaptest-rebel

* standard adduser output here *

$ su -l snaptest-goodfella

$ pwd
/home/snaptest-goodfella

$ hello-world
Hello World!

$ tree
.
└── snap
    └── hello-world
        ├── 27
        └── common

$ exit

$ su -l snaptest-rebel

$ pwd
/nonstandard/home/snaptest-rebel

$ hello-world
cannot create user data directory: /nonstandard/home/snaptest-rebel/snap/hello-world/27: Read-only file system

$ tree
.
└── snap
    └── hello-world
        ├── 27
        └── common

$ exit

There's always the chance that there's some peculiarity that has crept into my system over time, so I can test this on a few of my other machines as time permits.

[Terry L. Triplett (terry-triplett)]

It seems not. I just stopped using snaps since they don't work for me.
I've got a good reason for not placing certain homedirs under /home,
which is after all just a convention and not a requirement. Haven't had
a single problem with using non-conventional homedirs for 35 years with
nary a problem until now.

On 12/20/2017 09:18 AM, Usievaład Kimajeŭ wrote:
> No updates, I see.
>

[Jamie Strandboge (jdstrand)]

There are updated workaround procedures listed here:

https://forum.snapcraft.io/t/how-can-i-use-snap-when-i-dont-use-home-user/3352/2

Note that progress has been made on this bug by Zygmunt through the addition of the /var/lib/snapd/apparmor/snap-confine directory. Currently snapd only uses it itself for NFS home, but the design was intended to also address this bug.

[Usievaład Kimajeŭ (anibyl)]

This workaround doesn't work actually.

[Jamie Strandboge (jdstrand)]

I'm not sure how far you read in the forum, but see: https://forum.snapcraft.io/t/how-can-i-use-snap-when-i-dont-use-home-user/3352/10. Basically, you need to perform a bind mount for it to work due to changes to snap-confine.

[Daan W. (dwynen)]

Just to add to this, I got "Too many symbolic links" because ~/snap was a symlink.
The reason for this was that ~ is on NFS with a tight quota, so this was meant as a workaround.

[System V. Unix (sysv)]

Adding the non-standard home directories to {HOMEDIRS}, adding a bind mount from /nonstandard/home to /home still causes errors, and it not a valid workaround.

$ opera
cannot create user data directory: /nonstandard/home/sysv/snap/opera/10: Read-only file system

This is really disappointing in the quality of apparmor/snap and Ubuntu 18.04 release. Do better. People have been using non-standard paths for homedirs now, like,...for 30 years on Unix/Linux/BSD. Deal with it.

[Bill Korb (korb)]

I agree with the others that have objected to this, as this has entirely broken Ubuntu 18.04 at our company, too. We have a NAS which hosts all user HOME directories, from wherever they may log in.

Unfortunately, this snap stuff fails miserably in such an environment, and there should be a way for you to accomplish your goal without breaking something that we've been using on various *nixes for 30+ years (e.g. Solaris, AIX, HP-UX, SUSE, RHEL, and even Ubuntu up to recently). We are still running Ubuntu 12.04 in some places as your NFS support has gradually degraded over the years, and we're seriously contemplating moving to CentOS/RHEL which seems to be much more enterprise-friendly.

[reetp (jcrisp)]

We have experienced this on Trusty 14.04 I was testing to see what might happen on an upgrade.

We run SSSD for logons and a have directory on disk (not a remote mount) of:

/home/server/files/users/username

I've tried the suggested apparmor fixes with no joy.

sudo dpkg-reconfigure apparmor

cat /etc/apparmor.d/tunables/home.d/ubuntu
# This file is auto-generated. It is recommended you update it using:
# $ sudo dpkg-reconfigure apparmor
#
# The following is a space-separated list of where additional user home
# directories are stored, each must have a trailing '/'. Directories added
# here are appended to @{HOMEDIRS}. See tunables/home for details.
@{HOMEDIRS}+=/home/server/files/users/

sudo apparmor_parser -r /etc/apparmor.d/*snap-confine*
sudo apparmor_parser -r /var/lib/snapd/apparmor/profiles/*

Note the following seems to have stripped /users/ off the above directory:

Nov 22 16:42:29 desktop kernel: [ 2875.968601] audit: type=1400 audit(1542901349.258:67): apparmor="DENIED" operation="open" profile="/snap/core/5897/usr/lib/snapd/snap-confine" name="/home/server/files/" pid=6254 comm="snap-confine" requested_mask="r" denied_mask="r" fsuid=5001 ouid=0

I also tried:

/var/lib/snapd/apparmor/snap-confine/my-homes

# home directories are in /foo/bar, not /home
mount options=(rw rbind) /home/server/files/users/ -> /tmp/snap.rootfs_*/home/,

sudo apparmor_parser -r /etc/apparmor.d/*snap-confine*
sudo apparmor_parser -r /var/lib/snapd/apparmor/profiles/*

That didn't seem to survive a reboot either.

Package: snapd
Status: install ok installed
Priority: optional
Section: devel
Installed-Size: 80575
Maintainer: Ubuntu Developers <email address hidden>
Architecture: amd64
Version: 2.34.2~14.04

I don't want to do a fstab bind mount as my users get confused easily enough already..... one home is enough.

Sorry, but this is shockingly bad.

[Justyn Butler (justyn)]

I have also been unable to get the workarounds to work on Ubuntu 18.04.

I'm genuinely surprised and disappointed at how poor this is.

[Robert Gerstman (fishratcow)]

 I used bindfs to solve this problem.
The bind option to mount wasn't being accepted and none of the other
solutions posted above worked.

I added this to /etc/fstab for /root with the permissions of 0700.

/somewhere/else/root /root fuse.bindfs perms=0700,nonempty 0 0

[Steve Langasek (vorlon)]

> cat /etc/apparmor.d/tunables/home.d/ubuntu
> # This file is auto-generated. It is recommended you update it using:
> # $ sudo dpkg-reconfigure apparmor
> #
> # The following is a space-separated list of where additional user home
> # directories are stored, each must have a trailing '/'. Directories added
> # here are appended to @{HOMEDIRS}. See tunables/home for details.
> @{HOMEDIRS}+=/home/server/files/users/

> sudo apparmor_parser -r /etc/apparmor.d/*snap-confine*
> sudo apparmor_parser -r /var/lib/snapd/apparmor/profiles/*

> Note the following seems to have stripped /users/ off the above
> directory:

> Nov 22 16:42:29 desktop kernel: [ 2875.968601] audit: type=1400
> audit(1542901349.258:67): apparmor="DENIED" operation="open"
> profile="/snap/core/5897/usr/lib/snapd/snap-confine"
> name="/home/server/files/" pid=6254 comm="snap-confine"
> requested_mask="r" denied_mask="r" fsuid=5001 ouid=0

What this in fact shows is that something is trying to access /home/server/files/ and is denied access. Do you perhaps have users who have a home directory of /home/server/files? Or, are there symlinks somewhere pointing to /home/server/files as the target? From your description of your setup, it is not expected that snaps are trying to access /home/server/files directly.

[reetp (jcrisp)]

>What this in fact shows is that something is trying to access /home/server/files/ and is denied access.

Yes, I figured that bit out..... the question is why.

>Do you perhaps have users who have a home directory of /home/server/files?

Not as far as I am aware - see below.

I do have one 'local' user in

/home/localadmin

That accounts is no SSSD and used for local administration only.

>Or, are there symlinks somewhere pointing to /home/server/files as the target?

Not as far as I am aware

>From your description of your setup, it is not expected that snaps are trying to access /home/server/files directly.

No they should be accessing a user directory which make it even more peculiar.

Each user that logs in has a /home/server/files/users directory with all local files.

They also have one Mounts directory in their home that has mapped shares on the server that are added with SSSD/Pam. When you log out they are disconnected.

john@desktop:~$ ll /home/server/files
total 12
drwxr-xr-x 3 root root 4096 Feb 11 2017 ./
drwxr-xr-x 3 root root 4096 Feb 11 2017 ../
drwxr-xr-x 9 root root 4096 Nov 29 2017 users/

john@desktop:~$ ll /home/server/files/users
total 36
drwxr-xr-x 9 root root 4096 Nov 29 2017 ./
drwxr-xr-x 3 root root 4096 Feb 11 2017 ../
drwx------ 32 denise denise 4096 Mar 8 2018 denise/
drwx------ 59 john john 4096 Nov 25 20:40 john/
drwx------ 58 nicky nicky 4096 Jul 27 11:51 nicky/

This should be correct, but clearly something doesn't agree.

@{HOMEDIRS}+=/home/server/files/users/

[Fred Drake (fdrake)]

I'm seeing the "cannot create user data directory" error when my $HOME is an ordinary home directory in /home/DOMAIN/, where we're using AD for authentication, and DOMAIN is the AD domain being used. (Of course, the actual ~/snap/kubectl/677 directory is created at some point.)

Does /home/DOMAIN/ need to be added to @{HOMEDIRS}, or is this another unsupported configuration?

This is all on a vanilla Ubuntu 18.10 (cosmic) desktop install.

[Yannis Milios (yannis-milios)]

Hello,

I'm testing snaps in CentOS7 and I ran through this problem. Is there a workaround for CentOS, given that AppArmor is an Ubuntu thing AFAIK (CentOS equivalent is SELinux I think, which is currently disabled on my machine) ?

Thanks,
Yannis

[Ivo Cavalcante (ivo-cavalcante)]

Hi all,

Exactly same problem as #27 : CentOS 7, non-default home dir (corporate environment, too many users to change). Any way to solve? Snaps are currently unusable.

Thanks,
Ivo

[Gargoyle (g-rgoyle)]

+1

# Steps to reproduce
1.) Clean Ubuntu 18.04 install + Jenkins + lxd snap.
2.) su - Jenkins
3.) lxc list

Above fails with readonly filesystem error because by default Jenkins' homedir is /var/lib/jenkins.

# Workaround
1.) Shutdown Jenkins
2.) mv /var/lib/jenkins /home/jenkins
3.) edit /etc/password and change jenkins users homedir (or use usermod)
4.) edit /etc/default/jenkins and change $JENKINS_HOME to match (ie JENKINS_HOME=/home/$NAME)

Adding a file /etc/apparmor.d/tunables/home.d/jenkins with the @{HOMEDIRS} as suggested above didn't work for me either.

I would have thought using lxc containers to perform automated builds and testing with jenkins is a fairly common & practical use case, so it would be good to figure out a reliable fix for this.

[Patrick Banholzer (patrick-banholzer)]

I have the same issue for about 1500 Users having their home not at /home

[Jacques-Olivier Farcy (jakatak)]

Hi,

On OpenSUSE 15.0, i have a homedir mounted on a CIFS network share for all the student and teacher in our university.

I installed two applications : visualfm and cloudcompare.
I have the error for both :

cannot open path of the current working directory: Permission denied

Even if snap is able to create my /homedir/snap directory.

[Vijay Raghav (iron-phoenix)]

I have also the same issue (Corporate Environment), now snap is unusable

[cement_head (andorjkiss)]

HOLY COWS BATMAN! This is completely ridonculous. We have a system that uses a small (1 TB SSD) for OS/Login and $HOME(s) are on a 25TB RAID10 system. SNAPs are now unusable? This is bonkers....

[Zygmunt Krynicki (zyga)]

Dear @cement_head, you can always mount your home directory under /home/$LOGNAME, even if it is on a 25TB RAID10 system. It's not about where it comes from, it's about where it is in the system. That's all.

[John Lenton (chipaca)]

Setting this to Won'tFix, as it's unlikely we'll fix it for the foreseeable future -- the workaround for people insisting on non-standard home locations should work for most.

[John Lenton (chipaca)]

FWIW this is mentioned explicitly in "Limitations in snapd", https://forum.snapcraft.io/t/limitations-in-snapd/9718

[ericc (eric-cheminot)]

Hello,

I have a similar behaviour, as mentioned in comment #1 - that is not (I think) covered by the limitations linked just above (#36).

The homedirs is standard (/home), but for certain users, their home dir is a symlink to another folder: /home/user1 -> /accounts/user1. In this case, /home/user1/snap is "standard" layout, but snap keeps pretending that e.g. "/home/user1/snap/spotify/36" is not a directory - which is not true.

Comment #34 is not correct for me - even though the argument is logical, it is unfortunately not true. It may come from the facts that I'm using BTRFS volumes (I've already seen specific behaviours with containers and BTRFS, but I'm not technical enough to draw a conclusion). So, no, it's not "all".

Also, adding /accounts to apparmor homedirs is without effect here.

[Oliver Grawert (ogra)]

@ericc if you switch these dirs from symlinks to being bind mounts it will work ...

i guess @zyga should add this to his list of limitations (symlinks do in general not work with apparmor, but bind-mounts are a safe replacement for them)

[ericc (eric-cheminot)]

@ogra Thanks, I was not sure I could do this (again, not my domain). I have multiple users having their home under /accounts mount:
  /home/user1 -> (links to) /accounts/user1
  /home/user2 -> /accounts/user2

So the only volume candidate for bind mounting for me would be /accounts. The only solution would be to have a sub-volume for each user. I've just tried with a dummy user, but while I can I can mount @dummy subvol to /home/dummy, it complains that /home/dummy is not a folder when adding "bind" option:
  # mount -o subvol=@dummy /dev/sdc1 /home/dummy => [OK]
  # mount _o bind,subvol=@dummy /dev/sdc1 /home/dummy => [KO, /home/dummy not a folder (or directory, I'm not using an English console]

But... actually, following your suggestion I've insisted on the proposed solution and I can do a mount bind at the user folder level, that is exactly instead of links ((I did not know that). And, yes, it works now! Thanks!

[Peter Teuben (teuben)]

Sad to see this restriction on snaps.

One other work around is to maintain an alias list. I actually only use one snap (now), and I so I have:

alias yakyak=/snap/yakyak/current/yakyak/yakyak

but you have to hunt them down, not every snap uses that kind of directory structure. And some snaps maybe a lot more complicated. So this is no fool proof solution, but I only have one snap for now :-)

[Oliver Grawert (ogra)]

this alias completely breaks snap confinement an it is a matter of sheer luck that you can even run it at all this way (because none of the library paths the snap confinement setup establishes on app startup will be around) ...

you are just lucky that you are running the correct versions of libs on you host, but an update might break everything ...

if you'd actually want to run such a binary without confinement, i'd recommend reading all the different wrappers shipped in the snap and replicating their behaviour to have all environment variables set properly ...

[Peter Teuben (teuben)]

ok,thanks. I will actually try other suggested solutions, such at mount bind, and eventually maybe I'll just have to get a real /home again, sigh.

[reetp (jcrisp)]

> maybe I'll just have to get a real /home again, sigh.

You have a real home. It just isn't where they want you to have it. I don't believe there are any laws saying exactly where your personal home directory should be beyond somewhere in /home

And that /home can be anywhere. Same partition, different partition, different building.

This is just a case of them making an assumption about how people work, and then not being able to work out how to handle anything different, or can't be bothered, and instead deciding to force you to bend to their will. It is just easier for them to change you than it is to change themselves. I guess this is all tied up with systemd in some way. It's the same dictatorial style. Hey ho.

Personally I use /home/user for a LOCAL non networked user, and /home/server/whatever/user for domain logons. etc. so I know which is which at a glance.

I am not about to go round migrating all my desktops now. I'd rather dump *buntu.

If is isn't "allowed" then they should have done something about it. But they haven't bothered. Read into that what you will. Won't fix says it all.

Quite frankly I'd recommend that you do yourself a favour and dump the overblown inflexible bloatware nonsense that are snaps.

[Reinhard (reinhard-fink)]

Breaking Linux Network?

If we end up in this bug report we normally deal with Linux networks, where home directory are mounted from servers. For good reason we create structures in /home/... to manage more users.

In education sector we have to compete with sponsored MS - networks and we can only survive, when our performance is good.
So restricted snaps like chromium and also apparmor profiles like for evince break programs for many users in our networks.

Please, let additional binds or similar things be a workaround. Linux networks are great because we can build good structures straight forward.

So, please think about us Linux network admins. May be it is our network, where users have their first, hopefully positive, Linux contact.

Thanks!

[Swapnil Shende (shendesc)]

OK!
So this I'm facing the same issue in Ubuntu 20.04.
I've upgraded from 18.04.
Snap was working fine previously but post update it started giving the same permission denied error.
I tried various fixes given in the whole thread but nothing seems to be working.
I also tried to reinstall snap, snapdm gnome-software packages using apt but after that as well its giving the same error.
This is so sad, Ubuntu was about have a great app store but in a normal LTS upgrade it doesn't work properly. :(
If anyone can suggest me some help, it'd be appriciated.

[Thiago Martins (martinx)]

More Snap craziness hitting me as well!

I'm considering for the first time since 2006, to get back to the Debian mothership.

The only reason I'm actually still with Ubuntu is that I'm heavy on the LXD thing... And it's a freaking Snap package.

I hope that https://wiki.debian.org/LXD will get there soon! So I can kiss Ubuntu a good by because Snap suck and Canonical is forcing this thing s*** down the throat.

Sad times for Ubuntu...

[Brian Knoll (brianknoll)]

I would encourage the folks working on this code to fix this, as it does affect a large number of users. I am encouraged by a comment I read that there is at least some interest in fixing it, although apparently there is a fair amount of work involved.

[Tapani Tarvainen (ubuntu-tapani)]

Snap causes problems even when your home directory is real, as they prevent opening anything symlinked or behind an NFS mount. This makes, for example, Chromium browser effectively unusable for me.

For Chromium there's the obvious workaround of installing non-snap version from Debian repository (which I have done), but if I have to start doing that for more and more packages, switching to Debian completely quickly becomes the path of least resistance.

Snap isn't really ready for prime time and its use should be limited to non-critical, experimental packages only.

[Markus Kuhn (markus-kuhn)]

I have so far looked at the reasons for why snapd is so hostile to symbolic links only in one particular context, but what I found in the golang source code left me rather worried, and with the impression that the author of that code has a rather pre-POSIX, 1980s understanding of Unix file-access-control security. The problems with symbolic links may be just one of many problem and potential security vulnerabilities caused by snapd not changing its "effective" UID/GID to that of the user when it accesses resources on behalf of the user!

I've so far only looked at the relevant code for when snapd makes a copy of the user's $XAUTHORITY file, in order to provide a snap app access to the X11 access cookie, which fails for no good reason if there is a symlink in the paths. The details are in my comments in bug #1902250, but in a nutshell: the developers had probably heard of historic time-of-check-to-time-of-use attacks on privileged processes (here: snapd) being tricked into accessing the wrong file via flipping symbolic links, and rather than implementing the real solution (namely switching the EUID to that of the user before accessing the resource, such that the kernel performs all applicable security checks properly) they try do a long list of known-to-not-work 1980s workarounds, from the time before POSIX introduced effective/real/saved UIDs/GIDs, involving manually checking file permissions and whether there are any symlinks in the path. This looks very much like a developer training issue, I'm afraid, i.e. not having understood the purpose of the seteuid()/setguid() system calls, perhaps combined with golang not documenting its corresponding wrapper functions very well. The relevant XAUTHORITY access code could be rewritten much simpler and more secure that way, eliminating not just the symlink problem. I now suspect the same may also apply for other parts of snapd.

I do strongly encourage in-depth code security reviews of snapd. There may be lots of low-hanging fruits there!

[Zygmunt Krynicki (zyga)]

Disclaimer: I’m no longer affiliated with snapd.

To the author of the last comment I have this advice: please keep on reading. Your understanding of the architectural reasons behind the /home limitation is widely incorrect. Your claims about security are equally off point.

I will give you the following hints:
- mount namespace configuration used by snapd to run snaps impacts /home
- semantics of symbolic links for apparmor
- apparmor profile used by snap-confine and particular snaps
- effective user transitions across snap run -> confine -> exec chain

Please look into those topic before making bold claims.
Good evening!

[Markus Kuhn (markus-kuhn)]

Zygmunt, the code I was specifically referring to in comment #49 is migrateXauthority() in https://github.com/snapcore/snapd/blob/master/cmd/snap/cmd_run.go (see bug #1902250). If I understood correctly, this merely reads and copies (on behalf of the calling user) the content of a credential file before the app is invoked, so this particular function shouldn't be constrained by "the semantics of symbolic links for apparmor" or "mount namespace configuration" considerations, right? But it still checks for symlinks in the path to the Xauthority file. Why? (preferably answer under bug #1902250)

I still think it was a fair comment that this particular function should simply have switched the EUID/EGID of the process temporarily to that of the calling user for the duration of the operation of reading the Xauthority file, and then let the kernel do the DACL checks, rather than trying to do access-control checks manually and worry about symlink trickery from user space.

From that one data point, forgive me for wondering aloud whether all other places in snapd where symbolic links are also rejected (which I haven't read yet!) really all do so for much better reasons than migrateXauthority() ...

If you have a detailed write up or presentation of all the architectural considerations that went into the design of snapd, especially the parts that conflict with symlinks and NFS mounting, I'd love to read that and would much appreciate a pointer.

[Ian Johnson (anonymouse67)]

Hi Markus, please note that the bit of code you are referencing in cmd_run.go, is run by the calling user, i.e. when I do

```
snap run hello-world
```

the bit of code in migrateXauthority is run by my user (id 1000), and only after that function completes does privileged code execute when we exec() snap-confine, which is setuid root, and so the things inside snap-confine are running with elevated privilege, and as Zygmunt pointed out, any file manipulation done inside snap-confine should be properly handled with dropping privileges to user-owned files before manipulating them, etc. but in general I think that we try to do any user-specific things like this XAUTHORITY code you find as the user directly before we execute snap-confine.

We take security very seriously and ensure that the Canonical/Ubuntu security team reviews as many relevant PR's as possible in snapd (including all such PR's which modify snap-confine). In the future, if you are concerned about security vulnerabilities inside snapd, please raise your concerns in a responsible manner by filing either a private security bug for the snapd project, or by emailing security @ ubuntu.com.

Thanks

[C. Alex. North-Keys (erlkonig-talisman)]

After reading through this and the related page with more info about @{HOMEDIRS} I have to point out what should by this time just be a "metoo":

  You absolutely cannot dictate where home directories "should" be or whether symlinks, weird mounts, fuser, or anything else bizarre is involved and still have snaps be viable in complex sites where homes literally can't be all stuffed /home and still have the site work properly.

Nor can you expect there to be a single directory containing all homes. Ever. "/root" e.g. and that's only the start.

I *think* it's safe to expect that every user, that has a $HOME directory, has a unique home directory. There might be some crazy site that has multiple users sharing a home using group access rights, but I'd expect they're already having problems so they don't really expect everything to be perfect.

The suggestion I read where you might locate the home directory and make it appear as /home/$USER inside the snap is... interesting, but may lead software running in the snap to write this possibly imaginary homepath (if the user's home in /homes/cs/staff/jws for example) into files that software running outside of the snap won't be able to use.

Additionally, don't be tempted to figure out whether the home "really" is instead of using the literal home directory from $HOME, since there are plenty of scenarios - *especially* certain variants of automount - where trying to be smart will miss a trigger that mounts the filesystem the home is in (or that keeps it mounted). In other words, you can't even trust "pwd" to give you a reusable, supposedly-more-real $HOME.

(Detail: Automounted dirs used to appear under /tmp_mnt/, but if you then used that output from "pwd" instead of the actual $HOME, usage wouldn't be noticed by automount, the user's home would get dismounted, and references via /tmp_mnt/ wouldn't trigger remounting it!)

So you need to make the literal, exact value of $HOME work *inside* of a snap in order for snaps to work, and keep Ubuntu viable, in many organizations.

Lastly, just using "dpkg-reconfigure apparmor" showed the right input, but it didn't fix hello-world, but I haven't actually rebooted yet. Sysadmins hate rebooting, so it's essential to have instructions that clearly show how to have the changes take affect without a reboot.

[Neil (danteuk-2)]

Me too.
I've tried the "dpkg-reconfigure apparmor" and removed various cache files and rebooted ( multiple times now )
Still I just get the stupid permission error ( on a folder in my own home directory that I can write to without any problems and was obviously created by snap in the first place )

If 'snap' can't support home directories that are not in /home then it's not fit for purpose.

I also can't use a bind mount since other users exist in /home.

[Wolfgang Fahl (wf-bitplan)]

The won't fix decision IMHO affects a few thousand people already. Some believe the snap doesn't work a all see:

https://askubuntu.com/questions/1240898/snap-broken-in-ubuntu-20-04

it would be great if this issue would get much higher priority since it is a showstopper for so many potential users. I have never been able to use snap in the past just because of this. I retried to day following the instructions at https://sqlitebrowser.org/dl/ and was astonished and disappointed that the problem is not fixed.

For today my workaround is:

sudo add-apt-repository -y ppa:linuxgndu/sqlitebrowser
sudo apt-get install sqlitebrowser

so basically not using snap. So for me snap is so far only a waste of time.

[Michael Vogt (mvo)]

I moved this out of "Won't fix" - I think that this state is not quite accurate. We want to fix this but unfortunately it's a bit of work and we have no clear schedule yet when we can do it.

[reetp (jcrisp)]

> We want to fix this but unfortunately it's a bit of work

:thumbsup:

We can't use snaps until it is fixed, and we are not using links - just /home/server/files/user instead of /home/user

That should not break it or prevent install.

[Alberto Mardegan (mardy)]

Hi reetp, I'm specifically addressing you because your use-case is exactly the case that I'm attempting to support in the "phase 1" of removing this annoying limitation: that is, when the home directories are located somewhere deeper under the /home directory (we are also working on fixing the generic case where the home directory is located elsewhere, but that looks like a bigger effort which will take more time).

I made a small change to snapd, to have it create the apparmor snippet mentioned in comment #6, and that seems to work for me: I do have a user whose home is

    /home/folks/<user>/

and snapd created a file /etc/apparmor.d/tunables/home.d/snapd with the line

    @{HOMEDIRS}+="/home/folks"

After a reboot (this should not be needed in the final version), I could install snaps and run them without issues. If a similar apparmor snippet does not work for you, can you please let me know what snaps are failing, and what errors messages you get?

[reetp (jcrisp)]

> Hi reetp, I'm specifically addressing you because your use-case is exactly the case that I'm attempting to support in the "phase 1" of removing this annoying limitation

Hi, and thanks for looking!

Let me try it. I need to reinstall snapd.....done.

Xenial, all packages updated.

ii snapd 2.54.3+18.04.2ubuntu0.2 amd64 Daemon and tooling that enable snap packages

Note I already had this which was a fail.

cat /etc/apparmor.d/tunables/home.d/ubuntu

(This is the file auto created with dpkg-reconfigre apparmor)

# This file is auto-generated. It is recommended you update it using:
# $ sudo dpkg-reconfigure apparmor
#
# The following is a space-separated list of where additional user home
# directories are stored, each must have a trailing '/'. Directories added
# here are appended to @{HOMEDIRS}. See tunables/home for details.
@{HOMEDIRS}+=/home/e-smith/files/users/

-------------------------------------------------------

Just to precis #6 and #7
https://bugs.launchpad.net/snappy/+bug/1620771/comments/6

Feel free to do one of:
$ sudo dpkg-reconfigure apparmor

Or drop a file into /etc/apparmor.d/tunables/home.d that has:
@{HOMEDIRS}+=/some/nonstandard/home/users/

I forgot to mention you'll need to reload the policy after doing this. So after adjust the HOMEDIRS apparmor variable as stated in comment #6, I suggest:

$ sudo rm -f /etc/apparmor.d/cache/* /var/cache/apparmor/snap.*
$ sudo reboot

(there are other ways to do this without a reboot, but this should work regardless of what apparmor policy you have already generated)

-------------------------------------------------------

So I set the apparmor as above. Cleared the caches and rebooted.

john@xubuntu:~$ sudo snap install hello-world
hello-world 6.4 from Canonical✓ installed

john@xubuntu:~$ hello-world
cannot create user data directory: /home/e-smith/files/users/john/snap/hello-world/29: Permission denied

john@xubuntu:~$ sudo hello-world
Hello World!

john@xubuntu:~$ hello-world
cannot create user data directory: /home/e-smith/files/users/john/snap/hello-world/29: Permission denied

And then installed snap-store as well but can't run it as a user:

john@xubuntu:~$ snap-store
cannot create user data directory: /home/e-smith/files/users/john/snap/snap-store/558: Permission denied

john@xubuntu-8300-1:~$ ll /home/e-smith/files/users/john/snap/
total 16
drwx------ 4 john john 4096 Apr 20 17:14 ./
drwx------ 108 john john 4096 Apr 20 17:06 ../
drwxr-xr-x 4 john john 4096 Apr 20 17:00 hello-world/
drwxr-xr-x 4 john john 4096 Apr 20 17:14 snap-store/
john@xubuntu-8300-1:~$ ll /home/e-smith/files/users/john/snap/snap-store
total 16
drwxr-xr-x 4 john john 4096 Apr 20 17:14 ./
drwx------ 4 john john 4096 Apr 20 17:14 ../
drwxr-xr-x 2 john john 4096 Apr 20 17:14 558/
drwxr-xr-x 2 john john 4096 Apr 20 17:14 common/
lrwxrwxrwx 1 john john 3 Apr 20 17:14 current -> 558/

journalctl -xe

-- Subject: Unit UNIT has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit UNIT has failed.
--
-- The result is RESULT.
Apr 20 18:21:12 xubuntu systemd[3772]: snap.hello-world.hello-world.4de7faab-fc60-4c57-b492-ce3416d6b6cd.scope: Failed to add PID...

[suside (suside)]

It seems that symlink inside /home/<user>/ is enough to trigger this, see https://github.com/bitwarden/desktop/issues/342#issuecomment-660539402

[Stefan Fleischmann (sfleischmann)]

Alberto, I was just wondering about the snippet you posted

```
  @{HOMEDIRS}+="/home/folks"
```

shouldn't that have a trailing slash? I never bothered to check why but the dpkg-reconfigure dialog for apparmor always tells me: "Please enter a space separated list of any additional locations for user home directories. These locations are in addition to those specified in /etc/apparmor.d/tunables/home and must end with a '/'"

[Sean P. Brennan (zettix)]

The gift that keeps on giving. Firefox is now broken in Ubuntu 22.04 LTS.

Home directory is a symlink:
% firefox
cannot open path of the current working directory: Permission denied
============================================================================================
Home directory is on NFS with no symlink:
% firefox
update.go:85: cannot change mount namespace according to change mount (/var/lib/snapd/hostfs/usr/share/libreoffice/help /usr/share/libreoffice/help none bind,ro 0 0): cannot create directory "/usr/share/libreoffice/help": permission denied
update.go:85: cannot change mount namespace according to change mount (/var/lib/snapd/hostfs/usr/share/xubuntu-docs /usr/share/xubuntu-docs none bind,ro 0 0): cannot open directory "/var/lib": permission denied
cannot create user data directory: /mnt/helios/home/sean/snap/firefox/1551: Permission denied
==============================================================================================

google-chrome still works.

[Tom Chiverton (bugs-launchpad-net-falkensweb)]

I had to re-organise a half dozen machines /home's to make them work with Snap Firefox, because of a 7 year old bug.

How was this not considered a blocker for moving a major Ubuntu component to Snap ?

Echo "If 'snap' can't support home directories that are not in /home then it's not fit for purpose." from above.

[reetp (jcrisp)]

> How was this not considered a blocker for moving a major Ubuntu component to Snap ?

You may well ask.

> Echo "If 'snap' can't support home directories that are not in /home then it's not fit for purpose." from above.

Nope. Disabled on every machine with me. Snaps are an overloaded overcomplicated bloatware mess, so they weren't really welcome but this was a complete blocker.

What's even more galling is being asked to write a report which I did - at a cost of MY time - in #59 and then....crickets.

If devs want help then act on the response, don't ignore it.

Duplicate:

https://bugs.launchpad.net/snapd/+bug/1776800

Even worse is it seems they are now force feeding snaps on us with 22.04

https://bugs.launchpad.net/snapd/+bug/1776800/comments/7

https://www.omgubuntu.co.uk/2022/04/how-to-install-firefox-deb-apt-ubuntu-22-04

Dear oh dear what a mess. Ubuntu please stop foisting this mess on us when you haven't even got the basics right.

https://bugs.launchpad.net/snapd/+bug/1776800/comments/14

It not our fault that you are having problems solving it. You should have got it fixed. You've only had what - 6 years so far?

#angry

[Wolfgang Fahl (wf-bitplan)]

After upgrading to Ubuntu 22.04 LTS firefox is a snap installed component now which
finally forced me to use the bind mount workround

i had to add entries to /etc/fstab such as:

```bash
# bind mount home/user to allow snap to work
/mnt/disk/home/user /home/user none bind
```

for bind mounting specific users

or
```
# bind mount home to allow snap to work
/mnt/disk/home /home none bind
```

it's interesting that there has not been such an example fstab configuration yet in this discussion yet and participants seem to assume that people know how to do bind mounts

Questions such as https://askubuntu.com/questions/1207472/how-do-you-actually-run-a-snap-package-when-your-home-directory-isnt-under-h

show that this is still a severe bug and should be fixed in a way that makes it possible for people to configure their snap based systems without first attending a linux system administrators course.

[Walter (wdoekes)]

> [...] in a way that makes it possible for people to configure
> their snap based systems without first attending a linux
> system administrators course.

Well. The problem is that you changed the home directory location. This means you're now among the linux system administrators. And you are therefore disqualified from using a snap-based browser.

I have learned that using a snap-based browser _only_ works if you have done _nothing_ to customize your system. In fact, if you are sysadmin enough to use _sudo_ and have a file owned by root in your homedir, snap will punish you: https://bugs.launchpad.net/ubuntu/+source/chromium-browser/+bug/1987945 -- and that is just one of many examples.

At the time of writing, a snap-based browser might be fine, but not for people who have the root password. Unfortunately.

[reetp (jcrisp)]

Oh great.

So 18.04 Bionic is going EOL shortly.

So forced to upgrade to Focal.

I had snaps disabled. Because they a) don't work on our desktops b) I am not a stupid user who needs to be treated like an idiot and c) are awful bloatware

I had no idea that when I upgraded to Focal I would be forced to use snaps - I thought it was from Jellyfish, but it seems it is insisting on doing it for Chromium - and cups as well. Joy. So can't print either, which has now hung my desktop as I don't use a standard /home.

Ah - and cups as well. Joy. So can't print either.

Oh thank you Canonical. And Ubuntu devs. 7 years and still not fixed.

Last straw. We can't put up with this amateur nonsense.

#fuming

[Ool (0ol)]

We have /home for local user
and /home-nonstandard for LDAP user with HOME in NFS (with squash)

with 20.04 not a big deal

with 22.04, I need to remove the snap (FF) , find a ppa and install it with the standard method (apt install…)
I hope to find ppa for everything … but not sure.

For newer version of ubuntu with all snap… I 'm a little bit affraid of the need to switch from my favorite distrib. Perhaps a fix will be find.

dapper and hardy begin to miss me.

[Walter (wdoekes)]

For Firefox there's a ppa [1].
For Chromium, there's the Linuxmint repo [2].
I switched to using Linuxmint repo for both and purged snapd. 0 regrets.

If Canonical decides to put more into snap then this workaround may not be sufficient. We'll have to see what happens in 24.04...

[1] https://www.omgubuntu.co.uk/2022/04/how-to-install-firefox-deb-apt-ubuntu-22-04
[2] https://www.osso.nl/blog/chromium-browser-without-ubuntu-snap-linux-mint/ -- I suggest finding/selecting a local mirror though, the main one can be slow

[Lorenzo (nakis)]

Found solution in comment from https://bugs.launchpad.net/snapd/+bug/1776800

    # to update to version supporting homedirs system config
    snap refresh --channel=latest/edge snapd

    # I don't have to do anything else for my AD domain homes, but just in case:
    snap set system homedirs=/media/Dane/.home/

[Sergey Zolotarev (szx)]

Is there a way to make snap work when its folder is symlinked to an eCryptFS ~/Private dir? I assume HOMEDIRS would not help in this case.

[Seth Arnold (seth-arnold)]

Sergey, it's probably better to ask in https://forum.snapcraft.io/ -- that will find a much larger audience than this bug report.

It's been ages since I've seen eCryptfs but my recollection is that all AppArmor policies for processes using eCryptfs data required *very* broad permissions, due to the encrypted names.

You might have some success with replacing your symlink with a bindmount instead, but I'm not optimistic that will be enough.

Thanks