Command of classic snap fails with denial when output is redirected

[Simon Fels]

I have a jenkins setup where a snap (jenkins-job-builder, custom build, not in the store) is used to deploy new jobs to jenkins. Until snapd 2.36.2 the snap was running fine and could be executed like

# Simplified version of what Jenkins is executing
$ jenkins-job-builder ... &> output.log

Now with snapd 2.37.2 (revision ) the execution always fails with the following denials:

[80831.069069] audit: type=1400 audit(1550131802.780:1697): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-android-x86-slave-2_<var-lib-lxd>" profile="/snap/core/6405/usr/lib/snapd/snap-confine" name="/var/lib/jenkins/workspace/indore-jenkins-jobs-ci/output.log" pid=71871 comm="snap-confine" requested_mask="w" denied_mask="w" fsuid=112 ouid=112
[80831.069073] audit: type=1400 audit(1550131802.780:1698): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-android-x86-slave-2_<var-lib-lxd>" profile="/snap/core/6405/usr/lib/snapd/snap-confine" name="/var/lib/jenkins/workspace/indore-jenkins-jobs-ci/output.log" pid=71871 comm="snap-confine" requested_mask="w" denied_mask="w" fsuid=112 ouid=112

Reverting back to snapd 2.36.2 fixes the problem and everything works again.

This is on Ubuntu 16.04 running inside a LXD container. The host is running Ubuntu 16.04 too. snapd version outside of the LXD container is at 2.37.2. Jenkins on the master is installed via debs and is running version 2.150.2

[Zygmunt Krynicki (zyga)]

Is the container that this is running in privileged or unprivileged?

[Simon Fels (morphis)]

It is running with:

security.nesting: "true"
security.privileged: "true"

[Zygmunt Krynicki (zyga)]

I managed to reproduce this as follows:

lxc launch ubuntu:x xenial -c security.nesting=true -c security.privileged=true
lxc exec xenial su - ubuntu

Inside the container shell:

sudo apt-get update
sudo apt-get dist-upgrade -y
sudo snap install core
sudo snap install --classic python0
python0 &> foo # then kill the process

The denials was:

lut 14 09:50:40 crusty kernel: audit: type=1400 audit(1550134240.748:160): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-xenial_<var-snap-lxd-common-lxd>" profile="/snap/core/6405/usr/lib/snapd/snap-confine" name="/home/ubuntu/foo" pid=15267 comm="snap-confine" requested_mask="w" denied_mask="w" fsuid=1000 ouid=1000

Installing the core snap raised first red flags:

Setup snap "core" (6350) security profiles /error: cannot perform the following tasks:
- Setup snap "core" (6350) security profiles (cannot setup udev for snap "core": cannot reload udev rules: exit status 2
udev output:
)
- Setup snap "core" (6350) security profiles (cannot reload udev rules: exit status 2
udev output:
)

I proceeded by installing a strictly confined snap:

ubuntu@xenial:~$ sudo snap install snapd-hacker-toolbelt
2019-02-14T08:49:30Z ERROR cannot setup udev for snap "snapd-hacker-toolbelt": cannot reload udev rules: exit status 2
udev output:

error: cannot perform the following tasks:
- Setup snap "snapd-hacker-toolbelt" (19) security profiles (cannot setup udev for snap "snapd-hacker-toolbelt": cannot reload udev rules: exit status 2
udev output:
)
- Setup snap "snapd-hacker-toolbelt" (19) security profiles (cannot reload udev rules: exit status 2
udev output:
)
- Connect snapd-hacker-toolbelt:opengl to core:opengl (cannot setup udev for snap "snapd-hacker-toolbelt": cannot reload udev rules: exit status 2
udev output:
)
- Connect snapd-hacker-toolbelt:opengl to core:opengl (cannot reload udev rules: exit status 2
udev output:
)

But I was unable to do so.

I will investigate what is the specific meaning of security.nesting: true and security.privileged: true.

[Zygmunt Krynicki (zyga)]

With some more investigation I would summarize that using privileged containers is unsupported in snapd.

ubuntu@xenial:~$ sudo snap install core --beta
error: cannot perform the following tasks:
- Setup snap "core" (6405) security profiles (cannot setup udev for snap "core": cannot reload udev rules: exit status 2
udev output:
)
- Setup snap "core" (6405) security profiles (cannot reload udev rules: exit status 2
udev output:
)

I will resort to fixing the file inheritance issue for now.

[Zygmunt Krynicki (zyga)]

I have a working fix, I will write a regression test now.

[Zygmunt Krynicki (zyga)]

This should be fixed with https://github.com/snapcore/snapd/pull/6507