Apparmor complains in strict confinement with base: core18

Bug #1802332 reported by Konstantinos Tsakalozos
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
snapd
Fix Released
Undecided
Zygmunt Krynicki

Bug Description

Hi,

This came up while trying to move microk8s to strict confinement. The source of the snap is on this branch: https://github.com/ubuntu/microk8s/tree/feature/strict

The snapcraft and snap versions are as follows:
snapcraft, version 3.0+git13.g04f18f5
snap 2.35.5
snapd 2.35.5
series 16
ubuntu 18.04
kernel 4.15.0-34-generic

The snap builds with snapcraft however after installation aparmor logs the following when calling a "microk8s.kubect get all":

```
Nov 08 17:10:20 jackal-VGN-FZ11M audit[21938]: AVC apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="snap-update-ns.microk8s" name="/tmp/.snap/snap/microk8s/x1/" pid=21938 comm="3" srcname="/snap/microk8s/x1/" flags="rw, rbind"
Nov 08 17:10:20 jackal-VGN-FZ11M kernel: audit: type=1400 audit(1541689820.753:1595): apparmor="DENIED" operation="mount" info="failed mntpnt match" error=-13 profile="snap-update-ns.microk8s" name="/tmp/.snap/snap/microk8s/x1/" pid=21938 comm="3" srcname="/snap/microk8s/x1/" flags="rw, rbind"
```

The `/var/lib/snapd/mount/snap.microk8s.fstab` is as follows:

```
none /var/lib/cni none x-snapd.kind=symlink,x-snapd.symlink=/var/snap/microk8s/common/var/lib/cni,x-snapd.origin=layout 0 0
/snap/microk8s/x1/var/log/containers /var/log/containers none rbind,rw,x-snapd.origin=layout 0 0
/snap/microk8s/x1/var/log/pods /var/log/pods none rbind,rw,x-snapd.origin=layout 0 0
```

The `/var/lib/snapd/apparmor/profiles/snap-update-ns.microk8s`

```
#include <tunables/global>

profile snap-update-ns.microk8s (attach_disconnected) {
  # The next four rules mirror those above. We want to be able to read [126/155]
  # and map snap-update-ns into memory but it may come from a variety of places.
  /usr/lib{,exec,64}/snapd/snap-update-ns mr,
  /var/lib/snapd/hostfs/usr/lib{,exec,64}/snapd/snap-update-ns mr,
  /{,var/lib/snapd/}snap/{core,snapd}/*/usr/lib/snapd/snap-update-ns mr,
  /var/lib/snapd/hostfs/{,var/lib/snapd/}snap/core/*/usr/lib/snapd/snap-update-ns mr,

  # Allow reading the dynamic linker cache.
  /etc/ld.so.cache r,
  # Allow reading, mapping and executing the dynamic linker.
  /{,usr/}lib{,32,64,x32}/{,@{multiarch}/}ld-*.so mrix,
  # Allow reading and mapping various parts of the standard library and
  # dynamically loaded nss modules and what not.
  /{,usr/}lib{,32,64,x32}/{,@{multiarch}/}libc{,-[0-9]*}.so* mr,
  /{,usr/}lib{,32,64,x32}/{,@{multiarch}/}libpthread{,-[0-9]*}.so* mr,

  # Allow reading the command line (snap-update-ns uses it in pre-Go bootstrap code).
  @{PROC}/@{pid}/cmdline r,

  # Allow reading file descriptor paths
  @{PROC}/@{pid}/fd/* r,

  # Allow reading the os-release file (possibly a symlink to /usr/lib).
  /{etc/,usr/lib/}os-release r,

  # Allow creating/grabbing global and per-snap lock files.
  /run/snapd/lock/microk8s.lock rwk,
  /run/snapd/lock/.lock rwk,

  # Allow reading stored mount namespaces,
  /run/snapd/ns/ r,
  /run/snapd/ns/microk8s.mnt r,

  # Allow reading per-snap desired mount profiles. Those are written by
  # snapd and represent the desired layout and content connections.
  /var/lib/snapd/mount/snap.microk8s.fstab r,
  /var/lib/snapd/mount/snap.microk8s.user-fstab r,

  # Allow reading and writing actual per-snap mount profiles. Note that
  # the wildcard in the rule to allow an atomic write + rename strategy.
  # Those files are written by snap-update-ns and represent the actual
  # mount profile at a given moment.
  /run/snapd/ns/snap.microk8s.fstab{,.*} rw,

  # NOTE: at this stage the /snap directory is stable as we have called
  # pivot_root already.

  # Needed to perform mount/unmounts.
  capability sys_admin,
  # Needed for mimic construction.
  capability chown,
  # Needed for dropping to calling user when processing per-user mounts
  capability setuid,
  capability setgid,
  # Allow snap-update-ns to override file ownership and permission checks.
  # This is required because writable mimics now preserve the permissions
  # of the original and hence we may be asked to create a directory when the
  # parent is a tmpfs without DAC write access.
  capability dac_override,

  # Allow freezing and thawing the per-snap cgroup freezers
  /sys/fs/cgroup/freezer/snap.microk8s/freezer.state rw,

  # Allow the content interface to bind fonts from the host filesystem
  mount options=(ro bind) /var/lib/snapd/hostfs/usr/share/fonts/ -> /snap/microk8s/*/**,
  umount /snap/microk8s/*/**,

  # set up user mount namespace
  mount options=(rslave) -> /,

  # Allow traversing from the root directory and several well-known places.
  # Specific directory permissions are added by snippets below.
  / r,
  /etc/ r,
  /snap/ r,
  /tmp/ r,
  /usr/ r,
  /var/ r,
  /var/snap/ r,

  # Allow reading timezone data.
  /usr/share/zoneinfo/** r,

  # Don't allow anyone to touch /snap/bin
  audit deny mount /snap/bin/** -> /**,
  audit deny mount /** -> /snap/bin/**,

  # Don't allow bind mounts to /media which has special
  # sharing and propagates mount events outside of the snap namespace.
  audit deny mount -> /media,

  # Layout /var/lib/cni: symlink $SNAP_COMMON/var/lib/cni
  /var/lib/cni rw,
  # Writable mimic /var/lib
  mount options=(rbind, rw) /var/lib/ -> /tmp/.snap/var/lib/,
  mount fstype=tmpfs options=(rw) tmpfs -> /var/lib/,
  mount options=(rbind, rw) /tmp/.snap/var/lib/** -> /var/lib/**,
  mount options=(bind, rw) /tmp/.snap/var/lib/* -> /var/lib/*,
  umount /tmp/.snap/var/lib/,
  umount /var/lib{,/**},
  /var/lib/** rw,
  /var/lib/ rw,
  /tmp/.snap/var/lib/** rw,
  /tmp/.snap/var/lib/ rw,
  /tmp/.snap/var/ rw,
  /tmp/.snap/ rw,

  # Layout /var/log/containers: bind $SNAP/var/log/containers
  mount options=(rbind, rw) /snap/microk8s/x1/var/log/containers/ -> /var/log/containers/,
  umount /var/log/containers/,
  # Writable mimic /var/log
  mount options=(rbind, rw) /var/log/ -> /tmp/.snap/var/log/,
  mount fstype=tmpfs options=(rw) tmpfs -> /var/log/,
  mount options=(rbind, rw) /tmp/.snap/var/log/** -> /var/log/**,
  mount options=(bind, rw) /tmp/.snap/var/log/* -> /var/log/*,
  umount /tmp/.snap/var/log/,
  umount /var/log{,/**},
  /var/log/** rw,
  /var/log/ rw,
  /tmp/.snap/var/log/** rw,
  /tmp/.snap/var/log/ rw,
  /tmp/.snap/var/ rw,
  /tmp/.snap/ rw,
  # Writable mimic /snap/microk8s/x1/var/log
  mount options=(rbind, rw) /snap/microk8s/x1/var/log/ -> /tmp/.snap/snap/microk8s/x1/var/log/,
  mount fstype=tmpfs options=(rw) tmpfs -> /snap/microk8s/x1/var/log/,
  mount options=(rbind, rw) /tmp/.snap/snap/microk8s/x1/var/log/** -> /snap/microk8s/x1/var/log/**,
  mount options=(bind, rw) /tmp/.snap/snap/microk8s/x1/var/log/* -> /snap/microk8s/x1/var/log/*,
  umount /tmp/.snap/snap/microk8s/x1/var/log/,
  umount /snap/microk8s/x1/var/log{,/**},
  /snap/microk8s/x1/var/log/** rw,
  /snap/microk8s/x1/var/log/ rw,
  /snap/microk8s/x1/var/ rw,
  /snap/microk8s/x1/ rw,
  /snap/microk8s/ rw,
  /tmp/.snap/snap/microk8s/x1/var/log/** rw,
  /tmp/.snap/snap/microk8s/x1/var/log/ rw,
  /tmp/.snap/snap/microk8s/x1/var/ rw,
  /tmp/.snap/snap/microk8s/x1/ rw,
  /tmp/.snap/snap/microk8s/ rw,
  /tmp/.snap/snap/ rw,
  /tmp/.snap/ rw,

  # Layout /var/log/pods: bind $SNAP/var/log/pods
  mount options=(rbind, rw) /snap/microk8s/x1/var/log/pods/ -> /var/log/pods/,
  umount /var/log/pods/,
  # Writable mimic /var/log
  mount options=(rbind, rw) /var/log/ -> /tmp/.snap/var/log/,
  mount fstype=tmpfs options=(rw) tmpfs -> /var/log/,
  mount options=(rbind, rw) /tmp/.snap/var/log/** -> /var/log/**,
  mount options=(bind, rw) /tmp/.snap/var/log/* -> /var/log/*,
  umount /tmp/.snap/var/log/,
  umount /var/log{,/**},
  /var/log/** rw,
  /var/log/ rw,
  /tmp/.snap/var/log/** rw,
  /tmp/.snap/var/log/ rw,
  /tmp/.snap/var/ rw,
  /tmp/.snap/ rw,
  # Writable mimic /snap/microk8s/x1/var/log
  mount options=(rbind, rw) /snap/microk8s/x1/var/log/ -> /tmp/.snap/snap/microk8s/x1/var/log/,
  mount fstype=tmpfs options=(rw) tmpfs -> /snap/microk8s/x1/var/log/,
  mount options=(rbind, rw) /tmp/.snap/snap/microk8s/x1/var/log/** -> /snap/microk8s/x1/var/log/**,
  mount options=(bind, rw) /tmp/.snap/snap/microk8s/x1/var/log/* -> /snap/microk8s/x1/var/log/*,
  umount /tmp/.snap/snap/microk8s/x1/var/log/,
  umount /snap/microk8s/x1/var/log{,/**},
  /snap/microk8s/x1/var/log/** rw,
  /snap/microk8s/x1/var/log/ rw,
  /snap/microk8s/x1/var/ rw,
  /snap/microk8s/x1/ rw,
  /snap/microk8s/ rw,
  /tmp/.snap/snap/microk8s/x1/var/log/** rw,
  /tmp/.snap/snap/microk8s/x1/var/log/ rw,
  /tmp/.snap/snap/microk8s/x1/var/ rw,
  /tmp/.snap/snap/microk8s/x1/ rw,
  /tmp/.snap/snap/microk8s/ rw,
  /tmp/.snap/snap/ rw,
  /tmp/.snap/ rw,

}
```

Thank you for you help,
Konstantinos

description: updated
Revision history for this message
Zygmunt Krynicki (zyga) wrote :

Hello.

Can you please refresh core to the candidate channel to see if the 2.36.1 release fixes it?

You can do that with:

"snap refresh core --candidate"

Please report back

Changed in snapd:
status: New → Incomplete
Zygmunt Krynicki (zyga)
Changed in snapd:
assignee: nobody → Zygmunt Krynicki (zyga)
Revision history for this message
Zygmunt Krynicki (zyga) wrote :
Download full text (39.2 KiB)

I've built microk8s from the feature/scrict branch, upon installing the snap and launching microk8s.kubectl I get only those denials:

[ 4349.918214] audit: type=1400 audit(1542369829.324:196): apparmor="DENIED" operation="open" profile="snap.microk8s.daemon-proxy" name="/sys/kernel/mm/hugepages/" pid=11145 comm="kube-proxy" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 4349.921932] audit: type=1400 audit(1542369829.328:197): apparmor="DENIED" operation="exec" profile="snap.microk8s.daemon-proxy" name="/bin/kmod" pid=11222 comm="kube-proxy" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[ 4349.922386] audit: type=1400 audit(1542369829.328:198): apparmor="DENIED" operation="exec" profile="snap.microk8s.daemon-proxy" name="/bin/kmod" pid=11223 comm="kube-proxy" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[ 4349.922744] audit: type=1400 audit(1542369829.328:199): apparmor="DENIED" operation="exec" profile="snap.microk8s.daemon-proxy" name="/bin/kmod" pid=11224 comm="kube-proxy" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[ 4349.923147] audit: type=1400 audit(1542369829.328:200): apparmor="DENIED" operation="exec" profile="snap.microk8s.daemon-proxy" name="/bin/kmod" pid=11225 comm="kube-proxy" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[ 4349.923556] audit: type=1400 audit(1542369829.328:201): apparmor="DENIED" operation="exec" profile="snap.microk8s.daemon-proxy" name="/bin/kmod" pid=11226 comm="kube-proxy" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[ 4349.924542] audit: type=1400 audit(1542369829.328:202): apparmor="DENIED" operation="open" profile="snap.microk8s.daemon-proxy" name="/proc/modules" pid=11227 comm="cut" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 4350.244108] audit: type=1400 audit(1542369829.648:203): apparmor="DENIED" operation="open" profile="snap.microk8s.daemon-proxy" name="/sys/kernel/mm/hugepages/" pid=11284 comm="kube-proxy" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 4350.247745] audit: type=1400 audit(1542369829.652:204): apparmor="DENIED" operation="exec" profile="snap.microk8s.daemon-proxy" name="/bin/kmod" pid=11349 comm="kube-proxy" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[ 4350.248020] audit: type=1400 audit(1542369829.652:205): apparmor="DENIED" operation="exec" profile="snap.microk8s.daemon-proxy" name="/bin/kmod" pid=11350 comm="kube-proxy" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
[ 4355.214907] audit: type=1400 audit(1542369834.620:241): apparmor="DENIED" operation="mkdir" profile="snap.microk8s.daemon-controller-manager" name="/run/kubernetes/" pid=12275 comm="kube-controller" requested_mask="c" denied_mask="c" fsuid=0 ouid=0
[ 4355.226638] audit: type=1400 audit(1542369834.632:242): apparmor="DENIED" operation="mkdir" profile="snap.microk8s.daemon-apiserver" name="/run/kubernetes/" pid=12150 comm="kube-apiserver" requested_mask="c" denied_mask="c" fsuid=0 ouid=0
[ 4356.005864] audit: type=1400 audit(1542369835.412:243): apparmor="DENIED" operation="mkdir" profile="snap.microk8s.daemon-apiserver" name="/run/kubernetes/" pid=12351 comm="kube-apiserver" requested_mask="c" denied_mask="c" fsuid=0 ouid=0

The mount profile contains:

zyga@fyke:...

Changed in snapd:
status: Incomplete → Fix Committed
Zygmunt Krynicki (zyga)
Changed in snapd:
milestone: none → 2.36.2
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.