Upgrade from 2.8.0-0ubuntu38 to 2.8.95~2430-0ubuntu2 breaks LXC containers

[Stéphane Graber]

I've been getting a few issues on a bunch of machines over the past few days, mostly unprivileged LXC containers reporting mount failures at boot time, leading to them failing miserably.

The failures in question are:
[ 1084.404894] type=1400 audit(1395617066.637:62): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default" name="/sys/fs/cgroup/" pid=12858 comm="mount" fstype="tmpfs" srcname="none" flags="rw"
[ 1084.405042] type=1400 audit(1395617066.637:63): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default" name="/sys/fs/cgroup/" pid=12858 comm="mount" fstype="tmpfs" srcname="none" flags="ro"
[ 1084.406013] type=1400 audit(1395617066.637:64): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default" name="/run/" pid=12859 comm="mount" fstype="tmpfs" srcname="none" flags="rw, nosuid, noexec"
[ 1084.406127] type=1400 audit(1395617066.637:65): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default" name="/run/" pid=12859 comm="mount" fstype="tmpfs" srcname="none" flags="ro, nosuid, noexec"

Those happen when running under our usual, unmodified lxc-container-default profile which includes container-based which contains:
root@vorash:~# grep tmpfs /etc/apparmor.d/abstractions/lxc/container-base
  # allow tmpfs mounts everywhere
  mount fstype=tmpfs,

Downgrading to 2.8.0-0ubuntu38 and reloading apparmor appears to resolve the issue, so this appears to be a parser bug rather than one of our usual kernel regressions...

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.8.95~2430-0ubuntu3

---------------
apparmor (2.8.95~2430-0ubuntu3) trusty; urgency=medium

  [ Jamie Strandboge ]
  * debian/lib/apparmor/functions: properly calculate number of profiles in
    /var/lib/apparmor/profiles (LP: #1295816)
  * autostart aa-notify via /etc/xdg/autostart instead of /etc/X11/Xsession.d
    (LP: #1288241)
    - remove debian/notify/90apparmor-notify
    - add debian/notify/apparmor-notify.desktop
    - debian/apparmor-notify.install: adjust for the above
    - add debian/apparmor-notify.maintscript to remove 90apparmor-notify
  * debian/notify/notify.conf: use_group should be set to "sudo" instead of
    "admin" (LP: #1009666)

  [ Tyler Hicks ]
  * debian/patches/initialize-mount-flags.patch: Initialize the variables
    containing mount rule flags to zero. Otherwise, the parser may set
    unexpected bits in the mount flags field for rules that do not specify
    mount flags. The uninitialized mount flag variables may have caused
    unexpected AppArmor denials during mount mediation. (LP: #1296459)
  * debian/patches/fix-typo-in-dbus_write.patch: Fix a bug in the
    apparmor/aa.py module that caused the utilities in the apparmor-utils
    package to write out network rules instead of dbus rules
  * debian/patches/limited-mount-rule-support.patch: Fix a bug in the
    apparmor/aa.py module that caused the utilities in the apparmor-utils
    package to traceback when encountering a mount rule (LP: #1294825)
  * debian/patches/bare-capability-rule-support.patch: Fix a bug in the
    apparmor/aa.py module that caused the utilities in the apparmor-utils
    package to traceback when encountering a bare capability rule
    (LP: #1294819)
  * debian/patches/check-config-for-sysctl.patch,
    debian/patches/increase-swap-size.patch: Fix bugs in the regression test
    suite that caused errors when running on ppc64el
  * debian/patches/test-v6-policy.patch,
    debian/patches/test-mount-mediation.patch: Improve the regression tests
    by increasing the mount rule test coverage
 -- Tyler Hicks <email address hidden> Thu, 27 Mar 2014 14:12:29 -0500

[shemgp (shemgp)]

I'm getting this when starting lxd images on Zesty:

  lxc 20160212143429.678 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:220 - If you really want to start this container, set
  lxc 20160212143429.678 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:221 - lxc.aa_allow_incomplete = 1
  lxc 20160212143429.678 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:222 - in your container configuration file
  lxc 20160212143429.679 ERROR lxc_sync - sync.c:__sync_wait:57 - An error occurred in another process (expected sequence number 5)
  lxc 20160212143429.679 ERROR lxc_start - start.c:__lxc_start:1346 - Failed to spawn container "nextcloud".
  lxc 20160212143430.314 ERROR lxc_conf - conf.c:run_buffer:405 - Script exited with status 1.

Perhaps it's related?

[Stéphane Graber (stgraber)]

@shemgp, this suggests that you're using a non-ubuntu kernel which comes with incomplete apparmor support. This typically happens when you're using a mainline kernel build rather than an official Ubuntu kernel.

You can force LXD to use such a kernel, though as mentioned, confinement will be partial.

    lxc profile default set raw.lxc lxc.aa_allow_incomplete=1

Should ensure it's set for all your containers.