apt-get install lxc doesn't load required apparmor profiles

[Oleg Strikov]

I'm trying to use LXC on my openstack instance which runs vivid daily:

$ sudo apt-get install lxc -y

$ sudo lxc-create -t ubuntu-cloud --name=vivid -- --flush-cache --stream=daily --release=vivid

$ sudo lxc-start --name vivid --logfile=lxc.log
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.

In the log file (lxc.log) I observe the following error:
lxc-start 1426516387.814 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:183 - No such file or directory - failed to change apparmor profile to lxc-container-default

This profile *exists* under /etc/apparmor.d/lxc/lxc-default but was not loaded appropriately.

This issue disappears if I:
(a) reload apparmor profile manually: sudo /etc/init.d/apparmor reload
or
(b) reboot the instance

I'd expect that 'apt-get install lxc' has to load all appropriate apparmor profiles to allow starting containers w/o profile reloading / rebooting.

[Serge Hallyn (serge-hallyn)]

I couldn't reproduce this. I installed a fresh vivid vm, did apt-get install lxc,
then

ubuntu@lxc-aa:~$ sudo aa-status
apparmor module is loaded.
9 profiles are loaded.
9 profiles are in enforce mode.
   /sbin/dhclient
   /usr/bin/lxc-start
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/tcpdump
   lxc-container-default
   lxc-container-default-with-mounting
   lxc-container-default-with-nesting

and

ubuntu@lxc-aa:~$ sudo lxc-create -t ubuntu-cloud --name=vivid
[...]
ubuntu@lxc-aa:~$ sudo lxc-start -n vivid
ubuntu@lxc-aa:~$ sudo lxc-ls -f
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
---------------------------------------------
vivid RUNNING - - - NO

[Serge Hallyn (serge-hallyn)]

Could you please show the result of

sudo aa-status

and the contents of /tmp/debug.out after

sudo lxc-start -n vivid -l trace -o /tmp/debug.out

[Serge Hallyn (serge-hallyn)]

Oh, sorry, and 'uname -a'

[Oleg Strikov (strikov)]

Hi Serge,

Many thanks for looking at the bug.
Here is information you requested.

ubuntu@vivid-lxc-bug:~$ uname -a
Linux vivid-lxc-bug 3.19.0-9-generic #9-Ubuntu SMP Wed Mar 11 17:50:03 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

ubuntu@vivid-lxc-bug:~$ sudo aa-status
apparmor module is loaded.
6 profiles are loaded.
6 profiles are in enforce mode.
   /sbin/dhclient
   /usr/bin/lxc-start
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/tcpdump
0 profiles are in complain mode.
1 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
1 processes are unconfined but have a profile defined.
   /sbin/dhclient (508)

/tmp/debug.out:
http://paste.ubuntu.com/10614837/

[Serge Hallyn (serge-hallyn)]

Thanks. The cause of this is that /lib/init/apparmor-profile-load is not installed. It comes with the 'upstart-bin' package. It is used by the script /usr/lib/x86_64-linux-gnu/lxc/lxc-apparmor-load (which is an ExecStartPre for lxc) to load the profiles.

[Robie Basak (racb)]

From my grep through the archive, apparmor-profile-load is mentioned only in the following packages:

apparmor avahi cups cups-filters lxc mysql-5.6 rsyslog squid3 sssd strongswan upstart

[Robie Basak (racb)]

(source packages that is)

[Serge Hallyn (serge-hallyn)]

Thanks, Robie - the only one on that list which needs an update is squid3.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.9.1-0ubuntu8

---------------
apparmor (2.9.1-0ubuntu8) vivid; urgency=medium

  [ Steve Beattie ]
  * debian/rules: run make check on the libapparmor library
  * add-chromium-browser.patch: add support for chromium policies
    (LP: #1419294)
  * debian/apparmor.{init,upstart}: add support for triggering
    aa-profile-hook runs when packages are updated via snappy system
    image updates (LP: #1434143)
  * parser-fix_modifier_compilation_+_tests.patch: fix compilation
    of audit modifiers for exec and pivot_root and deny modifiers on
    link rules as well as significantly expand related tests
    (LP: #1431717, LP: #1432045, LP: #1433829)
  * tests-fix_systemd_breakage_in_pivot_root-lp1436109.patch: work
    around pivot_root test failures due to init=systemd (LP: #1436109)
  * GDM_X_authority-lp1432126.patch: add location GDM creates Xauthority
    file to X abstraction (LP: #1432126)

  [ Jamie Strandboge ]
  * easyprof-framework-policy.patch: add --include-templates-dir and
    --include-policy-groups-dir options to easyprof to support framework
    policy on snappy

  [ Robie Basak ]
  * Add /lib/apparmor/profile-load; moved from
    /lib/init/apparmor-profile-load from the upstart package. A wrapper at
    the original path is now provided by init-system-helpers. (LP: #1432683)
 -- Jamie Strandboge <email address hidden> Sat, 28 Mar 2015 07:22:30 -0500

[Serge Hallyn (serge-hallyn)]

This is fixed for lxc through the other package uploads. lxc already depends on init-system-helpers so will need no packaging changes itself.

[Launchpad Janitor (janitor)]

This bug was fixed in the package init-system-helpers - 1.22ubuntu6

---------------
init-system-helpers (1.22ubuntu6) vivid; urgency=medium

  * Add /lib/init/apparmor-profile-load; moved from the upstart package to the
    apparmor package and wrapped here under the old path. This name can
    continue to be used by init scripts to save them individually testing if
    apparmor is installed, as this wrapper performs this task.
    (LP: #1432683)
 -- Robie Basak <email address hidden> Thu, 02 Apr 2015 11:13:36 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package upstart - 1.13.2-0ubuntu11

---------------
upstart (1.13.2-0ubuntu11) vivid; urgency=medium

  * Remove /lib/init/apparmor-profile-load, as it is being moved to the
    init-system-helpers package. Add new dependency on init-system-helpers
    to transition all current systems. (LP: #1432683)
 -- Serge Hallyn <email address hidden> Thu, 02 Apr 2015 11:21:23 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package squid3 - 3.3.8-1ubuntu14

---------------
squid3 (3.3.8-1ubuntu14) vivid; urgency=medium

  * Add versioned dependency on init-system-helpers (>> 1.22ubuntu5) to ensure
    we have the apparmor-profile-load script at boot time. (LP: #1432683)
 -- Serge Hallyn <email address hidden> Thu, 02 Apr 2015 11:12:27 -0500

[Martin Pitt (pitti)]

This really doesn't belong into init-system-helpers. It was just right in upstart, as it's only being used in upstart jobs. I don't want to have a permanent delta just for this.

[Martin Pitt (pitti)]

lxc wasn't fixed for this yet -- /usr/lib/x86_64-linux-gnu/lxc/lxc-apparmor-load still calls the wrapper, and not /lib/apparmor/profile-load.

[Martin Pitt (pitti)]

Upstream pull request for LXC: https://github.com/lxc/lxc/pull/512

[Jens Elkner (jelmd)]

It appears, that something is still broken. Because systemd doesn't work, I installed upstart + upstart-sysv (and uninstalled systemd-sysv), but unfortunately sssd doesn't come up (has exactly the same config, as in other < 14.10 zones, where it works as expected). And because sssd doesn't come up, other depending services like autofs doesn't come up either.

The problem seems to be /lib/init/apparmor-profile-load as well, which returns with 1 and thus probably causes start always fail.
As a workaround I modified /etc/init/sssd.conf:
...
pre-start script
 test -f /etc/sssd/sssd.conf || { stop; exit 0; }
 /lib/init/apparmor-profile-load usr.sbin.sssd || true
end script
...

which makes it work, however, I still wonder, what apparmor-profile-load causes to return != 0 ...

[Serge Hallyn (serge-hallyn)]

Does /bin/running-in-container exist in those containers?

If so, what does

/bin/running-in-container; echo $?

show?

[Martin Pitt (pitti)]

Wily's lxc contains the upstream MP, i. e. it calls /lib/apparmor/profile-load.

[Launchpad Janitor (janitor)]

This bug was fixed in the package init-system-helpers - 1.24ubuntu1

---------------
init-system-helpers (1.24ubuntu1) xenial; urgency=medium

  * Merge with Debian unstable. Remaining Ubuntu changes:
    - init: Drop sysvinit-core as alternative pre-depends.
    - Temporarily add /bin/running-in-container until it finds a more
      appropriate place (LP: #1442228)
  * Drop /lib/init/apparmor-profile-load wrapper, it got moved to
    upstart (LP: #1432683)

init-system-helpers (1.24) unstable; urgency=medium

  [ Helmut Grohne ]
  * Fix FTCBFS: Annotate perl dependency with :any. (Closes: #794339)

  [ Reiner Herrmann ]
  * dh_systemd_{enable,start}: Sort list of unit files, to make generated
    maintainer scripts reproducible. (Closes: #801470)

 -- Martin Pitt <email address hidden> Mon, 26 Oct 2015 09:06:27 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package upstart - 1.13.2-0ubuntu17

---------------
upstart (1.13.2-0ubuntu17) xenial; urgency=medium

  * Put back /lib/init/apparmor-profile-load shim, as some upstart jobs still
    use that instead of /lib/apparmor/profile-load. (LP: #1432683)

 -- Martin Pitt <email address hidden> Mon, 26 Oct 2015 08:59:04 +0100