comments in common.conf must be updated

[Diogo Matsubara]

I tried to run a juju charm (jenkins-lxc) that starts a lxc container so I added lxc.aa_profile = lxc-container-default-with-nesting and lxc.hook.mount = /usr/share/lxc/hooks/mountcgroups to /var/lib/lxc/juju-trusty-template/config and then tried to deploy the service. I got a failure from juju:

agent-state-info: 'error executing "lxc-start": The container failed to start.;
      To get more details, run the container in foreground mode.; Additional information
      can be obtained by setting the --logfile and --log-priority options.'

So I tried to start the container manually:

$ sudo lxc-start -n matsubara-local-machine-1 --logpriority DEBUG --logfile /tmp/lxc.log which gave me this log: http://paste.ubuntu.com/7805486/

I removed lxc.hook.mount = /usr/share/lxc/hooks/mountcgroups from the /var/lib/lxc/juju-trusty-template/config and tried again. Got the same error

I created the file /etc/default/cgmanager and added cgmanager_opts="--debug"

And got in /var/log/upstart/cgmanager.log: http://paste.ubuntu.com/7805602/

Additional info:
<hallyn> release, kernel version, lxc version, cgmanager version
<matsubara> hallyn, I'm running this on Trusty, 3.13.0-30-generic, lxc 1.0.4-0ubuntu0.1 and 0.24-0ubuntu7

/proc/self/cgroup content: http://paste.ubuntu.com/7805492/

The config for the juju template used to start local provider containers in /var/lib/lxc/juju-trusty-template/config: http://paste.ubuntu.com/7805606/
And the config for /var/lib/lxc/matsubara-local-machine-1/config: http://paste.ubuntu.com/7805610/

[Serge Hallyn (serge-hallyn)]

Thanks for submitting this bug.

In the description you say that you removed the lxc.hook.mount = mountcgroups line, but can you please re-try that? When I add that line to a fresh trusty container, I get the exact results you do. When I remove it, the container starts fine.

[Diogo Matsubara (matsubara)]

Hi Serge, you're right. I removed the lxc.hook.mount = mountcgroups from the wrong template. Now that I destroyed the whole juju environment, removed lxc.hook.mount = mountcgroups from the correct one and bootstrapped again, I got juju's lxc containers to start correctly. Thank you for helping debug this.

In any case, I got the instructions to include that lxc.hook.mount = mountcgroups in /usr/share/lxc/config/ubuntu.common.conf which says:

# To support container nesting on an Ubuntu host while retaining most of
# apparmor's added security, use the following two lines instead.
#lxc.aa_profile = lxc-container-default-with-nesting
#lxc.hook.mount = /usr/share/lxc/hooks/mountcgroups

So I think this is at least a documentation bug.

[Diogo Matsubara (matsubara)]

Fwiw, I ran into another problem, different from the originally reported here. This time the error happened when I tried to start the nested container. It's well described here: https://lists.linuxcontainers.org/pipermail/lxc-users/2014-April/006584.html and the solution in that thread worked for me. That is, using lxc.auto.mount = cgroups in /var/lib/lxc/juju-trusty-template/config avoids the following error:

 lxc-start: cgroupfs failed to detect cgroup metadata
 lxc-start: failed initializing cgroup support

[Serge Hallyn (serge-hallyn)]

Thanks for verifying. What is happening is actually vaguely explained in the mountcgroup hook itself, and is an unfortunate side effect of a somewhat recent kernel change:

cd /sys/fs/cgroup/devices
sudo mkdir a
echo a | sudo tee -a a/devices.deny # succeeds
sudo mkdir -p b/c
echo a | sudo tee -a b/devices.deny # fails

If a devices cgroup has any child cgroups, then you can no longer make certain changes to it.

Marking this confirmed and changing the title to reflect that the comments in /usr/share/lxc/config/ubuntu.common.conf need to be changed.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lxc - 1.1.0~alpha1-0ubuntu2

---------------
lxc (1.1.0~alpha1-0ubuntu2) utopic; urgency=medium

  * d/p/0001-lxc-test-unpriv-usernic.in-make-sure-to-chgrp-as-wel.patch:
    Fix test failures in jenkins.
  * d/p/0002-Remove-mention-of-mountcgroups-in-ubuntu.common-conf.patch:
    Fix the comment in the ubuntu common config about how to support nesting.
    (LP: #1342960)
 -- Serge Hallyn <email address hidden> Thu, 17 Jul 2014 16:42:46 -0500