Unprivileged nested Xenial container will not start inside a privileged Xenial container

[Christopher Townsend]

I have a host where I create a privileged Xenial container. Inside that privileged container, I create an unprivileged Xenial container using the ubuntu user in the privileged container. When I try starting the unprivileged container, I get the following error:

      lxc-start 20160209133200.970 INFO lxc_cgroup - cgroup.c:cgroup_init:65 - cgroup driver cgroupfs initing for test-lxc
      lxc-start 20160209133200.971 ERROR lxc_cgfs - cgfs.c:cgfs_init:2248 - cgroupfs failed to detect cgroup metadata
      lxc-start 20160209133200.971 ERROR lxc_start - start.c:lxc_spawn:1023 - failed initializing cgroup support
      lxc-start 20160209133200.971 ERROR lxc_start - start.c:__lxc_start:1273 - failed to spawn 'test-lxc'

I've tried both Trusty host w/ ppa:ubuntu-lxc/lxc-stable installed and a Xenial host with the same results. I've also tried using ppa:ubuntu-lxc/daily in the privileged Xenial LXC.

[Christopher Townsend (townsend)]

I have tried this again and the error is different now. I've attached a debug log.

[Serge Hallyn (serge-hallyn)]

A fix for this has been sent upstream. A new rc will be released very soon with the fix.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lxc - 2.0.0~rc5-0ubuntu1

---------------
lxc (2.0.0~rc5-0ubuntu1) xenial; urgency=medium

  * New usptream release (2.0.0~rc5)
    - Fix a number of cgfs issues (LP: #1549363, LP: #1543697, LP: #1552355)
    - Fix attach failing to allocate a tty (LP: #1551960)
    - Fix LXC rebooting the container despite post-stop failure
    - Fix lxc-copy output (LP: #1551935)
    - Documentation, manpagen and manpage translations update
    - Update to the plamo template

 -- Stéphane Graber <email address hidden> Thu, 03 Mar 2016 11:05:25 -0500

[Christopher Townsend (townsend)]

This still doesn't seem fixed. I've attached the latest debug log.

[Serge Hallyn (serge-hallyn)]

Please show which version you are currently using?

[Christopher Townsend (townsend)]

I have version 2.0.0~rc5-0ubuntu1 on both the Xenial host and Xenial privileged container installed. I have rebooted the host and have also stopped and restarted the privileged container.

[Serge Hallyn (serge-hallyn)]

Hi,

I'm having a problem as well, but a different one in recent xenial kernels.

Can you tell us the release and kernel (uname -a) of your host?

[Christopher Townsend (townsend)]

Sure, here is the requested info:

$ uname -a
Linux Slave1 4.4.0-9-generic #24-Ubuntu SMP Mon Feb 29 19:33:19 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

[Serge Hallyn (serge-hallyn)]

I'm running 4.4.0-10-generic #25-Ubuntu with lxc version 2.0.0~rc5-0ubuntu1 , and the privileged container has:

lxc.include = /usr/share/lxc/config/nesting.conf

in the configuration file. This is a clean VM with no ppas installed. Inside the privileged container (also xenial with no ppas), I can create containers as user ubuntu.

Can you show the privileged container configuration file, and /proc/self/cgroup for the shell inside the privileged container from which you are failing to start an unprivileged container?

[Christopher Townsend (townsend)]

I tried adding "lxc.include = /usr/share/lxc/config/nesting.conf" as I didn't know this existed. However, it didn't help.

As you say, I can create unprivileged containers as user ubuntu, I just can't start them. Are you able to start unprivileged containers as user ubuntu? If so, how are you starting them? I need to start them from the host's shell, so I'm doing something like this:

$ sudo lxc-attach -n test-libertine -- sudo -u ubuntu -H lxc-start -n test

where test-libertine is the privileged container and test is the unprivileged container.

Also, here is the privileged container's configuration file:

# Template used to create this container: /usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -a amd64 -d ubuntu -r xenial
# For additional config options, please look at lxc.container.conf(5)

# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.arch = x86_64

# Container specific configuration
lxc.rootfs = /var/lib/lxc/test-libertine/rootfs
lxc.utsname = test-libertine

# Network configuration
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:da:08:f7

lxc.aa_profile = lxc-container-default-with-nesting
lxc.include = /usr/share/lxc/config/nesting.conf

Here is the output of /proc/self/cgroup while in a shell inside the privileged container:

# cat /proc/self/cgroup
11:devices:/
10:freezer:/
9:pids:/
8:memory:/
7:cpuset:/
6:hugetlb:/
5:blkio:/
4:net_cls,net_prio:/
3:perf_event:/
2:cpu,cpuacct:/
1:name=systemd:/

[Serge Hallyn (serge-hallyn)]

Quoting Christopher Townsend (<email address hidden>):
> I tried adding "lxc.include = /usr/share/lxc/config/nesting.conf" as I
> didn't know this existed. However, it didn't help.
>
> As you say, I can create unprivileged containers as user ubuntu, I just
> can't start them. Are you able to start unprivileged containers as user
> ubuntu? If so, how are you starting them? I need to start them from

Yes. I start them by ssh'ing in as user ubuntu and doing lxc-start.
Does that work for you?

> the host's shell, so I'm doing something like this:
>
> $ sudo lxc-attach -n test-libertine -- sudo -u ubuntu -H lxc-start -n
> test

This may not be enough to trigger pam-cgfs to create cgroups for you,
so that may be the reason it fails. What does

sudo lxc-attach -n test-libertine -- sudo -u ubuntu -H cat /proc/self/cgroup

show?

[Christopher Townsend (townsend)]

Hi Serge,

Thanks for working with me on this.

So it does work when I ssh into the privileged container as user ubuntu and then start the unprivileged container.

I'm guessing there may be no way to start the unprivileged container from lxc-attach like I want to, right?

Here is the output you requested:

$ sudo lxc-attach -n test-libertine -- sudo -u ubuntu -H cat /proc/self/cgroup
11:cpuset:/
10:pids:/
9:memory:/
8:blkio:/
7:perf_event:/
6:freezer:/
5:hugetlb:/
4:devices:/
3:cpu,cpuacct:/
2:net_cls,net_prio:/
1:name=systemd:/

[Serge Hallyn (serge-hallyn)]

Hi - there should be a way, we just need to get sudo running the pam module.

Perhaps libpam-cgfs should be added to /etc/pam.d/common-session-noninteractive.

[Serge Hallyn (serge-hallyn)]

this should be fixed by the fix to bug 1554269.