lxc-start fails with 1.1.5-0ubuntu1

Bug #1516037 reported by Colin Watson on 2015-11-13
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Canonical System Image
Critical
Unassigned
apparmor (Ubuntu)
Undecided
Unassigned
lxc (Ubuntu)
Undecided
Unassigned

Bug Description

After upgrading to lxc 1.1.5-0ubuntu1, lxc-start fails like this:

lxc-start: start.c: preserve_ns: 149 Permission denied - failed to open '/proc/7170/ns/mnt'
lxc-start: start.c: lxc_spawn: 993 failed to store namespace references
lxc-start: start.c: __lxc_start: 1192 failed to spawn 'trusty-lpdev'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.

This is with a trusty system container. precise system containers behave similarly. I don't have others to try. Downgrading liblxc1, lxc, lxc-templates, and python3-lxc to version 1.1.4-0ubuntu3 causes lxc-start to work again.

Colin Watson (cjwatson) wrote :
Serge Hallyn (serge-hallyn) wrote :

Could you post the container configuration file?

Colin Watson (cjwatson) wrote :
Colin Watson (cjwatson) wrote :
Colin Watson (cjwatson) wrote :

But I should also note that the same thing happened with a freshly-created container, using "sudo lxc-create -n trusty-test -t ubuntu -- -r trusty". So I don't think my custom apparmor hack there can be at fault.

Stéphane Graber (stgraber) wrote :

Hi Colin,

Could you:
 - Tell us what architecture you're running on (noticed the container config shows i386)
 - Confirm that both lxc and liblxc1 are at the same version
 - Tell us the exact kernel version you're running
 - Look for any apparmor denials in dmesg

Thanks!

Colin Watson (cjwatson) wrote :

My host is xenial amd64; the guest is i386 because a million lines of Python is rather more memory-efficient that way. :-)

Yes, lxc and liblxc1 are at the same version.

Kernel is linux-image-4.2.0-17-generic 4.2.0-17.21, though this also happened with 4.2.0-16.19 (I rebooted to see if that cleared it up).

There are no apparmor denials in dmesg.

Serge Hallyn (serge-hallyn) wrote :

What does

ls -l /proc/sys/kernel/yama/
cat /proc/sys/kernel/yama/ptrace_scope

show?

Serge Hallyn (serge-hallyn) wrote :

Nm, afaict yama should never prevent this.

Stéphane Graber (stgraber) wrote :

This bug was caused by a silent apparmor denial due to a local modification of /etc/apparmor.d/usr.bin.lxc-start (dropped attach_disconnected flag).

Changed in lxc (Ubuntu):
status: New → Invalid
Francis (francisd) wrote :

Mmm I'm having the same problem with lxc 1.1.5 on ubuntu 12.04 (using the ppa). I had to downgrade to 1.1.4 to make my containers bootable again.

I don't think I manually modified the file /etc/apparmor.d/usr.bin.lxc-start. It contains:

#include <tunables/global>

/usr/bin/lxc-start flags=(complain) {
  #include <abstractions/lxc/start-container>
}

I'm using LXC on two ubuntu 12.04 box and only one is failing with lxc-1.1.5.

Stéphane Graber (stgraber) wrote :

That most definitely is not our default profile, so you or some script you ran modified it.

You may want to install "debsums" and run "debsums -sa lxc" to get a list of configuration files which have been modified on your system.

Francis (francisd) wrote :

debsums: changed file /etc/default/lxc (from lxc package)
debsums: changed file /etc/apparmor.d/usr.bin.lxc-start (from lxc package)

So I restored the /etc/apparmor.d/usr.bin.lxc-start file to:

#include <tunables/global>

/usr/bin/lxc-start flags=(attach_disconnected) {
  #include <abstractions/lxc/start-container>
}

The only thing I remember I did is running aa-complain /usr/bin/lxc-start a while ago (I don't remember why) and this is what caused the file to be modified. I reverted aa-complain with aa-enforce /usr/bin/lxc-start, I restored the apparmor profil and redid the aa-complain and the file changed again to:

#include <tunables/global>

/usr/bin/lxc-start flags=(complain) {
  #include <abstractions/lxc/start-container>
}

Stéphane Graber (stgraber) wrote :

Adding an apparmor task as the apparmor tools really shouldn't mangle our profiles in a way that can't be reverted...

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apparmor (Ubuntu):
status: New → Confirmed

Ah, interesting. So this may be a bug in aa-complain, as it did not
retain (iiuc) the attach_disconnected flags.

Christian Boltz (cboltz) wrote :

Which apparmor version do you use?

I can't reproduce the problem using the latest version from the 2.9 branch and bzr trunk. (I tested with abstractions/base instead of abstractions/lxc/start-container in the profile, but that shouldn't matter.)

It is also affecting devel-proposed phone images cf bug 1516971

I confirm that the phone boots successfully after downgrading liblxc1, lxc and python3-lxc to version 1.1.4-0ubuntu3

Changed in canonical-devices-system-image:
importance: Undecided → Critical
status: New → Confirmed
Francis (francisd) wrote :

apparmor 2.7.102-0ubuntu3.10 running Ubuntu 12.04 LTS.

Stéphane Graber (stgraber) wrote :

Oh, the phone problem is different I think, there the problem is that your kernel is pretty old and doesn't have the right /proc/self/ns files available. Looks like that's an actual bug with the new implementation. I'll report it upstream.

Jean-Baptiste, can you un-duplicate the phone bug and open a lxc task for it? It's a different issue from what was reported in this bug report.

Stéphane Graber (stgraber) wrote :

Note that Jean-Baptiste's problem will only be seen on kernels higher than 3.2 and lower than 3.8. So it's not affecting any official distro kernel but it does affect those outdated android kernels.

Christian Boltz (cboltz) wrote :

AppArmor 2.7 is _very_ old - especially given the fact that the tools were rewritten in python for 2.9.

I just checked the perl code (which was used in 2.8.x and older) - it _sets_ the flags (instead of adding or removing them), so it's not surprising that attach_disconnected gets lost. (This is one of the fixes that went into the 2.9 during the rewrite to python.)

If someone is interested to fix this - the code is in Immunix/AppArmor.pm, sub complain() and sub enforce(), which both call setprofileflags().

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments