lxc-start failure on vivid

[Eric Snow]

When I run the following (after populating the template directory with 2 files) it fails:

    sudo lxc-start --daemon --name juju-vivid-lxc-template --logfile /var/lib/juju/containers/juju-vivid-lxc-template/container.log --logpriority WARN --console-log /var/lib/juju/containers/juju-vivid-lxc-template/console.log

I'm trying to determine if this is an issue in juju or in lxc. The two files are a cloud-init script and lxc.conf. The conf file contains the following:

# network config
# interface "eth0"
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.mtu = 9001

After running the command I find a new container.log with a bunch of logging and an empty console.log (also new). In the container.log file everything looks fine until the following lines right after a bunch of "cgroup" messages:

      lxc-start 1426888101.167 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:183 - No such file or directory - failed to change apparmor profile to lxc-container-default
      lxc-start 1426888101.167 ERROR lxc_sync - sync.c:__sync_wait:51 - invalid sequence number 1. expected 4
      lxc-start 1426888101.167 ERROR lxc_start - start.c:__lxc_start:1157 - failed to spawn 'juju-vivid-lxc-template'

This is followed by a bunch of other error messages, but I figured this is the key bit. If necessary I can attach the whole log.

[Eric Snow (ericsnowcurrently)]

If I run in foreground mode, this is what gets printed out:

lxc-start: lsm/apparmor.c: apparmor_process_label_set: 183 No such file or directory - failed to change apparmor profile to lxc-container-default
lxc-start: sync.c: __sync_wait: 51 invalid sequence number 1. expected 4
lxc-start: start.c: __lxc_start: 1157 failed to spawn 'juju-vivid-lxc-template'
lxc-start: cgmanager.c: cgm_remove_cgroup: 518 call to cgmanager_remove_sync failed: invalid request
lxc-start: cgmanager.c: cgm_remove_cgroup: 520 Error removing all:lxc/juju-vivid-lxc-template-21
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.

[Eric Snow (ericsnowcurrently)]

This *may* be related to https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1347020.

[Eric Snow (ericsnowcurrently)]

LXC was installed with "sudo apt-get install lxc" on vivid.

$ lxc-ls --version
1.1.0

I'm not exactly sure which vivid revision we are running other than than it's from at most within the last week. If I recall correctly we're updating it nightly, but I'll need to double-check.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu Vivid Vervet (development branch)
Release: 15.04
Codename: vivid

[Eric Snow (ericsnowcurrently)]

FWIW, I don't think this is related to lp:1347020.

[Curtis Hovey (sinzui)]

Per strikov, this might be bug 1432683. I ran
     sudo /etc/init.d/apparmor reload
on the vivid-slave-b machine that is reporting the issue. Then reran the juju local deploy test and it failed. This is a different kind of issue.

[Eric Snow (ericsnowcurrently)]

FTR:

$ sudo aa-status
apparmor module is loaded.
11 profiles are loaded.
11 profiles are in enforce mode.
   /sbin/dhclient
   /usr/bin/lxc-start
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/libvirt/virt-aa-helper
   /usr/sbin/libvirtd
   /usr/sbin/tcpdump
   lxc-container-default
   lxc-container-default-with-mounting
   lxc-container-default-with-nesting
0 profiles are in complain mode.
21 processes have profiles defined.
21 processes are in enforce mode.
   /usr/bin/lxc-start (31271)
   /usr/sbin/libvirtd (6819)
   lxc-container-default (578)
   lxc-container-default (850)
   lxc-container-default (918)
   lxc-container-default (935)
   lxc-container-default (1083)
   lxc-container-default (1092)
   lxc-container-default (1093)
   lxc-container-default (1109)
   lxc-container-default (1184)
   lxc-container-default (1248)
   lxc-container-default (1250)
   lxc-container-default (7676)
   lxc-container-default (7678)
   lxc-container-default (12780)
   lxc-container-default (12815)
   lxc-container-default (16641)
   lxc-container-default (17358)
   lxc-container-default (31290)
   lxc-container-default (32719)
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

[Eric Snow (ericsnowcurrently)]

The root cause in lp-1432683 sounds likely for this issue as well. I have verified that "/lib/init/apparmor-profile-load" does not exist on the host.

[Serge Hallyn (serge-hallyn)]

Sorry for the mess - I was thinking this was the cgmangaer bug. yes, this is due to not having cgroup-bin package installed