[vivid] lxc container with systemd fails to boot under libvirt-lxc

[Harald Hetzner]

Under vivid, a vivid container fails to boot with systemd, printing the following error message in console:

Failed to mount cgroup at /sys/fs/cgroup/systemd: Permission denied
systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN)
Detected virtualization 'lxc-libvirt'.
Detected architecture 'x86-64'.

Welcome to Ubuntu Vivid Vervet (development branch)!

Set hostname to <test>.
Failed to install release agent, ignoring: No such file or directory
Failed to create root cgroup hierarchy: No such file or directory
Failed to allocate manager object: No such file or directory
[!!!!!!] Failed to allocate manager object, freezing.

On the host, the following dmesg is found:

[ 805.407722] audit: type=1400 audit(1429295378.619:150): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" pid=3796 comm="apparmor_parser"
[ 805.431061] device vnet0 entered promiscuous mode
[ 805.446988] IPv6: ADDRCONF(NETDEV_UP): vnet0: link is not ready
[ 806.043772] eth0: renamed from vnet1
[ 806.067844] IPv6: ADDRCONF(NETDEV_CHANGE): vnet0: link becomes ready
[ 806.067942] virbr0: port 2(vnet0) entered listening state
[ 806.067959] virbr0: port 2(vnet0) entered listening state
[ 806.096686] audit: type=1400 audit(1429295379.307:151): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/systemd/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 806.096914] audit: type=1400 audit(1429295379.307:152): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/systemd/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 806.098253] audit: type=1400 audit(1429295379.307:153): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/freezer/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 806.098474] audit: type=1400 audit(1429295379.307:154): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/net_cls,net_prio/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 806.098640] audit: type=1400 audit(1429295379.307:155): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/devices/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 806.098805] audit: type=1400 audit(1429295379.307:156): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/cpu,cpuacct/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 806.098978] audit: type=1400 audit(1429295379.307:157): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/blkio/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 806.099149] audit: type=1400 audit(1429295379.307:158): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="libvirt-9d578815-a1e9-4596-aef9-a70717574f0e" name="/sys/fs/cgroup/cpuset/" pid=3834 comm="systemd" fstype="cgroup" srcname="cgroup" flags="rw, nosuid, nodev, noexec"
[ 808.073724] virbr0: port 2(vnet0) entered learning state
[ 810.079825] virbr0: topology change detected, propagating
[ 810.079854] virbr0: port 2(vnet0) entered forwarding state

To reproduce the bug, do the following (libvirt XML file is attached):

$ lxc-create -P /lxc -n test -t download -B btrfs -- --dist=ubuntu --release=vivid --arch=amd64
$ virsh -c lxc:/// define test.xml
$ virsh -c lxc:/// start test

Then use e.g. virt-manager to view the console output.

Package versions:

apparmor = 2.9.1-0ubuntu9
cgmanager = 0.36-2ubuntu5
libvirt-bin = 1.2.12-0ubuntu12
lxc = 1.1.2-0ubuntu3
lxcfs = 0.7-0ubuntu2
systemd = 219-7ubuntu2

[Harald Hetzner (haraldhetzner)]

The container can be booted via lxc-start using the attached config.

The container can also be booted using libvirt-lxc as long as it is configured to use /sbin/upstart as init.

However, when configured to boot on /bin/systemd, it still fails as reported above.

Is anyone else experiencing this problem?

[Serge Hallyn (serge-hallyn)]

> The container can be booted via lxc-start using the attached config.
>
> The container can also be booted using libvirt-lxc as long as it is
> configured to use /sbin/upstart as init.
>
> However, when configured to boot on /bin/systemd, it still fails as
> reported above.
>
> Is anyone else experiencing this problem?

lxc-start (lxc) is using a mount hook to provide /sys/fs/cgroup/*
mounts which systemd reuqires in order to boot. libvirt-lxc is not
currently doing that. This is a known missing feature at the moment.
(There are probably other things such as /dev needing to be a on
separate device from /)

[Harald Hetzner (haraldhetzner)]

Thank you for pointing this out.

Based on your reply, I decided to have the host system boot on /bin/systemd and have the containers boot on /sbin/upstart. So far, this works without problems.