cgroup-lite should be added as an explicit dependency

[Adam Ryczkowski]

Otherwise you'll get an error

sudo lxc-start -n mycontainer
lxc: call to cgmanager_move_pid_abs_sync(blkio) failed: invalid request
lxc-start: Could not find writable mount point for cgroup hierarchy 11 while trying to create cgroup.
lxc-start: Device or resource busy - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio/grp_1
lxc-start: Device or resource busy - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu/sys_daemon
lxc-start: Device or resource busy - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset/
lxc-start: failed creating cgroups
lxc-start: failed to spawn 'mycontainer'

Or at least let the "lxc-start" check of the presence of the "cgroup-lite" and show a meaningful error message.

Ubuntu vanilla installs the "cgroup-lite" automatically, so it doesn't affect fresh installs.

System: Ubuntu 14.04 64 bit
---
ApportVersion: 2.14.1-0ubuntu2
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: adam 2813 F.... pulseaudio
 /dev/snd/pcmC0D0p: adam 2813 F...m pulseaudio
CurrentDesktop: Unity
DistroRelease: Ubuntu 14.04
EcryptfsInUse: Yes
InstallationDate: Installed on 2014-04-04 (9 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Daily amd64 (20140404)
MachineType: ASUSTeK COMPUTER INC. N56VZ
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-24-generic.efi.signed root=UUID=926fa7cc-6f97-4672-85a7-a1ed8f5bd842 ro rootflags=subvol=@ quiet splash intel_pstate=enable vt.handoff=7
ProcVersionSignature: Ubuntu 3.13.0-24.46-generic 3.13.9
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-24-generic N/A
 linux-backports-modules-3.13.0-24-generic N/A
 linux-firmware 1.127
Tags: trusty
Uname: Linux 3.13.0-24-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm bumblebee cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 12/06/2012
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: N56VZ.216
dmi.board.asset.tag: ATN12345678901234567
dmi.board.name: N56VZ
dmi.board.vendor: ASUSTeK COMPUTER INC.
dmi.board.version: 1.0
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: ASUSTeK COMPUTER INC.
dmi.chassis.version: 1.0
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrN56VZ.216:bd12/06/2012:svnASUSTeKCOMPUTERINC.:pnN56VZ:pvr1.0:rvnASUSTeKCOMPUTERINC.:rnN56VZ:rvr1.0:cvnASUSTeKCOMPUTERINC.:ct10:cvr1.0:
dmi.product.name: N56VZ
dmi.product.version: 1.0
dmi.sys.vendor: ASUSTeK COMPUTER INC.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1306916

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Adam Ryczkowski (adam-ryczkowski)]

I think the log files are irrelevant here, but here they are.

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Adam Ryczkowski (adam-ryczkowski)]

apport information

[Tim Gardner (timg-tpi)]

Does not seem to be a kernel issue.

[Stéphane Graber (stgraber)]

cgroup-lite isn't necessary at all now that we have cgmanager. Adding a cgmanager task to investigate what's causing it to fail for you.

Can you please attach /var/log/upstart/cgmanager.log ?

[Adam Ryczkowski (adam-ryczkowski)]

Here you are.

$ sudo cat /var/log/upstart/cgmanager.log
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Shih-Yuan Lee (fourdollars)]

$ sudo cat /var/log/upstart/cgmanager.log | sort -u
cgmanager: could not open cgroup file for [PID]
...
cgmanager:do_move_pid_main: victim's cgroup is not under proxy's (p.uid 0)
cgmanager:get_pid_cgroup_main: Could not determine the requestor cgroup
cgmanager: Invalid path /run/cgmanager/fs/none,name=systemd///user (No such file or directory)
cgmanager:list_children_main: Could not determine the requested cgroup
cgmanager:victim_under_proxy_cgroup: Could not determine the victim's cgroup

[Serge Hallyn (serge-hallyn)]

Could you please show

1. contents of /proc/cgroups and /proc/self/cgroup

2. the contents of debug.out after doing, without cgroup-lite installed, 'lxc-start -n mycontainer -l trace -o debug.out'

[Adam Ryczkowski (adam-ryczkowski)]

Contents of sudo lxc-start -n puppetmaster -l trace -o /tmp/lxc.log.out

[Adam Ryczkowski (adam-ryczkowski)]

sudo cat /proc/self/cgroup>/tmp/self-cgroup.txt

#subsys_name hierarchy num_cgroups enabled
cpuset 3 1 1
cpu 4 93 1
cpuacct 5 4 1
memory 6 4 1
devices 7 4 1
freezer 8 4 1
blkio 9 172 1
perf_event 10 4 1
hugetlb 11 4 1
[adam:/home/adam] $ cat /tmp/self-cgroup.txt
11:hugetlb:/user/1000.user/c2.session
10:perf_event:/user/1000.user/c2.session
9:blkio:/grp_26763
8:freezer:/user/1000.user/c2.session
7:devices:/user/1000.user/c2.session
6:memory:/user/1000.user/c2.session
5:cpuacct:/user/1000.user/c2.session
4:cpu:/usr_1000/grp_26763
3:cpuset:/
2:name=systemd:/user/1000.user/c2.session

[Adam Ryczkowski (adam-ryczkowski)]

cat /proc/cgroups

#subsys_name hierarchy num_cgroups enabled
cpuset 3 1 1
cpu 4 93 1
cpuacct 5 4 1
memory 6 4 1
devices 7 4 1
freezer 8 4 1
blkio 9 172 1
perf_event 10 4 1
hugetlb 11 4 1

[Adam Ryczkowski (adam-ryczkowski)]

(To reproduce the bug I had to uninstall cgrups-lite back. )

[Serge Hallyn (serge-hallyn)]

The lxc.log.out shows

lxc-start 1397651569.113 INFO lxc_cgroup - cgroup driver cgroupfs initing for puppetmaster
lxc-start 1397651569.113 ERROR lxc_cgfs - Could not find writable mount point for cgroup hierarchy while trying to create cgroup.

So cgmanager is not being used.

How did you remove cgroup-lite? Did you reboot after removing it? What does 'ls /sys/fs/cgroup' and 'ls /sys/fs/cgroup/cgmanger' now show?

Note what is happening here is different from what happened in the Description when you initially filed the bug. In this case my guess would be that the cgmanager socket has been unmounted. restarting cgmanager may fix that.

[Adam Ryczkowski (adam-ryczkowski)]

Yes, I didn't restart. I just did `apt-get remove cgroup-lite`. After I'm done with my work, I'll uninstall cgroup-lite, restart the system and post feedback then. It would be in about 1 hour.

Do I restart cgmanager with `service cgmanager restart`?

To say the truth, the only part I know about cgroup is to run commands `lxc-create`, `lxc-stop`, `lxc-start`, `lxc-info` and occassionaly `lxc-console`.

[Serge Hallyn (serge-hallyn)]

> Yes, I didn't restart. I just did `apt-get remove cgroup-lite`. After
> I'm done with my work, I'll uninstall cgroup-lite, restart the system
> and post feedback then. It would be in about 1 hour.

Great, thanks.

> Do I restart cgmanager with `service cgmanager restart`?

If you are rebooting then that won't be necessary. But 'service xxx
restart' often doesn't suffice. Doing

sudo stop cgmanager
sudo start cgmanager

will make sure that it is restarted, not just signaled.

> To say the truth, the only part I know about cgroup is to run commands
> `lxc-create`, `lxc-stop`, `lxc-start`, `lxc-info` and occassionaly `lxc-
> console`.

Just to be sure, also please show us the contents of /etc/fstab and
dpkg -l | grep libcgroup.

Thanks, hopefully that'll let us figure out what's going on!

[Adam Ryczkowski (adam-ryczkowski)]

After reboot lxc works fine. I can no longer reproduce the problem.

BTW can you tell me (or at least point me somewhere), where I can learn about the cgroup-lite vs cgmanager. What is the difference, and for what exactly they are needed. I thought cgroup support is built into the kernel.

[Serge Hallyn (serge-hallyn)]

Please see https://help.ubuntu.com/14.04/serverguide/cgroups.html ,
and please let me know if there's anything you feel should be added
there. There is also the cgmanager web page itself at
https://cgmanager.linuxcontainers.org/

 status: fixreleased

[Stéphane Graber (stgraber)]

You can read some more about cgmanager at https://cgmanager.linuxcontainers.org

cgroup-lite's goal was simply to mount the cgroup filesystem in a consistent manner, cgmanager instead is a daemon which handles all cgroup actions through a dbus API and allows for proper user delegation of cgroups including across user namespaces.

Since Ubuntu 14.04, cgmanager is the preferred way of managing cgroups.