Comment 0 for bug 1196295

After running and terminating around 6000 containers overnight, something happened on my box that is affecting every new LXC container I try to start. The DEBUG log file looks like:

      lxc-start 1372615570.399 WARN lxc_start - inherited fd 9
      lxc-start 1372615570.399 INFO lxc_apparmor - aa_enabled set to 1

      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/302' (5/6)
      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/303' (7/8)
      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/304' (10/11)
      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/305' (12/13)
      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/306' (14/15)
      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/307' (16/17)
      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/308' (18/19)
      lxc-start 1372615570.399 DEBUG lxc_conf - allocated pty '/dev/pts/309' (20/21)
      lxc-start 1372615570.399 INFO lxc_conf - tty's configured
      lxc-start 1372615570.399 DEBUG lxc_start - sigchild handler set
      lxc-start 1372615570.399 INFO lxc_start - 'vm-59' is initialized
      lxc-start 1372615570.404 DEBUG lxc_start - Not dropping cap_sys_boot or watching utmp

      lxc-start 1372615570.404 INFO lxc_start - stored saved_nic #0 idx 12392 name vethP59

      lxc-start 1372615570.404 INFO lxc_conf - opened /home/x/vm/vm-59.hold as fd 25

It stops there. In 'ps faux', it looks like:

root 31621 0.0 0.0 25572 1272 ? D 14:06 0:00 \_ lxc-start -n vm-59 -f /tmp/tmp.fG6T6ERZpS -l DEBUG -o /home/x/lxcdebug/vm-59.txt -- /usr/sbin/dropbear -F -E -m

On a successful LXC run (prior to the server getting into this state), this hangs just before:

      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/' (rootfs)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/sys' (sysfs)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/proc' (proc)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/dev' (devtmpfs)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/dev/pts' (devpts)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/run' (tmpfs)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/' (btrfs)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/sys/fs/cgroup' (tmpfs)
      lxc-start 1372394092.208 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/cpuset' (cgroup)
      lxc-start 1372394092.208 INFO lxc_cgroup - [1] found cgroup mounted at '/sys/fs/cgroup/cpuset',opts='rw,relatime,cpuset,clone_children'
      lxc-start 1372394092.208 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

It looks like a resource leak, but I'm not yet sure of what that would be.

If it matters, I SIGKILL my lxc-start processes instead of using lxc-stop. Could that have any negative implications?

Oh, and cgroups had almost 6000 entries for VMs that are long dead (I'm guessing it's due to my SIGKILL). I've run cgclear and my /sys/fs/cgroup/*/ dirs are now totally empty, but the new containers still hang.