juju deploy stuck on Starting container...

[Luis Arias]

This is on juju 0.5+bzr532-1juju5~precise1.

My environments.yaml:

environments:
  local:
    type: local
    data-dir: /tmp/juju-local
    admin-secret: abcdefghijklmnopqrstuvwxyz1234567890
    control-bucket: foo-bucket
    juju-origin: ppa
    default-series: precise

After bootstraping and deploying the mysql charm the debug-log will get stuck on Starting container...

luis@athena:/var/cache/apt-cacher-ng$ juju debug-log
2012-04-30 21:04:56,040 INFO Enabling distributed debug log.
2012-04-30 21:04:56,054 INFO Tailing logs - Ctrl-C to stop.
2012-04-30 21:05:04,722 Machine:0: juju.agents.machine DEBUG: Units changed old:set([]) new:set(['mysql/0'])
2012-04-30 21:05:04,722 Machine:0: juju.agents.machine DEBUG: Starting service unit: mysql/0 ...
2012-04-30 21:05:04,790 Machine:0: unit.deploy DEBUG: Downloading charm cs:precise/mysql-0 to /tmp/juju-local/luis-local/charms
2012-04-30 21:05:04,842 Machine:0: unit.deploy DEBUG: Using <juju.machine.unit.UnitContainerDeployment object at 0x33e4ad0> for mysql/0 in /tmp/juju-local/luis-local
2012-04-30 21:05:04,860 Machine:0: unit.deploy DEBUG: Starting service unit mysql/0...
2012-04-30 21:05:04,861 Machine:0: unit.deploy DEBUG: Creating master container...
2012-04-30 21:19:36,672 Machine:0: unit.deploy DEBUG: Created master container luis-local-0-template
2012-04-30 21:19:36,674 Machine:0: unit.deploy INFO: Creating container mysql-0...
2012-04-30 21:19:39,393 Machine:0: unit.deploy INFO: Container created for mysql/0
2012-04-30 21:19:39,408 Machine:0: unit.deploy DEBUG: Charm extracted into container
2012-04-30 21:19:39,420 Machine:0: unit.deploy DEBUG: Starting container...

The reason seems to be an lxc failure as per the container.log for the mysql-0 unit:

      lxc-start 1335813579.438 DEBUG lxc_conf - allocated pty '/dev/pts/2' (4/5)
      lxc-start 1335813579.438 DEBUG lxc_conf - allocated pty '/dev/pts/5' (6/7)
      lxc-start 1335813579.438 DEBUG lxc_conf - allocated pty '/dev/pts/7' (8/9)
      lxc-start 1335813579.438 DEBUG lxc_conf - allocated pty '/dev/pts/8' (10/11)
      lxc-start 1335813579.438 INFO lxc_conf - tty's configured
      lxc-start 1335813579.438 DEBUG lxc_console - using '/dev/null' as console
      lxc-start 1335813579.438 DEBUG lxc_start - sigchild handler set
      lxc-start 1335813579.438 INFO lxc_start - 'luis-local-mysql-0' is initialized
      lxc-start 1335813579.504 DEBUG lxc_start - Not dropping cap_sys_boot or watching utmp

      lxc-start 1335813579.515 DEBUG lxc_conf - instanciated veth 'veth8r7CN0/veth4k3zZf', index is '19'
      lxc-start 1335813579.515 INFO lxc_conf - opened /var/lib/lxc/luis-local-mysql-0/rootfs.hold as fd 18

      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/' (rootfs)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/sys' (sysfs)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/proc' (proc)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/dev' (devtmpfs)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/dev/pts' (devpts)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/run' (tmpfs)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/' (ext4)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/sys/fs/cgroup' (tmpfs)
      lxc-start 1335813579.520 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/cpuset' (cgroup)
      lxc-start 1335813579.520 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/cpuset'
      lxc-start 1335813579.521 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.521 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/cpuset has flags 0x2
      lxc-start 1335813579.544 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/cpuset//lxc/luis-local-mysql-0'
      lxc-start 1335813579.544 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/cpu' (cgroup)
      lxc-start 1335813579.544 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/cpu'
      lxc-start 1335813579.544 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.544 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/cpu has flags 0x2
      lxc-start 1335813579.576 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/cpu//lxc/luis-local-mysql-0'
      lxc-start 1335813579.576 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/cpuacct' (cgroup)
      lxc-start 1335813579.576 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/cpuacct'
      lxc-start 1335813579.576 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.576 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/cpuacct has flags 0x2
      lxc-start 1335813579.596 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/cpuacct//lxc/luis-local-mysql-0'
      lxc-start 1335813579.596 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/memory' (cgroup)
      lxc-start 1335813579.596 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/memory'
      lxc-start 1335813579.596 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.596 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/memory has flags 0x2
      lxc-start 1335813579.632 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/memory//lxc/luis-local-mysql-0'
      lxc-start 1335813579.632 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/devices' (cgroup)
      lxc-start 1335813579.632 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/devices'
      lxc-start 1335813579.632 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.632 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/devices has flags 0x2
      lxc-start 1335813579.633 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/devices//lxc/luis-local-mysql-0'
      lxc-start 1335813579.633 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/freezer' (cgroup)
      lxc-start 1335813579.633 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/freezer'
      lxc-start 1335813579.633 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.633 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/freezer has flags 0x2
      lxc-start 1335813579.696 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/freezer//lxc/luis-local-mysql-0'
      lxc-start 1335813579.696 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/blkio' (cgroup)
      lxc-start 1335813579.696 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/blkio'
      lxc-start 1335813579.696 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.696 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/blkio has flags 0x2
      lxc-start 1335813579.697 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/blkio//lxc/luis-local-mysql-0'
      lxc-start 1335813579.697 DEBUG lxc_cgroup - checking '/sys/fs/cgroup/perf_event' (cgroup)
      lxc-start 1335813579.697 INFO lxc_cgroup - found cgroup mounted at '/sys/fs/cgroup/perf_event'
      lxc-start 1335813579.697 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.697 DEBUG lxc_cgroup - cgroup /sys/fs/cgroup/perf_event has flags 0x2
      lxc-start 1335813579.697 INFO lxc_cgroup - created cgroup '/sys/fs/cgroup/perf_event//lxc/luis-local-mysql-0'
      lxc-start 1335813579.697 DEBUG lxc_cgroup - checking '/sys/fs/fuse/connections' (fusectl)
      lxc-start 1335813579.697 DEBUG lxc_cgroup - checking '/sys/kernel/debug' (debugfs)
      lxc-start 1335813579.697 DEBUG lxc_cgroup - checking '/sys/kernel/security' (securityfs)
      lxc-start 1335813579.698 DEBUG lxc_cgroup - checking '/run/lock' (tmpfs)
      lxc-start 1335813579.698 DEBUG lxc_cgroup - checking '/run/shm' (tmpfs)
      lxc-start 1335813579.698 DEBUG lxc_cgroup - checking '/proc/sys/fs/binfmt_misc' (binfmt_misc)
      lxc-start 1335813579.698 DEBUG lxc_cgroup - checking '/home/luis/.gvfs' (fuse.gvfs-fuse-daemon)
      lxc-start 1335813579.699 DEBUG lxc_conf - move '(null)' to '29587'
      lxc-start 1335813579.699 INFO lxc_conf - 'luis-local-mysql-0' hostname has been setup
      lxc-start 1335813579.702 DEBUG lxc_conf - mac address '00:16:3e:15:b9:65' on 'eth0' has been setup
      lxc-start 1335813579.705 DEBUG lxc_conf - 'eth0' has been setup
      lxc-start 1335813579.705 INFO lxc_conf - network has been setup
      lxc-start 1335813579.705 DEBUG lxc_conf - mounted '/var/lib/lxc/luis-local-mysql-0/rootfs' on '/usr/lib/lxc/root'
      lxc-start 1335813579.706 DEBUG lxc_conf - mounted 'proc' on '/usr/lib/lxc/root/proc', type 'proc'
      lxc-start 1335813579.706 DEBUG lxc_conf - mounted 'sysfs' on '/usr/lib/lxc/root/sys', type 'sysfs'
      lxc-start 1335813579.706 INFO lxc_conf - mount points have been setup
      lxc-start 1335813579.706 INFO lxc_conf - mount points have been setup
      lxc-start 1335813579.706 DEBUG lxc_cgroup - lxc_cgroup_path_get: called for subsys devices name luis-local-mysql-0

      lxc-start 1335813579.706 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys devices at /

      lxc-start 1335813579.706 DEBUG lxc_cgroup - using cgroup mounted at '/sys/fs/cgroup/devices//lxc'
      lxc-start 1335813579.706 DEBUG lxc_cgroup - lxc_cgroup_path_get: returning /sys/fs/cgroup/devices//lxc/luis-local-mysql-0 for subsystem devices.deny
      lxc-start 1335813579.706 DEBUG lxc_conf - cgroup 'devices.deny' set to 'a'
      lxc-start 1335813579.706 DEBUG lxc_cgroup - lxc_cgroup_path_get: called for subsys devices name luis-local-mysql-0

      lxc-start 1335813579.706 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys devices at /

      lxc-start 1335813579.707 DEBUG lxc_cgroup - using cgroup mounted at '/sys/fs/cgroup/devices//lxc'
      lxc-start 1335813579.707 DEBUG lxc_cgroup - lxc_cgroup_path_get: returning /sys/fs/cgroup/devices//lxc/luis-local-mysql-0 for subsystem devices.allow
      lxc-start 1335813579.707 ERROR lxc_cgroup - write /sys/fs/cgroup/devices//lxc/luis-local-mysql-0/devices.allow : Invalid argument
      lxc-start 1335813579.707 ERROR lxc_conf - failed to setup the cgroups for 'luis-local-mysql-0'
      lxc-start 1335813579.707 ERROR lxc_start - failed to setup the container
      lxc-start 1335813579.707 ERROR lxc_sync - invalid sequence number 1. expected 2
      lxc-start 1335813579.707 ERROR lxc_start - failed to spawn 'luis-local-mysql-0'
      lxc-start 1335813579.707 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.707 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/cpuset//lxc/luis-local-mysql-0

      lxc-start 1335813579.764 DEBUG lxc_cgroup - '/sys/fs/cgroup/cpuset//lxc/luis-local-mysql-0' unlinked
      lxc-start 1335813579.764 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.764 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/cpu//lxc/luis-local-mysql-0

      lxc-start 1335813579.800 DEBUG lxc_cgroup - '/sys/fs/cgroup/cpu//lxc/luis-local-mysql-0' unlinked
      lxc-start 1335813579.800 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.800 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/cpuacct//lxc/luis-local-mysql-0

      lxc-start 1335813579.858 DEBUG lxc_cgroup - '/sys/fs/cgroup/cpuacct//lxc/luis-local-mysql-0' unlinked
      lxc-start 1335813579.858 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.858 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/memory//lxc/luis-local-mysql-0

      lxc-start 1335813579.900 DEBUG lxc_cgroup - '/sys/fs/cgroup/memory//lxc/luis-local-mysql-0' unlinked
      lxc-start 1335813579.900 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.900 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/devices//lxc/luis-local-mysql-0

      lxc-start 1335813579.956 DEBUG lxc_cgroup - '/sys/fs/cgroup/devices//lxc/luis-local-mysql-0' unlinked
      lxc-start 1335813579.956 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813579.956 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/freezer//lxc/luis-local-mysql-0

      lxc-start 1335813580.020 DEBUG lxc_cgroup - '/sys/fs/cgroup/freezer//lxc/luis-local-mysql-0' unlinked
      lxc-start 1335813580.020 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813580.020 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/blkio//lxc/luis-local-mysql-0

      lxc-start 1335813580.057 DEBUG lxc_cgroup - '/sys/fs/cgroup/blkio//lxc/luis-local-mysql-0' unlinked
      lxc-start 1335813580.058 DEBUG lxc_cgroup - get_init_cgroup: found init cgroup for subsys (null) at /

      lxc-start 1335813580.058 DEBUG lxc_cgroup - destroying /sys/fs/cgroup/perf_event//lxc/luis-local-mysql-0

      lxc-start 1335813580.064 DEBUG lxc_cgroup - '/sys/fs/cgroup/perf_event//lxc/luis-local-mysql-0' unlinked

[Luis Arias (kaaloo)]

As per the mailing list thread on this

https://lists.ubuntu.com/archives/juju/2012-May/001525.html

there is an issue in lxc-clone but that causes this and I can now report that it was triggered by the presence of a file named

C:\nppdf32Log\debuglog.txt

at the root of my filesystem. Deleting this file solved this issue.

[Clint Byrum (clint-fewbar)]

Hi Luis! Thanks for reporting this. It appears Serge has already got the fix into quantal. The precise fix should land in precise-proposed soon (today most likely) and hit precise-updates in 7 days as long as the fix is verified to work. This is tracked in bug #993515, which I'm marking this a duplicate of.