init: lxc-instance main process terminated with status 255

[Will Bryant]

We have a trusty host which has 9 trusty guests running under LXC. On the most recent reboot, 3 of the LXC processes crashed:

[ 10.912793] init: lxc-instance (nz-prod-bdbbackup-akl3) main process (3175) terminated with status 255
[ 11.031717] init: lxc-instance (au-prod-db-akl3) main process (2974) terminated with status 255
[ 11.079716] init: lxc-instance (au-test-backup-akl3) main process (3042) terminated with status 255

We were able to manually start them afterwards, but it's not clear why they crashed on boot.

Each of those containers has identical software configurations to at least one of the 6 containers that didn't crash on boot, so it appears to be random.

We're running the 3.13.0-68-generic kernel and LXC 1.0.7-0ubuntu0.10. All other packages are current.

Here is a sample config:

# Template used to create this container: /usr/share/lxc/templates/lxc-ubuntu
# Parameters passed to the template: -r trusty --user <myuser> --auth-key /home/<myuser>/.ssh/authorized_keys
# For additional config options, please look at lxc.container.conf(5)

# Common configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf

# Container specific configuration
lxc.rootfs = /var/lib/lxc/au-test-backup-akl3/rootfs
lxc.mount = /var/lib/lxc/au-test-backup-akl3/fstab
lxc.utsname = au-test-backup-akl3
lxc.arch = amd64

# Network configuration
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = 00:16:3e:9e:7c:41
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br1
lxc.network.hwaddr = 00:16:3f:18:88:d9
lxc.start.auto = 1

And fstab:

/var/backups/au-test-backup-akl3 var/backups bind defaults,bind 0 0

[Will Bryant (willbryant)]

One out of 9 LXC containers crashed when we rebooted one of the sister servers (which are configured just like the first server), so it's somewhat reproducible.

[Stéphane Graber (stgraber)]

trusty is EOL and upstart is unsupported at this time.