Comment 0 for bug 1214085

Using lxc with a btrfs filesystem can lead to system corruption and containers which are unable to boot when many containers are created and started simulatenously.

Packages involved:

    - btrfs-tools
    - ubuntu-lxc/daily from http://ppa.launchpad.net/ubuntu-lxc/daily/ubuntu

`lsb_release -rd`:

    ubuntu@ip-10-91-30-131:~$ lsb_release -rd
    Description: Ubuntu 12.04.2 LTS
    Release: 12.04

Here is the relevant discussion thread: http://<email address hidden>/msg05472.html

And here is a key except regarding reproducing the problem:

The general system state is something like:
N containers already running happily
Launch N+ more containers in rapid succession (in parallell, not serially).

Here is a test script which closely reflects what my application is actually doing. It slowly launches 10 containers and then uses "&" to rapidly fork an additional 10 clone/start operations. I have it doing 2 cycles of this, and running the script many times (2-6 times) eventually triggers the problem.

test.sh:
#!/usr/bin/env bash

prefix=$1

test -z "${prefix}" && echo 'error: missing required parameter: prefix' 1>&2 && exit 1

path=/mnt

sudo lxc-destroy -n c1 2>/dev/null
sudo lxc-create -t ubuntu -B btrfs -n c1

for i in `seq 1 10`; do
    sudo lxc-clone -s -B btrfs -P $path -o c1 -n $prefix$i
    sudo lxc-start -d -n $prefix$i
done
for i in `seq 11 20`; do
    echo $(sudo lxc-clone -s -B btrfs -P $path -o c1 -n $prefix$i; sudo lxc-start -d -n $prefix$i) &
done

sleep 10

# Create even more.
for i in `seq 21 30`; do
    sudo lxc-clone -s -B btrfs -P $path -o c1 -n $prefix$i
    sudo lxc-start -d -n $prefix$i
done
for i in `seq 31 40`; do
    echo $(sudo lxc-clone -s -B btrfs -P $path -o c1 -n $prefix$i; sudo lxc-start -d -n $prefix$i) &
done

stop.sh:
#!/usr/bin/env bash

prefix=$1

test -z "${prefix}" && echo 'error: missing required parameter: prefix' 1>&2 && exit 1

sudo lxc-destroy -n c1;

for i in `seq 1 40`; do
    echo $(sudo lxc-stop -k -n $prefix$i; sudo lxc-destroy -n $prefix$i) &
done

bash ./test.sh x
bash ./test.sh y
bash ./test.sh z

If it doesn't manifest at first, try starting/stopping/destroying varying quantities of containers for several cycles. Eventually I consistently end up not ever getting ip addresses or being able to even successfully start a container:

x1 RUNNING - - NO
x10 RUNNING - - NO
x11 RUNNING - - NO
x12 RUNNING - - NO
x13 RUNNING - - NO
x14 RUNNING - - NO
x15 RUNNING - - NO
x16 RUNNING - - NO
x17 RUNNING - - NO
x18 RUNNING - - NO
x19 RUNNING - - NO
x2 RUNNING - - NO
x20 RUNNING - - NO
x21 RUNNING - - NO
x22 RUNNING - - NO
x23 RUNNING - - NO
x24 RUNNING - - NO
x25 RUNNING - - NO
x26 RUNNING - - NO
x27 RUNNING - - NO
x28 RUNNING - - NO
x29 RUNNING - - NO
x3 RUNNING - - NO
x30 RUNNING - - NO
x31 RUNNING - - NO
x32 RUNNING - - NO
x33 RUNNING - - NO
x34 RUNNING - - NO
x35 RUNNING - - NO
x36 RUNNING - - NO
x37 RUNNING - - NO
x38 RUNNING - - NO
x39 RUNNING - - NO
x4 RUNNING - - NO
x40 RUNNING - - NO
x5 RUNNING - - NO
x6 RUNNING - - NO
x7 RUNNING - - NO
x8 RUNNING - - NO
x9 RUNNING - - NO

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: linux-image-3.2.0-40-virtual 3.2.0-40.64
ProcVersionSignature: Ubuntu 3.2.0-40.64-virtual 3.2.40
Uname: Linux 3.2.0-40-virtual x86_64
AcpiTables:

AlsaDevices:
 total 0
 crw-rw---T 1 root audio 116, 1 Aug 14 20:40 seq
 crw-rw---T 1 root audio 116, 33 Aug 14 20:40 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.0.1-0ubuntu17.3
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found.
Date: Mon Aug 19 18:20:19 2013
Ec2AMI: ami-d0f89fb9
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-east-1a
Ec2InstanceType: m1.large
Ec2Kernel: aki-88aa75e1
Ec2Ramdisk: unavailable
Lspci:

Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99
MarkForUpload: True
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
RelatedPackageVersions:
 linux-restricted-modules-3.2.0-40-virtual N/A
 linux-backports-modules-3.2.0-40-virtual N/A
 linux-firmware 1.79.4
RfKill: Error: [Errno 2] No such file or directory
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
WifiSyslog: