Comment 7 for bug 1538135

I've confirmed that a rebuilt ci node from a pristine openstack-ci setup produces the same odd behavior. The disk image create works when run under the stock ubuntu account but fails when running under the nodepool account. Confusing.

The ci image is built with the user-data file: http://paste.openstack.org/show/485097/

Launched with the command:

nova boot --flavor m1.large --image "ubuntu-14.04.3-server-cloudimg-amd64-disk1" --key-name mykey --security-groups default,ssh --user-data ./openstackci-base ci

Then follow the steps for setting up the ci node from http://docs.openstack.org/infra/openstackci/third_party_ci.html#configure-masterless-puppet

sudo su -
cp /etc/puppet/modules/openstackci/contrib/hiera.yaml /etc/puppet

cp /etc/puppet/modules/openstackci/contrib/single_node_ci_site.pp /etc/puppet/manifests/site.pp
cp /etc/puppet/modules/openstackci/contrib/single_node_ci_data.yaml /etc/puppet/environments/common.yaml

Edit common.yaml with the local config. Reference the project config:

https://gitlab.uabgrid.uab.edu/jpr/project-config-ci-test

Run puppet apply: http://docs.openstack.org/infra/openstackci/third_party_ci.html#run-masterless-puppet

Move the cache to /mnt (80GB) for more space:

sudo su -
cd /
rsync -a /opt /mnt/opt
mv opt opt-orig
ln -s /mnt/opt
mkdir /mnt/ubuntu
chown ubuntu.ubuntu /mnt/ubuntu
mkdir /mnt/ubuntu/dib_tmp

Start zuul.

Then test the build of a simple image:

As the Ubuntu user it succeeds:

TMPDIR=/mnt/ubuntu/dib_tmp DIB_IMAGE_CACHE=/mnt/ubuntu/dib_cache disk-image-create -x -t qcow2 --no-tmpfs -o ubuntuvm.qcow2 ubuntu vm

As the nodepool user it fails, breaking /dev:

TMPDIR=/opt/dib_tmp DIB_IMAGE_CACHE=/opt/dib_cache disk-image-create -x -t qcow2 --no-tmpfs -o ubuntuvm.qcow2 ubuntu vm