LXC instances fail with mount error, attempting to mount wrong partition

[Adam Gandelman]

Using the libvirt driver and libvirt_type=lxc on Grizzly, LXC -backed instances fail to start. Nova is failing to mount the nbd block device at the containers rootfs, because it does not address the partition (tested using an ubuntu cloud image and cirros).

2013-07-26 23:09:16.911 ERROR nova.virt.disk.api [req-7663ecc9-91ae-4fe0-9149-a59aed47c06b 1ac54004c9834e7dad0431fff3206d7d 64defdc3bf6247798c201914db714e7b] Failed to mount container filesystem '<nova.virt.disk.api._DiskImage object at 0x2d43f90>' on '/var/lib/nova/instances/0c8a3e06-2938-4423-8da3-1d9abbcbbfe9/rootfs':
--
Failed to mount filesystem: Unexpected error while running command.
Command: mount /dev/nbd7 /var/lib/nova/instances/0c8a3e06-2938-4423-8da3-1d9abbcbbfe9/rootfs
Exit code: 12
Stdout: ''
Stderr: "NTFS signature is missing.\nFailed to mount '/dev/nbd7': Invalid argument\nThe device '/dev/nbd7' doesn't seem to have a valid NTFS.\nMaybe the wrong device is used? Or the whole disk instead of a\npartition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?\n"
2013-07-26 23:09:16.912 ERROR nova.compute.manager [req-7663ecc9-91ae-4fe0-9149-a59aed47c06b 1ac54004c9834e7dad0431fff3206d7d 64defdc3bf6247798c201914db714e7b] [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Instance failed to spawn
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Traceback (most recent call last):
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1103, in _spawn
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] block_device_info)
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1528, in spawn
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] block_device_info)
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2444, in _create_domain_and_network
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] domain = self._create_domain(xml, instance=instance)
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2401, in _create_domain
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] use_cow=CONF.use_cow_images)
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] File "/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py", line 322, in setup_container
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] raise exception.NovaException(img.errors)
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] NovaException:
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] --
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Failed to mount filesystem: Unexpected error while running command.
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Command: mount /dev/nbd7 /var/lib/nova/instances/0c8a3e06-2938-4423-8da3-1d9abbcbbfe9/rootfs
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Exit code: 12
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Stdout: ''
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Stderr: "NTFS signature is missing.\nFailed to mount '/dev/nbd7': Invalid argument\nThe device '/dev/nbd7' doesn't seem to have a valid NTFS.\nMaybe the wrong device is used? Or the whole disk instead of a\npartition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?\n"
2013-07-26 23:09:16.912 31412 TRACE nova.compute.manager [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9]
2013-07-26 23:09:17.210 AUDIT nova.compute.manager [req-7663ecc9-91ae-4fe0-9149-a59aed47c06b 1ac54004c9834e7dad0431fff3206d7d 64defdc3bf6247798c201914db714e7b] [instance: 0c8a3e06-2938-4423-8da3-1d9abbcbbfe9] Terminating instance

http://paste.ubuntu.com/5916532/plain/

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Adam Gandelman (gandelman-a)]

The problem is not a bug but an issue with the image that was being booted.

Needed to properly publish the image to glance:

glance image-create --disk-format raw --container-format bare --name "Ubuntu LXC" --property hypervisor_type=lxc --is-public true < precise-server-cloudimg-amd64.img