Uncached instance builds fail with non-zero root disk sizes

[David Wittman]

A recent change in the Folsom code has resulted in issues building new instances with a non-zero root disk size. It is currently trying to compare the flavor size to the size of the base image before the base image is actually pulled down. As a result, builds fail with the following error in nova-compute.log:

2013-05-21 17:44:18 ERROR nova.compute.manager [req-a1b0cf46-aad0-48e4-8474-ea350de678d5 5dbf5c9e5c4740638fa563a942475b12 180d2f50f1384c65b456da82ed90429d] [instance: 67b18b5c-755a-4484-a75a-9ce3629eb0c5] Build error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 503, in _run_instance\n injected_files, admin_password)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 756, in _spawn\n block_device_info)\n', '
File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', '
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n self.gen.next()\n', '
File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', '
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1091, in spawn\n admin_pass=admin_password)\n', '
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1357, in _create_image\n project_id=instance[\'project_id\'])\n', '
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 131, in cache\n *args, **kwargs)\n', '
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 181, in create_image\n if size and size < disk.get_disk_size(base):\n', '
File "/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py", line 115, in get_disk_size\n size = images.qemu_img_info(path)[\'virtual size\']\n', '
File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 50, in qemu_img_info\n \'qemu-img\', \'info\', path)\n', '
File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 210, in execute\n cmd=\' \'.join(cmd))\n', 'ProcessExecutionError: Unexpected error while running command.\nCommand: env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/_base/ef98e9c3ea74090467df43c56068756bae1b8a4e\nExit code: 1\nStdout: \'\'\nStderr: "qemu-img: Could not open \'/var/lib/nova/instances/_base/ef98e9c3ea74090467df43c56068756bae1b8a4e\': No such file or directory\\n"\n']

System Info:
# lsb_release -rd
Description: Ubuntu 12.04.2 LTS
Release: 12.04

# apt-cache policy nova-compute
nova-compute:
  Installed: 2012.2.3-0ubuntu2.1~cloud0
  Candidate: 2012.2.3-0ubuntu2.1~cloud0
  Version table:
 *** 2012.2.3-0ubuntu2.1~cloud0 0
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu/ precise-proposed/folsom/main amd64 Packages
        100 /var/lib/dpkg/status
     2012.1.3+stable-20130423-e52e6912-0ubuntu1.1 0
        500 http://mirror.rackspace.com/ubuntu/ precise-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ precise-security/main amd64 Packages
     2012.1-0ubuntu2 0
        500 http://mirror.rackspace.com/ubuntu/ precise/main amd64 Packages

This issue does NOT occur if the image being used is already cached in the instances_path on the compute host.

Here is the commit where the bug was introduced:

https://github.com/openstack/nova/commit/6740c4141ea1152529b82cbf6e5b808eaba912e7

It appears to be a merge issue with Folsom, as the prepare_template line appears above the conditional in Grizzly.

[David Wittman (david-wittman)]

This may be invalid for Ubuntu. I intended to open it in OpenStack/nova originally.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/29982

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jay Pipes (jaypipes)]

We are experiencing this exact same issue, and it is a blocker. Folsom point release packages for us.

[WcMinor (risingofthemoon)]

I can confirm this bug is affecting my system, reverting the https://github.com/openstack/nova/commit/6740c4141ea1152529b82cbf6e5b808eaba912e7 commit solves it.