XenAPI: Boot from volume without image_ref broken

[Bob Ball]

https://review.openstack.org/#/c/78194/ changed tempest to clear image_ref for some BFV tests - in particular the test_volume_boot_pattern

This now results in a "KeyError: 'disk_format'" exception from Nova when using the XenAPI driver.

http://paste.openstack.org/show/73733/ is a nicer format of the below - but might disappear!

2014-03-18 11:20:07.475 ERROR nova.compute.manager [req-82096fe0-921a-4bc1-9c41-d0aafad4c923 TestVolumeBootPattern-581093620 TestVolumeBootPattern-1800543246] [instance: 2b047f24-675c-4921-8cf3-85584097f106] Error: 'disk_format'
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] Traceback (most recent call last):
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 1306, in _build_instance
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] set_access_ip=set_access_ip)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 394, in decorated_function
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] return function(self, context, *args, **kwargs)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 1708, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] LOG.exception(_('Instance failed to spawn'), instance=instance)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] six.reraise(self.type_, self.value, self.tb)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 1705, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] block_device_info)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/driver.py", line 236, in spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] admin_password, network_info, block_device_info)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 357, in spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] network_info, block_device_info, name_label, rescue)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 526, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] undo_mgr.rollback_and_reraise(msg=msg, instance=instance)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/utils.py", line 812, in rollback_and_reraise
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] self._rollback()
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] six.reraise(self.type_, self.value, self.tb)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 501, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] disk_image_type = determine_disk_image_type_step(undo_mgr)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 146, in inner
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] rv = f(*args, **kwargs)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 414, in determine_disk_image_type_step
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] return vm_utils.determine_disk_image_type(image_meta)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vm_utils.py", line 1647, in determine_disk_image_type
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] disk_format = image_meta['disk_format']
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] KeyError: 'disk_format'
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106]

I've confirmed that running without https://review.openstack.org/#/c/78194/ passes the tests (although there is a race condition, which is why test_volume_boot_pattern is disabled in the XenServer CI system) but will always fail with https://review.openstack.org/#/c/78194/ applied.

[John Garbutt (johngarbutt)]

Not sure this ever worked, so its not really blocking RC1

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/81252

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/81252
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1b2570877846598d3545a9c9f31680f2a995491f
Submitter: Jenkins
Branch: master

commit 1b2570877846598d3545a9c9f31680f2a995491f
Author: John Garbutt <email address hidden>
Date: Tue Mar 18 12:53:12 2014 +0000

    xenapi: boot from volume without image_ref

    Currently xenapi does not deal well with image_meta that does not
    contain "disk_format". This happens when we get image_meta from an
    associated volume, rather than from glance.

    Change-Id: Id673158442fde27e8d468ca412c9bd557a886e6b
    Closes-Bug: #1294069

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/81497
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=f1a2dbffe8ba369b0a8a125e975864a5d88f3e87
Submitter: Jenkins
Branch: master

commit f1a2dbffe8ba369b0a8a125e975864a5d88f3e87
Author: Bob Ball <email address hidden>
Date: Wed Mar 19 11:08:54 2014 +0000

    XenAPI: Cirros images must always boot as PV.

    The default for VHD disk-types is PV, which is why booting from a
    server works. However, creating a volume from the image needs to
    pass this parameter on to the volume. Note that
    Id673158442fde27e8d468ca412c9bd557a886e6b is also required to fix
    bug 1294069

    Change-Id: I7ea1d85d6082787ac4551f78300a04bf59074261
    Partial-Bug: 1294069