XenAPI: Boot from volume without image_ref broken

Bug #1294069 reported by Bob Ball
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
John Garbutt
devstack
Fix Released
High
Bob Ball

Bug Description

https://review.openstack.org/#/c/78194/ changed tempest to clear image_ref for some BFV tests - in particular the test_volume_boot_pattern

This now results in a "KeyError: 'disk_format'" exception from Nova when using the XenAPI driver.

http://paste.openstack.org/show/73733/ is a nicer format of the below - but might disappear!

2014-03-18 11:20:07.475 ERROR nova.compute.manager [req-82096fe0-921a-4bc1-9c41-d0aafad4c923 TestVolumeBootPattern-581093620 TestVolumeBootPattern-1800543246] [instance: 2b047f24-675c-4921-8cf3-85584097f106] Error: 'disk_format'
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] Traceback (most recent call last):
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 1306, in _build_instance
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] set_access_ip=set_access_ip)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 394, in decorated_function
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] return function(self, context, *args, **kwargs)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 1708, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] LOG.exception(_('Instance failed to spawn'), instance=instance)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] six.reraise(self.type_, self.value, self.tb)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/compute/manager.py", line 1705, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] block_device_info)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/driver.py", line 236, in spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] admin_password, network_info, block_device_info)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 357, in spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] network_info, block_device_info, name_label, rescue)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 526, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] undo_mgr.rollback_and_reraise(msg=msg, instance=instance)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/utils.py", line 812, in rollback_and_reraise
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] self._rollback()
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] six.reraise(self.type_, self.value, self.tb)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 501, in _spawn
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] disk_image_type = determine_disk_image_type_step(undo_mgr)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 146, in inner
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] rv = f(*args, **kwargs)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vmops.py", line 414, in determine_disk_image_type_step
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] return vm_utils.determine_disk_image_type(image_meta)
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] File "/opt/stack/nova/nova/virt/xenapi/vm_utils.py", line 1647, in determine_disk_image_type
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] disk_format = image_meta['disk_format']
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106] KeyError: 'disk_format'
2014-03-18 11:20:07.475 TRACE nova.compute.manager [instance: 2b047f24-675c-4921-8cf3-85584097f106]

I've confirmed that running without https://review.openstack.org/#/c/78194/ passes the tests (although there is a race condition, which is why test_volume_boot_pattern is disabled in the XenServer CI system) but will always fail with https://review.openstack.org/#/c/78194/ applied.

Tags: xenserver
Revision history for this message
John Garbutt (johngarbutt) wrote :

Not sure this ever worked, so its not really blocking RC1

Changed in nova:
status: New → Triaged
importance: Undecided → High
Bob Ball (bob-ball)
description: updated
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/81252

Changed in nova:
status: Triaged → In Progress
Bob Ball (bob-ball)
affects: nova → devstack
Changed in nova:
assignee: nobody → John Garbutt (johngarbutt)
importance: Undecided → High
status: New → In Progress
Changed in devstack:
assignee: John Garbutt (johngarbutt) → Bob Ball (bob-ball)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/81252
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1b2570877846598d3545a9c9f31680f2a995491f
Submitter: Jenkins
Branch: master

commit 1b2570877846598d3545a9c9f31680f2a995491f
Author: John Garbutt <email address hidden>
Date: Tue Mar 18 12:53:12 2014 +0000

    xenapi: boot from volume without image_ref

    Currently xenapi does not deal well with image_meta that does not
    contain "disk_format". This happens when we get image_meta from an
    associated volume, rather than from glance.

    Change-Id: Id673158442fde27e8d468ca412c9bd557a886e6b
    Closes-Bug: #1294069

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to devstack (master)

Reviewed: https://review.openstack.org/81497
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=f1a2dbffe8ba369b0a8a125e975864a5d88f3e87
Submitter: Jenkins
Branch: master

commit f1a2dbffe8ba369b0a8a125e975864a5d88f3e87
Author: Bob Ball <email address hidden>
Date: Wed Mar 19 11:08:54 2014 +0000

    XenAPI: Cirros images must always boot as PV.

    The default for VHD disk-types is PV, which is why booting from a
    server works. However, creating a volume from the image needs to
    pass this parameter on to the volume. Note that
    Id673158442fde27e8d468ca412c9bd557a886e6b is also required to fix
    bug 1294069

    Change-Id: I7ea1d85d6082787ac4551f78300a04bf59074261
    Partial-Bug: 1294069

Changed in devstack:
status: In Progress → Fix Released
Changed in nova:
milestone: none → icehouse-rc1
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.