OpenStack Compute (nova)

image to local block device mapping doesnt work but accepted by the API

Bug #1377958 reported by Balazs Gibizer on 2014-10-06

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	Low	Balazs Gibizer	OpenStack Compute (nova) 2015.1.0 "kilo"

Bug Description

When I try to boot an instance with the boot image specified as BDM the API accepts the request but the instance ends up in ERROR state due to an exception in the nova-compute. Furter investigation shows that image to local mapping is not supported in the --block-device parameter at all but the API does not validate and reject this kind of requests.

stack@stack:~/devstack$ nova flavor-create install-test 100 1024 0 1
+-----+--------------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+-----+--------------+-----------+------+-----------+------+-------+-------------+-----------+
| 100 | install-test | 1024 | 0 | 0 | | 1 | 1.0 | True |
+-----+--------------+-----------+------+-----------+------+-------+-------------+-----------+
stack@stack:~/devstack$ glance image-create --file ../mini.iso --name ubuntu-mini.iso --is-public=True --disk-format iso --container-format bare+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | a2502844750ecb6477d8fb4ff6b9aaf8 |
| container_format | bare |
| created_at | 2014-10-06T13:45:48 |
| deleted | False |
| deleted_at | None |
| disk_format | iso |
| id | e3bb2e81-e5ed-43a0-9cc5-62ff9de97c37 |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | ubuntu-mini.iso |
| owner | 8acb24eaeaee44f481aa7c73f179a4da |
| protected | False |
| size | 32505856 |
| status | active |
| updated_at | 2014-10-06T13:45:48 |
| virtual_size | None |
+------------------+--------------------------------------+
stack@stack:~/devstack$ nova boot --flavor 100 --block-device source=image,id=e3bb2e81-e5ed-43a0-9cc5-62ff9de97c37,type=cdrom,dest=local,bootindex=0,shutdown=delete vm4
+--------------------------------------+-------------------------------------------------+
| Property | Value |
+--------------------------------------+-------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | PHD2yuKTVbDA |
| config_drive | |
| created | 2014-10-06T13:46:01Z |
| flavor | install-test (100) |
| hostId | |
| id | bbc3a863-a62f-4676-9100-a34a93b191cc |
| image | Attempt to boot from volume - no image supplied |
| key_name | - |
| metadata | {} |
| name | vm4 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | 8acb24eaeaee44f481aa7c73f179a4da |
| updated | 2014-10-06T13:46:01Z |
| user_id | 2957e97d89fd4aeb80e314653ea34663 |
+--------------------------------------+-------------------------------------------------+
stack@stack:~/devstack$ nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+------------------+
| bbc3a863-a62f-4676-9100-a34a93b191cc | vm4 | ERROR | - | NOSTATE | private=10.0.0.2 |
+--------------------------------------+------+--------+------------+-------------+------------------+
stack@stack:~/devstack$
stack@stack:~/devstack$ nova show vm4
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | stack |
| OS-EXT-SRV-ATTR:hypervisor_hostname | stack |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2014-10-06T13:46:01Z |
| fault | {"message": "No valid host was found. ", "code": 500, "details": " File \"/opt/stack/nova/nova/conductor/manager.py\", line 614, in build_instances |
| | request_spec, filter_properties) |
| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 49, in select_destinations |
| | context, request_spec, filter_properties) |
| | File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 35, in __run_method |
| | return getattr(self.instance, __name)(*args, **kwargs) |
| | File \"/opt/stack/nova/nova/scheduler/client/query.py\", line 34, in select_destinations |
| | context, request_spec, filter_properties) |
| | File \"/opt/stack/nova/nova/scheduler/rpcapi.py\", line 108, in select_destinations |
| | request_spec=request_spec, filter_properties=filter_properties) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py\", line 152, in call |
| | retry=self.retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo/messaging/transport.py\", line 90, in _send |
| | timeout=timeout, retry=retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py\", line 408, in send |
| | retry=retry) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py\", line 399, in _send |
| | raise result |
| | ", "created": "2014-10-06T13:46:05Z"} |
| flavor | install-test (100) |
| hostId | 9c365e0da6ac63e8559079a640a533242d5252a56bf76ed160011e25 |
| id | bbc3a863-a62f-4676-9100-a34a93b191cc |
| image | Attempt to boot from volume - no image supplied |
| key_name | - |
| metadata | {} |
| name | vm4 |
| os-extended-volumes:volumes_attached | [] |
| private network | 10.0.0.2 |
| security_groups | default |
| status | ERROR |
| tenant_id | 8acb24eaeaee44f481aa7c73f179a4da |
| updated | 2014-10-06T13:46:05Z |
| user_id | 2957e97d89fd4aeb80e314653ea34663 |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
stack@stack:~/devstack$

The following exception is visible in the nova-cpu.log

2014-10-06 15:46:04.712 ERROR nova.compute.manager [req-a8ccaca2-b3e8-492c-8f82-20a72ce1d7ff admin demo] [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] Instance failed to spawn
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] Traceback (most recent call last):
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/opt/stack/nova/nova/compute/manager.py", line 2231, in _build_resources
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] yield resources
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/opt/stack/nova/nova/compute/manager.py", line 2101, in _build_and_run_instance
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] block_device_info=block_device_info)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2600, in spawn
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] block_device_info, disk_info=disk_info)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4393, in _create_domain_and_network
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] power_on=power_on)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4317, in _create_domain
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] LOG.error(err)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/opt/stack/nova/nova/openstack/common/excutils.py", line 82, in __exit__
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] six.reraise(self.type_, self.value, self.tb)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4308, in _create_domain
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] domain.createWithFlags(launch_flags)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] rv = execute(f, *args, **kwargs)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] six.reraise(c, e, tb)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] rv = meth(*args, **kwargs)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] libvirtError: Failed to open file '/opt/stack/data/nova/instances/bbc3a863-a62f-4676-9100-a34a93b191cc/disk': No such file or directory
2014-10-06 15:46:04.712 TRACE nova.compute.manager [instance: bbc3a863-a62f-4676-9100-a34a93b191cc]
2014-10-06 15:46:04.899 AUDIT nova.compute.manager [req-a8ccaca2-b3e8-492c-8f82-20a72ce1d7ff admin demo] [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] Terminating instance
2014-10-06 15:46:04.907 INFO nova.virt.libvirt.driver [-] [instance: bbc3a863-a62f-4676-9100-a34a93b191cc] Instance destroyed successfully.

Here is the related section of the domain xml:
  <devices>
    <emulator>/usr/bin/kvm-spice</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/opt/stack/data/nova/instances/bbc3a863-a62f-4676-9100-a34a93b191cc/disk'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/opt/stack/data/nova/instances/bbc3a863-a62f-4676-9100-a34a93b191cc/disk.config'/>
      <target dev='hdd' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='1' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>

Also if thea boot images is specified with the --image parameter and an extra image to local mapping is specified with the --block-device parameter then the instance is booted from the boot image but the secondary image is not attach.

stack@stack:~/devstack$ nova boot --flavor 100 --image b30661ed-f176-4b7c-997f-32ca724cbaad --block-device source=image,id=e3bb2e81-e5ed-43a0-9cc5-62ff9de97c37,type=cdrom,dest=local,bootindex=1,shutdown=delete vm4
+--------------------------------------+----------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000002 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | Mir6yrXKehJP |
| config_drive | |
| created | 2014-10-06T13:57:03Z |
| flavor | install-test (100) |
| hostId | |
| id | 1f7ab5b2-6388-4b0f-b6cd-8ddc43295d43 |
| image | cirros-0.3.2-x86_64-uec (b30661ed-f176-4b7c-997f-32ca724cbaad) |
| key_name | - |
| metadata | {} |
| name | vm4 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | 8acb24eaeaee44f481aa7c73f179a4da |
| updated | 2014-10-06T13:57:03Z |
| user_id | 2957e97d89fd4aeb80e314653ea34663 |
+--------------------------------------+----------------------------------------------------------------+
stack@stack:~/devstack$ nova show
usage: nova show [--minimal] <server>
error: too few arguments
Try 'nova help show' for more information.
stack@stack:~/devstack$ nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+------------------+
| 1f7ab5b2-6388-4b0f-b6cd-8ddc43295d43 | vm4 | ACTIVE | - | Running | private=10.0.0.2 |
+--------------------------------------+------+--------+------------+-------------+------------------+

Here is the related part of the domain xml:
  <devices>
    <emulator>/usr/bin/kvm-spice</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/opt/stack/data/nova/instances/1f7ab5b2-6388-4b0f-b6cd-8ddc43295d43/disk'/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/opt/stack/data/nova/instances/1f7ab5b2-6388-4b0f-b6cd-8ddc43295d43/disk.config'/>
      <target dev='hdd' bus='ide'/>
      <readonly/>
      <alias name='ide0-1-1'/>
      <address type='drive' controller='0' bus='1' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>

Tags:

Balazs Gibizer (balazs-gibizer) on 2014-10-06

Changed in nova:
assignee:	nobody → Balazs Gibizer (balazs-gibizer)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-10-06: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/126303

Changed in nova:
status:	New → In Progress

melanie witt (melwitt) on 2015-01-27

tags:	added: volumes
Changed in nova:
importance:	Undecided → Low

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-02-02: Fix merged to nova (master)

Reviewed: https://review.openstack.org/126303
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=67bcff461497e674ccdf5d335d983fe9a16639fa
Submitter: Jenkins
Branch: master

commit 67bcff461497e674ccdf5d335d983fe9a16639fa
Author: Balazs Gibizer <email address hidden>
Date: Mon Oct 6 15:38:46 2014 +0200

Reject unsupported image to local BDM

    Specifying image to local BDM at nova boot is not supported.
    This patch adds a validation to the API to reject this not supported
    mapping.

Closes-bug: #1377958
Change-Id: I0c38134e2aacca83d98ed0b341f15c480afe14f1

Changed in nova:
status:	In Progress → Fix Committed

Thierry Carrez (ttx) on 2015-02-05

Changed in nova:
milestone:	none → kilo-2
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2015-04-30

Changed in nova:
milestone:	kilo-2 → 2015.1.0

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.