OpenStack Compute (nova)

When cross_az_attach set to false and creating VM with bootable volumes created before, InvalidBDM error raised.

Bug #1648324 reported by John Haan
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
In Progress
Undecided
John Haan

Bug Description

Description
===========

When cross_az_attach option is set to False and you try to boot an instance with bootable volume created before, InvalidBDM error raised.

Steps to reproduce
==================

1. Configure multiple AZs in Nova and Cinder.
  - for Nova : AZ1 and AZ2
  - for Cinder : AZ1 and AZ2

2. cross_az_attach set to false in nova.conf
[cinder]
cross_az_attach = False

3. Restart nova-compute service

4. Create volume from glance image
$ cinder create 10 --display-name test-bootable-volume --image-id {image-id} --availability-zone AZ1

5. Create instance from the volume created
$ nova boot --flavor m1.small --boot-volume 7c15ae47-12bd-41e7-822d-37e3619082be --nic net-id=1493b02c-c257-45db-84ea-789f87ed7924 --availability-zone AZ1 demo-instance

Expected result
===============

Instance is booted successfully in AZ1 zone.

Actual result
=============

Instance failed to boot and raise InvalidBDM Error

nova command output says:

ERROR (BadRequest): Block Device Mapping is Invalid: failed to get volume 7c15ae47-12bd-41e7-822d-37e3619082be . (HTTP 400) (Request-ID: req-8c36c5d1-b18d-4db5-a86f-db33c621e4e4)

nova-api.log says:

2016-12-02 20:02:48.724 150051 DEBUG cinderclient.client [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53 579e9f03f761431dbe194c8cf2baeed9 - default default] GET call to cinderv2 for http://10.161.32.13:8776/v2/579e9f03f761431dbe194c8cf2baeed9/volumes/7c15ae47-12bd-41e7-822d-37e3619082be used request id req-4f4170af-bc29-4b92-9c7d-c9a5ce457aa8 _log_request_id /usr/local/lib/python2.7/dist-packages/cinderclient/client.py:95
2016-12-02 20:02:48.734 150051 DEBUG nova.quota [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53 579e9f03f761431dbe194c8cf2baeed9 - default default] Rolled back reservations [u'ff856b75-4457-4b23-abe6-cf5981cacab8', u'74271baa-3385-4a03-bb99-86d05e30846c', u'09c2f52f-8f1e-46bf-9876-96f5ef1cf553'] rollback /usr/local/lib/python2.7/dist-packages/nova/quota.py:1427
2016-12-02 20:02:48.734 150051 INFO nova.api.openstack.wsgi [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53 579e9f03f761431dbe194c8cf2baeed9 - default default] HTTP exception thrown: Block Device Mapping is Invalid: failed to get volume 7c15ae47-12bd-41e7-822d-37e3619082be.
2016-12-02 20:02:48.735 150051 DEBUG nova.api.openstack.wsgi [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53 579e9f03f761431dbe194c8cf2baeed9 - default default] Returning 400 to user: Block Device Mapping is Invalid: failed to get volume 7c15ae47-12bd-41e7-822d-37e3619082be. __call__ /usr/local/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1044
2016-12-02 20:02:48.736 150051 INFO nova.osapi_compute.wsgi.server [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53 579e9f03f761431dbe194c8cf2baeed9 - default default] 10.161.32.13 "POST /v2.1/579e9f03f761431dbe194c8cf2baeed9/os-volumes_boot HTTP/1.1" status: 400 len: 494 time: 0.4048491

Environment
===========

Ubuntu 16.04, qemu, Cinder ceph
Nova version: stable/newton
Two AZs configured in Nova and Cinder

John Haan (yongiman)
Changed in nova:
assignee: nobody → John Haan (yongiman)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/409680

Changed in nova:
status: New → In Progress
Revision history for this message
Miguel Alejandro Cantu (miguel-cantu) wrote :
Download full text (4.3 KiB)

I'm also seeing this in my environment, which has cross_az_attach=False.

Nova AZ:
root@cantu-rpco-master01-utility-container-e592a0d6:~# nova availability-zone-list
+----------------------------------------------------------+----------------------------------------+
| Name | Status |
 +----------------------------------------------------------+----------------------------------------+
| internal | available |
| |- cantu-rpco-master01-nova-cert-container-0af67b8a | |
| | |- nova-cert | enabled :-) 2017-04-19T16:54:00.000000 |
| |- cantu-rpco-master01-nova-conductor-container-2fabdc9a | |
| | |- nova-conductor | enabled :-) 2017-04-19T16:53:57.000000 |
| |- cantu-rpco-master01-nova-console-container-7e2e9044 | |
| | |- nova-consoleauth | enabled :-) 2017-04-19T16:54:00.000000 |
| |- cantu-rpco-master01-nova-scheduler-container-62144ac1 | |
| | |- nova-scheduler | enabled :-) 2017-04-19T16:53:56.000000 |
| alex | available |
| |- cantu-rpco-master01 | |
| | |- nova-compute | enabled :-) 2017-04-19T16:53:57.000000 |
 +----------------------------------------------------------+----------------------------------------+

Cinder AZ:
root@cantu-rpco-master01-utility-container-e592a0d6:~# cinder availability-zone-list
+------+-----------+
| Name | Status |
 +------+-----------+
| alex | available |
 +------+-----------+

Created bootable volume:
root@cantu-rpco-master01-utility-container-e592a0d6:~# openstack volume create --image $IMAGE --availability-zone alex --size 2 boot_vol_alex
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
+---------------------+--------------------------------------+
| Field | Value |
 +---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | alex |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-04-19T16:52:31.729774 |
| description | None |
| encrypted | False |
| id | 9ee8b04e-edc4-4d70-9842-2d0781d414f9 |
| migration_status | None |
| multiattach | False |
| name | boot_vol_alex |
| properties | ...

Read more...

Revision history for this message
John Haan (yongiman) wrote : Re: [Bug 1648324] Re: When cross_az_attach set to false and creating VM with bootable volumes created before, InvalidBDM error raised.
Download full text (4.2 KiB)

Hi, Matt

Thank you for patching the bug I reported.

John.

2017-05-26 10:33 GMT+09:00 Matt Riedemann <email address hidden>:

> *** This bug is a duplicate of bug 1693600 ***
> https://bugs.launchpad.net/bugs/1693600
>
> ** This bug has been marked a duplicate of bug 1693600
> Boot from volume fails when cross_az_attach=False due to:
> ObjectActionError: Object action obj_load_attr failed because: attribute
> host not lazy-loadable
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1648324
>
> Title:
> When cross_az_attach set to false and creating VM with bootable
> volumes created before, InvalidBDM error raised.
>
> Status in OpenStack Compute (nova):
> In Progress
>
> Bug description:
> Description
> ===========
>
> When cross_az_attach option is set to False and you try to boot an
> instance with bootable volume created before, InvalidBDM error raised.
>
> Steps to reproduce
> ==================
>
> 1. Configure multiple AZs in Nova and Cinder.
> - for Nova : AZ1 and AZ2
> - for Cinder : AZ1 and AZ2
>
> 2. cross_az_attach set to false in nova.conf
> [cinder]
> cross_az_attach = False
>
> 3. Restart nova-compute service
>
> 4. Create volume from glance image
> $ cinder create 10 --display-name test-bootable-volume --image-id
> {image-id} --availability-zone AZ1
>
> 5. Create instance from the volume created
> $ nova boot --flavor m1.small --boot-volume 7c15ae47-12bd-41e7-822d-37e3619082be
> --nic net-id=1493b02c-c257-45db-84ea-789f87ed7924 --availability-zone AZ1
> demo-instance
>
> Expected result
> ===============
>
> Instance is booted successfully in AZ1 zone.
>
> Actual result
> =============
>
> Instance failed to boot and raise InvalidBDM Error
>
> nova command output says:
>
> ERROR (BadRequest): Block Device Mapping is Invalid: failed to get
> volume 7c15ae47-12bd-41e7-822d-37e3619082be . (HTTP 400) (Request-ID:
> req-8c36c5d1-b18d-4db5-a86f-db33c621e4e4)
>
> nova-api.log says:
>
> 2016-12-02 20:02:48.724 150051 DEBUG cinderclient.client
> [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53
> 579e9f03f761431dbe194c8cf2baeed9 - default default] GET call to cinderv2
> for http://10.161.32.13:8776/v2/579e9f03f761431dbe194c8cf2baee
> d9/volumes/7c15ae47-12bd-41e7-822d-37e3619082be used request id
> req-4f4170af-bc29-4b92-9c7d-c9a5ce457aa8 _log_request_id
> /usr/local/lib/python2.7/dist-packages/cinderclient/client.py:95
> 2016-12-02 20:02:48.734 150051 DEBUG nova.quota
> [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53
> 579e9f03f761431dbe194c8cf2baeed9 - default default] Rolled back
> reservations [u'ff856b75-4457-4b23-abe6-cf5981cacab8',
> u'74271baa-3385-4a03-bb99-86d05e30846c', u'09c2f52f-8f1e-46bf-9876-96f5ef1cf553']
> rollback /usr/local/lib/python2.7/dist-packages/nova/quota.py:1427
> 2016-12-02 20:02:48.734 150051 INFO nova.api.openstack.wsgi
> [req-8356085e-2594-4086-808e-42faba6b5ed2 22e5c7efda0345e3b4326946b8e8dc53
> 579e9f03f761431dbe194c8cf2baeed9 - default default] HTTP exception
> thrown: Block Device Mapping is Inval...

Read more...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by John Haan (<email address hidden>) on branch: master
Review: https://review.openstack.org/409680
Reason: This bug is patched all.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.