OpenStack Compute (nova)

Nova returns HTTP500 if attaching the attached volume

Bug #1632513 reported by Ken'ichi Ohmichi on 2016-10-11

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	Medium	Ken'ichi Ohmichi
	Newton	Fix Committed	Medium	Lee Yarwood

Bug Description

We tried to add some negative test case to Tempest for the other bug.
However, Nova still returns HTTP500 error if attaching the already attached volume like:

http://logs.openstack.org/83/382083/12/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/66f700f/logs/screen-n-api.txt.gz#_2016-10-11_19_28_54_097

2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions [req-3a860748-ab8f-4ce7-8412-90f787dd151d tempest-AttachVolumeNegativeTest-1404533522 tempest-AttachVolumeNegativeTest-1404533522] Unexpected exception in API method
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/api/openstack/extensions.py", line 338, in wrapped
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions return f(*args, **kwargs)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/api/validation/__init__.py", line 73, in wrapper
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/api/openstack/compute/volumes.py", line 325, in create
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions volume_id, device)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/compute/api.py", line 164, in inner
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions return function(self, context, instance, *args, **kwargs)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/compute/api.py", line 145, in inner
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions return f(self, context, instance, *args, **kw)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/compute/api.py", line 3481, in attach_volume
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions disk_bus, device_type)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/compute/api.py", line 3424, in _attach_volume
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions volume_bdm.destroy()
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions self.force_reraise()
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions six.reraise(self.type_, self.value, self.tb)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/compute/api.py", line 3420, in _attach_volume
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions self._check_attach_and_reserve_volume(context, volume_id, instance)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/compute/api.py", line 3407, in _check_attach_and_reserve_volume
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions self.volume_api.reserve_volume(context, volume_id)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/volume/cinder.py", line 196, in wrapper
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions _reraise(exception.InvalidInput(reason=err_msg))
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/volume/cinder.py", line 246, in _reraise
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2])
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/volume/cinder.py", line 188, in wrapper
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions res = method(self, ctx, *args, **kwargs)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/volume/cinder.py", line 210, in wrapper
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions res = method(self, ctx, volume_id, *args, **kwargs)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/opt/stack/new/nova/nova/volume/cinder.py", line 327, in reserve_volume
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions cinderclient(context).volumes.reserve(volume_id)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/cinderclient/v3/volumes.py", line 383, in reserve
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions return self._action('os-reserve', volume)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/cinderclient/v3/volumes.py", line 346, in _action
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions resp, body = self.api.client.post(url, body=body)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 146, in post
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions return self._cs_request(url, 'POST', **kwargs)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 134, in _cs_request
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions return self.request(url, method, **kwargs)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 123, in request
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions raise exceptions.from_response(resp, body)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions InvalidInput: Invalid input received: Invalid volume: Volume status must be available to reserve. (HTTP 400) (Request-ID: req-1af67f41-54a7-4539-89f2-c08d45184153)
2016-10-11 19:28:54.097 3865 ERROR nova.api.openstack.extensions

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-11: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/385200

Changed in nova:
assignee:	nobody → Ken'ichi Ohmichi (oomichi)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-14: Fix merged to nova (master)

Reviewed: https://review.openstack.org/385200
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=21961e7bc5c9c9650597a9ed7f15a913fcab09d9
Submitter: Jenkins
Branch: master

commit 21961e7bc5c9c9650597a9ed7f15a913fcab09d9
Author: Ken'ichi Ohmichi <email address hidden>
Date: Tue Oct 11 16:27:10 2016 -0700

Add InvalidInput handling for attach-volume

    If attaching the already attached volume to a server, Cinder returns
    HTTP400 response to Nova but Nova didn't except the response.
    Then Nova returned HTTP500 response to a client.
    Tempest test I594566704b9794457d224031802d9cbf132e765f reproduces
    this error case.

Closes-Bug: #1632513
Change-Id: I6718883cb6b42d9b816e3799324a166cbfe92b40

Changed in nova:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-14: Fix proposed to nova (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/386718

Matt Riedemann (mriedem) on 2016-10-14

Changed in nova:
importance:	Undecided → Medium

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-10-16: Fix merged to nova (stable/newton)

Reviewed: https://review.openstack.org/386718
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d2a4decffe3cc3a12624ff8e8f50b8c27c3b8775
Submitter: Jenkins
Branch: stable/newton

commit d2a4decffe3cc3a12624ff8e8f50b8c27c3b8775
Author: Ken'ichi Ohmichi <email address hidden>
Date: Tue Oct 11 16:27:10 2016 -0700

Add InvalidInput handling for attach-volume

    Closes-Bug: #1632513
    Change-Id: I6718883cb6b42d9b816e3799324a166cbfe92b40
    (cherry picked from commit 21961e7bc5c9c9650597a9ed7f15a913fcab09d9)