cinder ImageUnacceptable exception not returned to called

[Mehdi Abaakouk]

When I try to create a volume from an image with a size too small to receive the image, the error message is not returned to cinder-api and reported to the user.

$ cinder create --image-id bdcdf39f-a37f-4788-bd5f-c38ba587aaf8 --name test4 1
$ cinder show test4
+--------------------------------+--------------------------------------+
| Property | Value |
+--------------------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| created_at | 2014-04-02T08:26:17.000000 |
| description | None |
| encrypted | False |
| id | b73c8639-9235-4483-b08c-75b0a798461b |
| metadata | {} |
| name | test4 |
| os-vol-host-attr:host | master |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 87adafba5b1d42109f9ac43523567b1e |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | error |
| user_id | 13c605bbec5340eeb8e9b9d1a8847d42 |
| volume_type | None |
+--------------------------------+--------------------------------------+

In cinder-volume daemon log:

2014-04-02 08:26:19.676 ERROR oslo.messaging._drivers.common [req-42ee1874-6d27-44d2-8cc5-efeb6869a11b 13c605bbec5340eeb8e9b9d1a8847d42 87adafba5b1d42109f9ac43523567b1e] ['Traceback (most recent call last):\n', ' File "/vagrant/stack-master/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n incoming.message))\n', ' File "/vagrant/stack-master/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/vagrant/stack-master/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n result = getattr(endpoint, method)(ctxt, **new_args)\n', ' File "/vagrant/stack-master/cinder/cinder/volume/manager.py", line 363, in create_volume\n _run_flow()\n', ' File "/vagrant/stack-master/cinder/cinder/volume/manager.py", line 356, in _run_flow\n flow_engine.run()\n', ' File "/vagrant/stack-master/taskflow/taskflow/utils/lock_utils.py", line 54, in wrapper\n return f(*args, **kwargs)\n', ' File "/vagrant/stack-master/taskflow/taskflow/engines/action_engine/engine.py", line 96, in run\n self._run()\n', ' File "/vagrant/stack-master/taskflow/taskflow/engines/action_engine/engine.py", line 111, in _run\n misc.Failure.reraise_if_any(failures.values())\n', ' File "/vagrant/stack-master/taskflow/taskflow/utils/misc.py", line 649, in reraise_if_any\n failures[0].reraise()\n', ' File "/vagrant/stack-master/taskflow/taskflow/utils/misc.py", line 656, in reraise\n six.reraise(*self._exc_info)\n', ' File "/vagrant/stack-master/taskflow/taskflow/engines/action_engine/executor.py", line 34, in _execute_task\n result = task.execute(**arguments)\n', ' File "/vagrant/stack-master/cinder/cinder/volume/flows/manager/create_volume.py", line 594, in execute\n **volume_spec)\n', ' File "/vagrant/stack-master/cinder/cinder/volume/flows/manager/create_volume.py", line 556, in _create_from_image\n image_id, image_location, image_service)\n', ' File "/vagrant/stack-master/cinder/cinder/volume/flows/manager/create_volume.py", line 463, in _copy_image_to_volume\n raise exception.ImageUnacceptable(ex)\n', "ImageUnacceptable: Image bdcdf39f-a37f-4788-bd5f-c38ba587aaf8 is unacceptable: Size is 2GB and doesn't fit in a volume of size 1GB.\n"]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 346, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 197, in main
    self._resolve_links()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 212, in _resolve_links
    f(self, *ca, **ckw)
  File "/vagrant/stack-master/oslo.messaging/oslo/messaging/_executors/impl_eventlet.py", line 47, in complete
    thread.wait()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
    return self._exit_event.wait()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 120, in wait
    current.throw(*self._exc)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
    result = function(*args, **kwargs)
  File "/vagrant/stack-master/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 128, in <lambda>
    yield lambda: self._dispatch_and_reply(incoming)
  File "/vagrant/stack-master/oslo.messaging/oslo/messaging/rpc/dispatcher.py", line 143, in _dispatch_and_reply
    incoming.reply(failure=exc_info)
  File "/vagrant/stack-master/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py", line 67, in reply
    self._send_reply(conn, reply, failure, log_failure=log_failure)
  File "/vagrant/stack-master/oslo.messaging/oslo/messaging/_drivers/amqpdriver.py", line 48, in _send_reply
    log_failure)
  File "/vagrant/stack-master/oslo.messaging/oslo/messaging/_drivers/common.py", line 328, in serialize_remote_exception
    json_data = jsonutils.dumps(data)
  File "/vagrant/stack-master/oslo.messaging/oslo/messaging/openstack/common/jsonutils.py", line 164, in dumps
    return json.dumps(value, default=default, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
ValueError: Circular reference detected

I use devstack master (c72cc4b2345e9e271205d1de00c9812d0a2684ec) and NFS driver

Thanks in advance.

[git-harry (git-harry)]

The issue appears to be line 463 of cinder/volume/flows/manager/create_volume.py

raise exception.ImageUnacceptable(ex)

ex is the caught exception

[git-harry (git-harry)]

Just to clarify. For cinder-api to check the size of the image a request would need to be made to glance for the information. Blocking calls such as this are not included in the api by design and so this needs to be done by cinder-volume.

The trace does include another error mentioned in my previous comment and I will try to address that.

[Openstack Gerrit (openstack-gerrit)]

Fix proposed to branch: master
Review: https://review.openstack.org/90990

[Mike Perez (thingee)]

git-harry, I looked closer at this and found a validation check. It appears it is fetching it afterall. https://github.com/openstack/cinder/blob/master/cinder/volume/flows/api/create_volume.py#L185

[git-harry (git-harry)]

Mike, yeah there is a check but it doesn't happen as part of the initial acceptance of the api request. The bug was raised to have the error returned in response to the initial api request by the user.

[Cory Stone (corystone)]

Can you post some more details about the image? Some of those checks are just against image metadata, and some of them are against the actual qemu output. The API sure seems to check the metadata.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/90990
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=31a09451394458273ae1ed9af867476da4ece9d6
Submitter: Jenkins
Branch: master

commit 31a09451394458273ae1ed9af867476da4ece9d6
Author: git-harry <email address hidden>
Date: Thu Apr 17 10:49:49 2014 +0100

    CinderException args to strings when exceptions

    CinderException message and kwargs are converted to strings if the
    argument is itself an exception.

    Exceptions passed as arguments to CinderException happen in many areas
    of the code. This causes an error when oslo.messaging tries to serialise
    the exception.

    Change-Id: I1399c939bbca47ab8362ac3bbe0e9a349c6d5572
    Closes-Bug: #1301249

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/148289

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jan Klare (<email address hidden>) on branch: stable/icehouse
Review: https://review.openstack.org/148289