cinder image cache fails

[Walt Boring]

Enabled your cinder driver to use the image cache feature

create a volume from image.

It fails to create the cache image every time.

There is a silent failure in the volume/manager.py's _clone_image_volume

https://github.com/openstack/cinder/blob/master/cinder/volume/manager.py#L1084

deleting the key volume_type fails, but since we wrap that entire block in a try Exception and return false, we never get notified why it failed.

I believe the patch that caused this failure is
https://review.openstack.org/#/c/240735/
as it changed the volume ref to a versionedobject and since we trap that failure and don't log it and return false, it silently fails.

I modified the except Exception to raise the exception that this is what I get

2015-12-11 16:56:37.948 TRACE cinder.volume.manager Traceback (most recent call last):
2015-12-11 16:56:37.948 TRACE cinder.volume.manager File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 82, in _execute_task
2015-12-11 16:56:37.948 TRACE cinder.volume.manager result = task.execute(**arguments)
2015-12-11 16:56:37.948 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 834, in execute
2015-12-11 16:56:37.948 TRACE cinder.volume.manager **volume_spec)
2015-12-11 16:56:37.948 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 781, in _create_from_image
2015-12-11 16:56:37.948 TRACE cinder.volume.manager image_meta)
2015-12-11 16:56:37.948 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 1053, in _create_image_cache_volume_entry
2015-12-11 16:56:37.948 TRACE cinder.volume.manager image_meta)
2015-12-11 16:56:37.948 TRACE cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 1085, in _clone_image_volume
2015-12-11 16:56:37.948 TRACE cinder.volume.manager del new_vol_values['volume_type']
2015-12-11 16:56:37.948 TRACE cinder.volume.manager KeyError: 'volume_type'
2015-12-11 16:56:37.948 TRACE cinder.volume.manager

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/256966

[John Griffith (john-griffith)]

Still need to figure out the unit tests here.. .but the fix is up.

There are some other problems I noticed here. Even though we have unit tests for it, we don't clean up the ref objects we create in failure scenarios (go figure, unit tests work, but it doesn't actually delete). Also, we silently eat just about every single error message that gets generated in all of this, making it pretty tough to ever debug anything.

I'll open a separate bug for the delete/cleanup stuff after I figure it out a bit more.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/256966
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=3f3a8f11571a71770f08bc2ef48562300b44d9f7
Submitter: Jenkins
Branch: master

commit 3f3a8f11571a71770f08bc2ef48562300b44d9f7
Author: John Griffith <email address hidden>
Date: Sat Dec 12 18:32:57 2015 +0000

    Fix invalid cache image-volume creation

    During the volume ref creation in clone_image_volume
    we were including a _name_id attribute which is invalid
    in the base olso object. The result is silent failure
    for a key error. Note that the comments for the column
    in the db specifcally state "do not call directly".
    The variable is also denoted as private by the _ prefix.

    Fix this by just popping the "_name_id' attribute.

    In addition there were a number of things that needed
    some cleanup:
    1. Don't iterate over list of tuples to get K/V for dict
       Just use Dict(xxxx)
    2. Don't use delete dict.key
       This isn't safe, because if the key DNE you'll get a key exception
       Instead use pop which is safe if key DNE
    3. Add an error message when this fails, because currently the logs
       don't give an operation a chance of knowing it failed, let alone
       an opportunity to debug it

    Finally the cache volume creation is still calling the db
    directly and doing a resource create as opposed to
    using the new volume object create handler. I went ahead
    and converted this to use objects, however that resulted in
    breaking two of the volume-->image unit tests that use cinder
    as a glance backend.

    I've opened bug 1525773 for those issues and added a skip
    to the tests in question.

    Change-Id: Ieda40c1111d4d4d69488944eb332962eb9658266
    Closes-Bug: #1525452

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/cinder 8.0.0.0b2 development milestone.