SolidFire template caching is broken

[John Griffith]

We seem to have broken a number of things in the template caching during this cycle, mostly around handling internal volumes, but also there seems to be some issues with public images and Glance V2.

[Mathieu Gagné (mgagne)]

The system doesn't work (with Glance v2) when creating a volume from a public image which isn't owned by the project.

Glance V2 renamed the "is_public" metadata name for "visiblity". Whatever is building the image metadata object in Cinder doesn't account for that and is_public ends up being None. Since the image isn't owned by the project, Cinder will never cache the image. Here is (tweaked log entry) about the failure:

Requested image is not accessible (is_public=None) by current tenant (image.owner=4312f8a908824f65b3467c88f6c6a287, volume.project_id=898f10a1935540d28d2e5c86176aa8fa)

[Mathieu Gagné (mgagne)]

Furthermore, if the image is not found in Glance (tested with v1) for whatever reasons, this line crashes Cinder:
https://github.com/openstack/cinder/blob/ccca5f5dacf3d2e36917d3005b0e536d1d235e5d/cinder/volume/drivers/solidfire.py#L686

template_vol is not a sf_volume but a model_info which doesn't have volumeID key available (tested against Cinder 2015.1.0):

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 67, in _execute_task
    result = task.execute(**arguments)
  File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 653, in execute
    **volume_spec)
  File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 584, in _create_from_image
    image_service)
  File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/solidfire.py", line 667, in clone_image
    image_meta['id']) is None:
  File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/solidfire.py", line 575, in _create_image_volume
    params['volumeID'] = template_vol['volumeID']
KeyError: 'volumeID'

[Mathieu Gagné (mgagne)]

Failure to find the key also means the vtemplate is not properly deleted on error.

This means the SolidFire volume still exist and users will now clone an empty volume instead of the image.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/222756

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/222756
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=809567661d6732eaf4a5e77d2f2df07621ee5114
Submitter: Jenkins
Branch: master

commit 809567661d6732eaf4a5e77d2f2df07621ee5114
Author: John Griffith <email address hidden>
Date: Fri Sep 11 19:57:32 2015 +0000

    Fix volume lookups in SolidFire template caching

    There's a number of issues with the SolidFire template caching
    code. There are a number of metadata items that aren't consistent
    between Glance API V1 and V2, there's also some problems with how
    the code is dealing with it's internal volume and just flat out
    wrong account checking.

    This patch focuses mainly on fixing up the account and volumeID
    issues. The _do_clone method currently expects to find the src
    volume under the same account as the dest volume. That's fine
    for most cases, but doesn't work for either the SolidFire template
    caching, or for the use of generic image caching in Cinder.

    We fix that easily by modifying the _get_sf_volume call to have an
    option to search all active volumes on the Cluster instead of just
    those for a specified account. For now we only change the clone
    methods, as they're the only case where we fall into the case of
    cross-ownership of volumes in the same context.

    The other issue is around the cleanup/delete in the case of a
    failed template-volume create. In this case we were passing in
    the Cinder UUID to a direct SolidFire API delete cmd, which
    is wrong. The direct call for the SolidFire API call requires
    that the actual SolidFire ID be used. So we leverage the modified
    _get_sf_volume call here as well.

    Another issue that was introduced was the introduction of
    a connector object to the intialize_connection call. We
    got around this once already by just passing in a fake
    connector that was actually a string. That was fine, but
    the initialize_connection routine also now checks the connector
    for multi_attach, so it fails the _get call on a string. We
    fix that by just building a dict and using that instead of
    a fake string.

    Finally, the method of relying on the Glance metadata virt size
    has been problematic, it's not set by default, it's not accurate
    and it's a bit inefficient. There have been a number of issues
    with this method of size determination, so we'll dump that and
    just leverage the image_utils methods directly.

    Change-Id: Icfd6668389049d3d5686acdb832847aa2d2fce52
    Closes-Bug: #1494830
    Closes-Bug: #1494927

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/256893

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dave Walker (<email address hidden>) on branch: stable/kilo
Review: https://review.openstack.org/256893
Reason:
stable/kilo closed for 2015.1.4

This release is now pending its final release and no freeze exception has
been seen for this changeset. Therefore, I am now abandoning this change.

If this is not correct, please urgently raise a thread on openstack-dev.

More details at: https://wiki.openstack.org/wiki/StableBranch