Glance Image Can Be Orphaned If Backing Cinder Volume Deleted

Bug #1655172 reported by Matt Smith
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
New
Undecided
Unassigned

Bug Description

If you create a cinder-volume-backed image in glance, then delete the volume the image in glance shows as being valid even though the backing volume is now invalid.

I think ideally glance should have some knowledge of which volume locations are valid otherwise you could end up with an orphaned image if the backing location is invalidated.

Reproduction Steps:

Follow the steps outlined on this page to enable the Glance volume-backed image-cache:
http://docs.openstack.org/admin-guide/blockstorage-volume-backed-image.html

1. Make sure glance-api.conf has these lines:
`stores = file, http, cinder`
`show_multiple_locations = True`

2. Make sure cinder.conf has these lines in DEFAULT
`glance_api_version = 2`
`allowed_direct_url_schemes = cinder`

3. Make sure current backend section in cinder.conf has these lines:
`image_upload_use_cinder_backend = True`
`image_upload_use_internal_tenant = True`

4. Restart glance and cinder services

5. Create a Cinder volume and write some data to it (the data writing isn't really necessary for the bug though)
`cinder create 5 --display-name testvol`

6. Create a glance image that has this volume as a location:
`openstack image create --disk-format raw --container-format bare my-test-image`
`glance location-add <my-test-image-uuid> --url cinder://<testvol-uuid>`

7. Create a new Cinder volume from the new image:
`openstack volume create testvol2 --image <my-test-image-uuid> --size 5` <-- This happen successfully

8. Delete testvol
`cinder delete <testvol-uuid>`

9. Perform step #7 again, this time it will fail with an HTTP 500 error in the cinder volume logs

2017-01-09 22:52:16.294 ERROR oslo_messaging.rpc.server [req-7b4326c9-0b45-43c3-bc04-618a6e4b5477 4077fa82897042c28b1765ee04c84d7c 89788f833b5d4c00b47626919c977b2d] Exception during message handling
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 4367, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server allow_reschedule=allow_reschedule, volume=volume)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 635, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _run_flow()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 627, in _run_flow
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server flow_engine.run()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failures[0].reraise()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(*self._exc_info)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = task.execute(**arguments)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 842, in execute
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server **volume_spec)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 761, in _create_from_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service, context, image_id) as tmp_image:
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self.gen.next()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 567, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server fetch_verify_image(context, image_service, image_id, tmp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 212, in fetch_verify_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server None, None)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 190, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service.download(context, image_id, image_file)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 335, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _reraise_translated_image_exception(image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 572, in _reraise_translated_image_exception
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(type(new_exc), new_exc, exc_trace)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 333, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_chunks = self._client.call(context, 'data', image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 197, in call
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 195, in data
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body = self.http_client.get(url)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 287, in get
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._request('GET', url, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 279, in _request
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body_iter = self._handle_response(resp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 107, in _handle_response
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server raise exc.from_response(resp, resp.content)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server HTTPInternalServerError: HTTPInternalServerError (HTTP 500)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server

Because the image in glance still lists this volume as a valid location
`glance image-show <my-test-image-uuid>`

Matt Smith (mss-4)
description: updated
description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.