If you create a cinder-volume-backed image in glance, then delete the volume the image in glance shows as being valid even though the backing volume is now invalid.
I think ideally glance should have some knowledge of which volume locations are valid otherwise you could end up with an orphaned image if the backing location is invalidated.
1. Make sure glance-api.conf has these lines:
`stores = file, http, cinder`
`show_multiple_locations = True`
2. Make sure cinder.conf has these lines in DEFAULT
`glance_api_version = 2`
`allowed_direct_url_schemes = cinder`
3. Make sure current backend section in cinder.conf has these lines:
`image_upload_use_cinder_backend = True`
`image_upload_use_internal_tenant = True`
4. Restart glance and cinder services
5. Create a Cinder volume and write some data to it (the data writing isn't really necessary for the bug though)
`cinder create 5 --display-name testvol`
6. Create a glance image that has this volume as a location:
`openstack image create --disk-format raw --container-format bare my-test-image`
`glance location-add <my-test-image-uuid> --url cinder://<testvol-uuid>`
7. Create a new Cinder volume from the new image:
`openstack volume create testvol2 --image <my-test-image-uuid> --size 5` <-- This happen successfully
8. Delete testvol
`cinder delete <testvol-uuid>`
9. Perform step #7 again, this time it will fail with an HTTP 500 error in the cinder volume logs
```
2017-01-09 22:52:16.294 ERROR oslo_messaging.rpc.server [req-7b4326c9-0b45-43c3-bc04-618a6e4b5477 4077fa82897042c28b1765ee04c84d7c 89788f833b5d4c00b47626919c977b2d] Exception during message handling
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 4367, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server allow_reschedule=allow_reschedule, volume=volume)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 635, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _run_flow()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 627, in _run_flow
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server flow_engine.run()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failures[0].reraise()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(*self._exc_info)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = task.execute(**arguments)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 842, in execute
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server **volume_spec)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 761, in _create_from_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service, context, image_id) as tmp_image:
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self.gen.next()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 567, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server fetch_verify_image(context, image_service, image_id, tmp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 212, in fetch_verify_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server None, None)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 190, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service.download(context, image_id, image_file)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 335, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _reraise_translated_image_exception(image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 572, in _reraise_translated_image_exception
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(type(new_exc), new_exc, exc_trace)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 333, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_chunks = self._client.call(context, 'data', image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 197, in call
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 195, in data
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body = self.http_client.get(url)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 287, in get
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._request('GET', url, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 279, in _request
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body_iter = self._handle_response(resp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 107, in _handle_response
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server raise exc.from_response(resp, resp.content)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server HTTPInternalServerError: HTTPInternalServerError (HTTP 500)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server```
Because the image in glance still lists this volume as a valid location
`glance image-show <my-test-image-uuid>`
If you create a cinder- volume- backed image in glance, then delete the volume the image in glance shows as being valid even though the backing volume is now invalid.
I think ideally glance should have some knowledge of which volume locations are valid otherwise you could end up with an orphaned image if the backing location is invalidated.
Reproduction Steps:
Follow the steps outlined on this page to enable the Glance volume-backed image-cache: docs.openstack. org/admin- guide/blockstor age-volume- backed- image.html
http://
1. Make sure glance-api.conf has these lines: locations = True`
`stores = file, http, cinder`
`show_multiple_
2. Make sure cinder.conf has these lines in DEFAULT direct_ url_schemes = cinder`
`glance_api_version = 2`
`allowed_
3. Make sure current backend section in cinder.conf has these lines: upload_ use_cinder_ backend = True` upload_ use_internal_ tenant = True`
`image_
`image_
4. Restart glance and cinder services
5. Create a Cinder volume and write some data to it (the data writing isn't really necessary for the bug though)
`cinder create 5 --display-name testvol`
6. Create a glance image that has this volume as a location: image-uuid> --url cinder: //<testvol- uuid>`
`openstack image create --disk-format raw --container-format bare my-test-image`
`glance location-add <my-test-
7. Create a new Cinder volume from the new image: image-uuid> --size 5` <-- This happen successfully
`openstack volume create testvol2 --image <my-test-
8. Delete testvol
`cinder delete <testvol-uuid>`
9. Perform step #7 again, this time it will fail with an HTTP 500 error in the cinder volume logs rpc.server [req-7b4326c9- 0b45-43c3- bc04-618a6e4b54 77 4077fa82897042c 28b1765ee04c84d 7c 89788f833b5d4c0 0b47626919c977b 2d] Exception during message handling rpc.server Traceback (most recent call last): rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ rpc/server. py", line 133, in _process_incoming rpc.server res = self.dispatcher .dispatch( message) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 150, in dispatch rpc.server return self._do_ dispatch( endpoint, method, ctxt, args) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ oslo_messaging/ rpc/dispatcher. py", line 121, in _do_dispatch rpc.server result = func(ctxt, **new_args) rpc.server File "/opt/stack/ cinder/ cinder/ volume/ manager. py", line 4367, in create_volume rpc.server allow_reschedul e=allow_ reschedule, volume=volume) rpc.server File "/opt/stack/ cinder/ cinder/ volume/ manager. py", line 635, in create_volume rpc.server _run_flow() rpc.server File "/opt/stack/ cinder/ cinder/ volume/ manager. py", line 627, in _run_flow rpc.server flow_engine.run() rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ taskflow/ engines/ action_ engine/ engine. py", line 247, in run rpc.server for _state in self.run_ iter(timeout= timeout) : rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ taskflow/ engines/ action_ engine/ engine. py", line 340, in run_iter rpc.server failure. Failure. reraise_ if_any( er_failures) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ taskflow/ types/failure. py", line 336, in reraise_if_any rpc.server failures[ 0].reraise( ) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ taskflow/ types/failure. py", line 343, in reraise rpc.server six.reraise( *self._ exc_info) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ taskflow/ engines/ action_ engine/ executor. py", line 53, in _execute_task rpc.server result = task.execute( **arguments) rpc.server File "/opt/stack/ cinder/ cinder/ volume/ flows/manager/ create_ volume. py", line 842, in execute rpc.server **volume_spec) rpc.server File "/opt/stack/ cinder/ cinder/ volume/ flows/manager/ create_ volume. py", line 761, in _create_from_image rpc.server image_service, context, image_id) as tmp_image: rpc.server File "/usr/lib/ python2. 7/contextlib. py", line 17, in __enter__ rpc.server return self.gen.next() rpc.server File "/opt/stack/ cinder/ cinder/ image/image_ utils.py" , line 567, in fetch rpc.server fetch_verify_ image(context, image_service, image_id, tmp) rpc.server File "/opt/stack/ cinder/ cinder/ image/image_ utils.py" , line 212, in fetch_verify_image rpc.server None, None) rpc.server File "/opt/stack/ cinder/ cinder/ image/image_ utils.py" , line 190, in fetch rpc.server image_service. download( context, image_id, image_file) rpc.server File "/opt/stack/ cinder/ cinder/ image/glance. py", line 335, in download rpc.server _reraise_ translated_ image_exception (image_ id) rpc.server File "/opt/stack/ cinder/ cinder/ image/glance. py", line 572, in _reraise_ translated_ image_exception rpc.server six.reraise( type(new_ exc), new_exc, exc_trace) rpc.server File "/opt/stack/ cinder/ cinder/ image/glance. py", line 333, in download rpc.server image_chunks = self._client. call(context, 'data', image_id) rpc.server File "/opt/stack/ cinder/ cinder/ image/glance. py", line 197, in call rpc.server return getattr(controller, method)(*args, **kwargs) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ glanceclient/ v2/images. py", line 195, in data rpc.server resp, body = self.http_ client. get(url) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ glanceclient/ common/ http.py" , line 287, in get rpc.server return self._request( 'GET', url, **kwargs) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ glanceclient/ common/ http.py" , line 279, in _request rpc.server resp, body_iter = self._handle_ response( resp) rpc.server File "/usr/local/ lib/python2. 7/dist- packages/ glanceclient/ common/ http.py" , line 107, in _handle_response rpc.server raise exc.from_ response( resp, resp.content) rpc.server HTTPInternalSer verError: HTTPInternalSer verError (HTTP 500) rpc.server` ``
```
2017-01-09 22:52:16.294 ERROR oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
2017-01-09 22:52:16.294 TRACE oslo_messaging.
Because the image in glance still lists this volume as a valid location image-uuid> `
`glance image-show <my-test-