| 2017-01-09 23:07:26 |
Matt Smith |
bug |
|
|
added bug |
| 2017-01-09 23:09:04 |
Matt Smith |
description |
If you create a cinder-volume-backed image in glance, then delete the volume the image in glance shows as being valid even though the backing volume is now invalid.
I think ideally glance should have some knowledge of which volume locations are valid otherwise you could end up with an orphaned image if the backing location is invalidated.
Reproduction Steps:
Follow the steps outlined on this page to enable the Glance volume-backed image-cache:
http://docs.openstack.org/admin-guide/blockstorage-volume-backed-image.html
1. Make sure glance-api.conf has these lines:
`stores = file, http, cinder`
`show_multiple_locations = True`
2. Make sure cinder.conf has these lines in DEFAULT
`glance_api_version = 2`
`allowed_direct_url_schemes = cinder`
3. Make sure current backend section in cinder.conf has these lines:
`image_upload_use_cinder_backend = True`
`image_upload_use_internal_tenant = True`
4. Restart glance and cinder services
5. Create a Cinder volume and write some data to it (the data writing isn't really necessary for the bug though)
`cinder create 5 --display-name testvol`
6. Create a glance image that has this volume as a location:
`openstack image create --disk-format raw --container-format bare my-test-image`
`glance location-add <my-test-image-uuid> --url cinder://<testvol-uuid>`
7. Create a new Cinder volume from the new image:
`openstack volume create testvol2 --image <my-test-image-uuid> --size 5` <-- This happen successfully
8. Delete testvol
`cinder delete <testvol-uuid>`
9. Perform step #7 again, this time it will fail with an HTTP 500 error in the cinder volume logs
```
2017-01-09 22:52:16.294 ERROR oslo_messaging.rpc.server [req-7b4326c9-0b45-43c3-bc04-618a6e4b5477 4077fa82897042c28b1765ee04c84d7c 89788f833b5d4c00b47626919c977b2d] Exception during message handling
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 4367, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server allow_reschedule=allow_reschedule, volume=volume)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 635, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _run_flow()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 627, in _run_flow
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server flow_engine.run()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failures[0].reraise()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(*self._exc_info)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = task.execute(**arguments)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 842, in execute
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server **volume_spec)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 761, in _create_from_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service, context, image_id) as tmp_image:
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self.gen.next()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 567, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server fetch_verify_image(context, image_service, image_id, tmp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 212, in fetch_verify_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server None, None)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 190, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service.download(context, image_id, image_file)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 335, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _reraise_translated_image_exception(image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 572, in _reraise_translated_image_exception
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(type(new_exc), new_exc, exc_trace)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 333, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_chunks = self._client.call(context, 'data', image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 197, in call
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 195, in data
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body = self.http_client.get(url)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 287, in get
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._request('GET', url, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 279, in _request
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body_iter = self._handle_response(resp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 107, in _handle_response
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server raise exc.from_response(resp, resp.content)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server HTTPInternalServerError: HTTPInternalServerError (HTTP 500)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server```
Because the image in glance still lists this volume as a valid location
`glance image-show <my-test-image-uuid>` |
If you create a cinder-volume-backed image in glance, then delete the volume the image in glance shows as being valid even though the backing volume is now invalid.
I think ideally glance should have some knowledge of which volume locations are valid otherwise you could end up with an orphaned image if the backing location is invalidated.
Reproduction Steps:
Follow the steps outlined on this page to enable the Glance volume-backed image-cache:
http://docs.openstack.org/admin-guide/blockstorage-volume-backed-image.html
1. Make sure glance-api.conf has these lines:
`stores = file, http, cinder`
`show_multiple_locations = True`
2. Make sure cinder.conf has these lines in DEFAULT
`glance_api_version = 2`
`allowed_direct_url_schemes = cinder`
3. Make sure current backend section in cinder.conf has these lines:
`image_upload_use_cinder_backend = True`
`image_upload_use_internal_tenant = True`
4. Restart glance and cinder services
5. Create a Cinder volume and write some data to it (the data writing isn't really necessary for the bug though)
`cinder create 5 --display-name testvol`
6. Create a glance image that has this volume as a location:
`openstack image create --disk-format raw --container-format bare my-test-image`
`glance location-add <my-test-image-uuid> --url cinder://<testvol-uuid>`
7. Create a new Cinder volume from the new image:
`openstack volume create testvol2 --image <my-test-image-uuid> --size 5` <-- This happen successfully
8. Delete testvol
`cinder delete <testvol-uuid>`
9. Perform step #7 again, this time it will fail with an HTTP 500 error in the cinder volume logs
:::code
2017-01-09 22:52:16.294 ERROR oslo_messaging.rpc.server [req-7b4326c9-0b45-43c3-bc04-618a6e4b5477 4077fa82897042c28b1765ee04c84d7c 89788f833b5d4c00b47626919c977b2d] Exception during message handling
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 4367, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server allow_reschedule=allow_reschedule, volume=volume)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 635, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _run_flow()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 627, in _run_flow
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server flow_engine.run()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failures[0].reraise()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(*self._exc_info)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = task.execute(**arguments)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 842, in execute
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server **volume_spec)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 761, in _create_from_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service, context, image_id) as tmp_image:
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self.gen.next()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 567, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server fetch_verify_image(context, image_service, image_id, tmp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 212, in fetch_verify_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server None, None)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 190, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service.download(context, image_id, image_file)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 335, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _reraise_translated_image_exception(image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 572, in _reraise_translated_image_exception
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(type(new_exc), new_exc, exc_trace)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 333, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_chunks = self._client.call(context, 'data', image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 197, in call
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 195, in data
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body = self.http_client.get(url)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 287, in get
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._request('GET', url, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 279, in _request
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body_iter = self._handle_response(resp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 107, in _handle_response
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server raise exc.from_response(resp, resp.content)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server HTTPInternalServerError: HTTPInternalServerError (HTTP 500)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server
Because the image in glance still lists this volume as a valid location
`glance image-show <my-test-image-uuid>` |
|
| 2017-01-09 23:09:57 |
Matt Smith |
description |
If you create a cinder-volume-backed image in glance, then delete the volume the image in glance shows as being valid even though the backing volume is now invalid.
I think ideally glance should have some knowledge of which volume locations are valid otherwise you could end up with an orphaned image if the backing location is invalidated.
Reproduction Steps:
Follow the steps outlined on this page to enable the Glance volume-backed image-cache:
http://docs.openstack.org/admin-guide/blockstorage-volume-backed-image.html
1. Make sure glance-api.conf has these lines:
`stores = file, http, cinder`
`show_multiple_locations = True`
2. Make sure cinder.conf has these lines in DEFAULT
`glance_api_version = 2`
`allowed_direct_url_schemes = cinder`
3. Make sure current backend section in cinder.conf has these lines:
`image_upload_use_cinder_backend = True`
`image_upload_use_internal_tenant = True`
4. Restart glance and cinder services
5. Create a Cinder volume and write some data to it (the data writing isn't really necessary for the bug though)
`cinder create 5 --display-name testvol`
6. Create a glance image that has this volume as a location:
`openstack image create --disk-format raw --container-format bare my-test-image`
`glance location-add <my-test-image-uuid> --url cinder://<testvol-uuid>`
7. Create a new Cinder volume from the new image:
`openstack volume create testvol2 --image <my-test-image-uuid> --size 5` <-- This happen successfully
8. Delete testvol
`cinder delete <testvol-uuid>`
9. Perform step #7 again, this time it will fail with an HTTP 500 error in the cinder volume logs
:::code
2017-01-09 22:52:16.294 ERROR oslo_messaging.rpc.server [req-7b4326c9-0b45-43c3-bc04-618a6e4b5477 4077fa82897042c28b1765ee04c84d7c 89788f833b5d4c00b47626919c977b2d] Exception during message handling
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 4367, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server allow_reschedule=allow_reschedule, volume=volume)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 635, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _run_flow()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 627, in _run_flow
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server flow_engine.run()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failures[0].reraise()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(*self._exc_info)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = task.execute(**arguments)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 842, in execute
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server **volume_spec)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 761, in _create_from_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service, context, image_id) as tmp_image:
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self.gen.next()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 567, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server fetch_verify_image(context, image_service, image_id, tmp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 212, in fetch_verify_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server None, None)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 190, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service.download(context, image_id, image_file)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 335, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _reraise_translated_image_exception(image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 572, in _reraise_translated_image_exception
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(type(new_exc), new_exc, exc_trace)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 333, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_chunks = self._client.call(context, 'data', image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 197, in call
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 195, in data
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body = self.http_client.get(url)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 287, in get
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._request('GET', url, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 279, in _request
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body_iter = self._handle_response(resp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 107, in _handle_response
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server raise exc.from_response(resp, resp.content)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server HTTPInternalServerError: HTTPInternalServerError (HTTP 500)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server
Because the image in glance still lists this volume as a valid location
`glance image-show <my-test-image-uuid>` |
If you create a cinder-volume-backed image in glance, then delete the volume the image in glance shows as being valid even though the backing volume is now invalid.
I think ideally glance should have some knowledge of which volume locations are valid otherwise you could end up with an orphaned image if the backing location is invalidated.
Reproduction Steps:
Follow the steps outlined on this page to enable the Glance volume-backed image-cache:
http://docs.openstack.org/admin-guide/blockstorage-volume-backed-image.html
1. Make sure glance-api.conf has these lines:
`stores = file, http, cinder`
`show_multiple_locations = True`
2. Make sure cinder.conf has these lines in DEFAULT
`glance_api_version = 2`
`allowed_direct_url_schemes = cinder`
3. Make sure current backend section in cinder.conf has these lines:
`image_upload_use_cinder_backend = True`
`image_upload_use_internal_tenant = True`
4. Restart glance and cinder services
5. Create a Cinder volume and write some data to it (the data writing isn't really necessary for the bug though)
`cinder create 5 --display-name testvol`
6. Create a glance image that has this volume as a location:
`openstack image create --disk-format raw --container-format bare my-test-image`
`glance location-add <my-test-image-uuid> --url cinder://<testvol-uuid>`
7. Create a new Cinder volume from the new image:
`openstack volume create testvol2 --image <my-test-image-uuid> --size 5` <-- This happen successfully
8. Delete testvol
`cinder delete <testvol-uuid>`
9. Perform step #7 again, this time it will fail with an HTTP 500 error in the cinder volume logs
2017-01-09 22:52:16.294 ERROR oslo_messaging.rpc.server [req-7b4326c9-0b45-43c3-bc04-618a6e4b5477 4077fa82897042c28b1765ee04c84d7c 89788f833b5d4c00b47626919c977b2d] Exception during message handling
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server Traceback (most recent call last):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 4367, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server allow_reschedule=allow_reschedule, volume=volume)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 635, in create_volume
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _run_flow()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/manager.py", line 627, in _run_flow
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server flow_engine.run()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server failures[0].reraise()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 343, in reraise
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(*self._exc_info)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server result = task.execute(**arguments)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 842, in execute
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server **volume_spec)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 761, in _create_from_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service, context, image_id) as tmp_image:
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self.gen.next()
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 567, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server fetch_verify_image(context, image_service, image_id, tmp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 212, in fetch_verify_image
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server None, None)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/image_utils.py", line 190, in fetch
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_service.download(context, image_id, image_file)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 335, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server _reraise_translated_image_exception(image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 572, in _reraise_translated_image_exception
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server six.reraise(type(new_exc), new_exc, exc_trace)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 333, in download
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server image_chunks = self._client.call(context, 'data', image_id)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/opt/stack/cinder/cinder/image/glance.py", line 197, in call
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/v2/images.py", line 195, in data
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body = self.http_client.get(url)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 287, in get
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server return self._request('GET', url, **kwargs)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 279, in _request
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server resp, body_iter = self._handle_response(resp)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server File "/usr/local/lib/python2.7/dist-packages/glanceclient/common/http.py", line 107, in _handle_response
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server raise exc.from_response(resp, resp.content)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server HTTPInternalServerError: HTTPInternalServerError (HTTP 500)
2017-01-09 22:52:16.294 TRACE oslo_messaging.rpc.server
Because the image in glance still lists this volume as a valid location
`glance image-show <my-test-image-uuid>` |
|
| 2020-01-01 00:38:00 |
MarkMielke |
bug |
|
|
added subscriber MarkMielke |
