OpenStack Compute (nova)

Fail to fetch non-raw image on image_backend=RBD

Bug #1370387 reported by Sébastien Han on 2014-09-17

This bug report is a duplicate of: Bug #1371040: glance store rbd driver get method returns tupe more than basic driver defines. Edit Remove

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Confirmed	Low	Unassigned

Bug Description

Using the image_backend RBD and booting a non-raw image results in a fallback to fetch_to_raw function where the goal is to download the qcow2 image on the compute, convert it into raw and import it into ceph.
Fetching the image fails with the following errors:

On the nova-compute logs:

2014-09-16 10:16:23.061 ERROR nova.compute.manager [req-4c42af30-bc04-4648-bd12-14419031be80 admin admin] [instance: 18d8659a-0247-4472-97be-9c6c9007689b] Instance failed to spaw
n
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] Traceback (most recent call last):
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/compute/manager.py", line 2203, in _build_resourc
es
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] yield resources
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/compute/manager.py", line 2082, in _build_and_run
_instance
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] block_device_info=block_device_info)
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2598, in spawn
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] admin_pass=admin_password)
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2974, in _create_im
age
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] project_id=instance['project_id'])2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 206, in cache2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] *args, **kwargs)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 396, in create_image2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] prepare_template(target=base, max_size=size, *args, **kwargs)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 270, in inner2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] return f(*args, **kwargs)
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 196, in fetch_func_sync
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] fetch_func(target=target, *args, **kwargs)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/libvirt/utils.py", line 452, in fetch_image
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] max_size=max_size)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/images.py", line 73, in fetch_to_raw
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] max_size=max_size)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/virt/images.py", line 67, in fetch
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] IMAGE_API.download(context, image_href, dest_path=path)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/image/api.py", line 178, in download
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] dst_path=dest_path)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/image/glance.py", line 352, in download
2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] _reraise_translated_image_exception(image_id)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/image/glance.py", line 350, in download2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] image_chunks = self._client.call(context, 1, 'data', image_id)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/nova/nova/image/glance.py", line 225, in call2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] return getattr(client.images, method)(*args, **kwargs)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/python-glanceclient/glanceclient/v1/images.py", line 142, in data2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] % urlparse.quote(str(image_id)))2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/python-glanceclient/glanceclient/common/http.py", line 253, in get2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] return self._request('GET', url, **kwargs)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] File "/opt/stack/python-glanceclient/glanceclient/common/http.py", line 221, in _request2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] raise exc.from_response(resp, resp.content)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b] HTTPInternalServerError: HTTPInternalServerError (HTTP 500)2014-09-16 10:16:23.061 TRACE nova.compute.manager [instance: 18d8659a-0247-4472-97be-9c6c9007689b]

On the glance-api logs:

2014-09-16 10:41:43.728 19722 INFO glance.wsgi.server [-] Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 433, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 394, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/osprofiler/web.py", line 106, in __call__
    return request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token.py", line 646, in __call__
    return self._call_app(env, start_response)
  File "/usr/local/lib/python2.7/dist-packages/keystonemiddleware/auth_token.py", line 624, in _call_app
    return self._app(env, _fake_start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 394, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 394, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 394, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 206, in __call__
    return app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
    response = self.app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 683, in __call__
    request, **action_args)
  File "/opt/stack/glance/glance/common/wsgi.py", line 707, in dispatch
    return method(*args, **kwargs)
  File "/opt/stack/glance/glance/api/v1/images.py", line 494, in show
    image_meta['location'])
  File "/opt/stack/glance/glance/api/v1/images.py", line 456, in _get_from_store
    image_data, image_size = src_store.get(loc, context=context)
ValueError: too many values to unpack

Using glance v2 should work, however it used to work on v1.

Tags:

Sébastien Han (sebastien-han) on 2014-09-17

tags:

added: ceph rbd

Sean Dague (sdague) on 2014-09-17

Changed in nova:
status:	New → Confirmed
importance:	Undecided → Medium
importance:	Medium → Low

Revision history for this message

Dmitry Borodaenko (angdraug) wrote on 2014-09-22:

The stack trace implies that CONF.glance.allowed_direct_url_schemes is True in nova/image/glance.py:339, isn't that only allowed with v2?

Revision history for this message

Yaguang Tang (heut2008) wrote on 2014-11-21:

The glance traceback shows that it's a bug in glance_store. This is actually a duplicate of https://bugs.launchpad.net/glance/+bug/1371040.
Please update your glance store to >=0.1.9 which fixes this bug.

Report a bug

This report contains Public information

Everyone can see this information.

Duplicate of bug #1371040 Remove

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.