Cannot create volume from glance image without checksum

[git-harry]

It is no longer possible to create a volume from an image that does not have a checksum set.

https://github.com/openstack/cinder/commit/da13c6285bb0aee55cfbc93f55ce2e2b7d6a28f2 - this patch removes the default of None from the getattr call.

If this is intended it would be nice to see something more informative in the logs.

2014-04-15 11:52:26.035 19000 ERROR cinder.api.middleware.fault [req-cf0f7b89-a9c1-4a10-b1ac-ddf415a28f24 c139cd16ac474d2184237ba837a04141 83d5198d5f5a461798c6b843f57540d
f - - -] Caught error: checksum
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault Traceback (most recent call last):
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/api/middleware/fault.py", line 75, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return req.get_response(self.application)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault application, catch_exc_info=False)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return resp(environ, start_response)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 615, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return self.app(env, start_response)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return resp(environ, start_response)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return resp(environ, start_response)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault response = self.app(environ, start_response)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return resp(environ, start_response)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault resp = self.call_func(req, *args, **self.kwargs)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/api/openstack/wsgi.py", line 895, in __call__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault content_type, body, accept)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/api/openstack/wsgi.py", line 943, in _process_stack
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault action_result = self.dispatch(meth, request, action_args)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/api/openstack/wsgi.py", line 1019, in dispatch
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return method(req=request, **action_args)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/api/v2/volumes.py", line 346, in create
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault **kwargs)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/volume/api.py", line 189, in create
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault flow_engine.run()
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/utils/lock_utils.py", line 54, in wrapper
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return f(*args, **kwargs)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 96, in run
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault self._run()
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 111, in _ru
n
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault misc.Failure.reraise_if_any(failures.values())
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/utils/misc.py", line 649, in reraise_if_any
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault failures[0].reraise()
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/utils/misc.py", line 656, in reraise
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault six.reraise(*self._exc_info)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 34, in _e
xecute_task
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault result = task.execute(**arguments)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 341, in execute
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault self._check_image_metadata(context, image_id, size)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 180, in _check_image_metadata
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault image_meta = self.image_service.show(context, image_id)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/image/glance.py", line 228, in show
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault base_image_meta = self._translate_from_glance(image)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/image/glance.py", line 336, in _translate_from_glance
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault image_meta = _extract_attributes(image)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/cinder/cinder/image/glance.py", line 434, in _extract_attributes
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault output[attr] = getattr(image, attr)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/python-glanceclient/glanceclient/openstack/common/apiclient/base.py", line 462, in __getattr__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault return self.__getattr__(k)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault File "/opt/stack/python-glanceclient/glanceclient/openstack/common/apiclient/base.py", line 464, in __getattr__
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault raise AttributeError(k)
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault AttributeError: checksum
2014-04-15 11:52:26.035 19000 TRACE cinder.api.middleware.fault
2014-04-15 11:52:26.037 19000 INFO cinder.api.middleware.fault [req-cf0f7b89-a9c1-4a10-b1ac-ddf415a28f24 c139cd16ac474d2184237ba837a04141 83d5198d5f5a461798c6b843f57540df - - -] http://162.13.156.43:8776/v2/83d5198d5f5a461798c6b843f57540df/volumes returned with HTTP 500

[Mike Perez (thingee)]

Jumped onto this one too fast. How are you ending up with an image without a checksum? Just trying to reproduce this.

[git-harry (git-harry)]

By not specifying one when I do glance image-create.

[Mike Perez (thingee)]

You can create an image with a name and it'll attribute error on that as well.

[Openstack Gerrit (openstack-gerrit)]

Related fix proposed to branch: master
Review: https://review.openstack.org/88089

[Openstack Gerrit (openstack-gerrit)]

Fix proposed to branch: master
Review: https://review.openstack.org/88090

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/88089
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=993facb3bf753d23332236dc05b397850bf99144
Submitter: Jenkins
Branch: master

commit 993facb3bf753d23332236dc05b397850bf99144
Author: Mike Perez <email address hidden>
Date: Thu Apr 17 18:46:36 2014 -0700

    Fallback to None on missing Glance image attrs

    It's possible for glance images to be missing attributes like name and
    checksum. We'll just set those to None by default to avoid missing key
    exceptions being raised.

    Closes-Bug: #1308058
    Change-Id: I85c42f8351763da201021a22f5ff0ebd62c6b2db

[Openstack Gerrit (openstack-gerrit)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/90644

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/93649

[Andrew Woodward (xarses)]

https://review.openstack.org/70518 caused the same issue in nova

[Andrew Woodward (xarses)]

This was also found in fuel.

@dborodaenko posted in https://bugs.launchpad.net/fuel/+bug/1319046 (now a duplicate)

The problem is with this commit (in Nova and Cinder respectively):
https://review.openstack.org/70518
https://review.openstack.org/84937

As can be seen from cinder debug output, metadata for TestVM image doesn't have 'deleted' attribute:
2014-05-13T12:42:11.397595+00:00 debug: {"status": "active", "name": "TestVM", "tags": [], "checksum": "9d526403713579f446434dd6190f9e9e", "created_at": "2014-05-12T13:54:12Z", "size": 14811136, "murano_image_info": "{\"title\": \"Murano Demo\", \"type\": \"cirros.demo\"}", "updated_at": "2014-05-12T13:54:13Z", "visibility": "public", "self": "/v2/images/43f6b4be-b689-46b2-9dab-412e929d8b6f", "container_format": "bare", "min_disk": 0, "protected": false, "min_ram": 64, "file": "/v2/images/43f6b4be-b689-46b2-9dab-412e929d8b6f/file", "disk_format": "qcow2", "owner": "4d52167746af4dd58a65a0370a6efaa5", "direct_url": "rbd://7361c191-9dda-4646-8798-a226439911b9/images/43f6b4be-b689-46b2-9dab-412e929d8b6f/snap", "id": "43f6b4be-b689-46b2-9dab-412e929d8b6f", "schema": "/v2/schemas/image"}

Which causes AttributeError: deleted

[Vitaly Parakhin (vparakhin)]

cinder
http://gerrit.vm.mirantis.net:8080/#/c/15570/

Regarding nova - it appears that an appropriate patch already applied to our code - could you please check and confirm?

[Dmitry Borodaenko (angdraug)]

Vitaly, Nova stable/icehouse doesn't have the commit that has introduced this bug in the first place (it's only present in master), so we don't need to patch it. We only need to patch Cinder.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/93649
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ef3f158a5aa1988be8ab2f295252373f3ee85809
Submitter: Jenkins
Branch: master

commit ef3f158a5aa1988be8ab2f295252373f3ee85809
Author: Andrew Woodward <email address hidden>
Date: Wed May 14 19:22:48 2014 -0700

    Catch missing Glance image attrs with None

    Glance is known to drop values from its response if they don't resolve to true.
     Previously in I67b7dd16a94fe60d873c012f6bd246ab24500d5a the None catch was
     removed as this WILL cause a call to glance for missing value and
     optimizations where made to attempt to pre-cache values that can be missing
     in expected ways. However there are still situations that can't be optimized.

    Notably this can occur with disk_format, container_format, name, deleted,
     and checksum.

    Test adapted from I85c42f8351763da201021a22f5ff0ebd62c6b2db by Mike Perez

    Closes-bug #1308058
    Co-Author: Mike Perez <email address hidden>

    Change-Id: Icddead6f746d4d3ba652858eaae425acce8c177d

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/90644
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=1e488339ded20a8bce602202bde7244f1c1b249e
Submitter: Jenkins
Branch: stable/icehouse

commit 1e488339ded20a8bce602202bde7244f1c1b249e
Author: Mike Perez <email address hidden>
Date: Thu Apr 17 18:46:36 2014 -0700

    Fallback to None on missing Glance image attrs

    It's possible for glance images to be missing attributes like name and
    checksum. We'll just set those to None by default to avoid missing key
    exceptions being raised.

    Closes-Bug: #1308058
    Change-Id: I85c42f8351763da201021a22f5ff0ebd62c6b2db
    (cherry picked from commit 993facb3bf753d23332236dc05b397850bf99144)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/116425

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/116425
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e1d6e1881b08827d1be95e35ed064364d222659e
Submitter: Jenkins
Branch: stable/icehouse

commit e1d6e1881b08827d1be95e35ed064364d222659e
Author: Andrew Woodward <email address hidden>
Date: Wed May 14 19:22:48 2014 -0700

    Catch missing Glance image attrs with None

    Glance is known to drop values from its response if they don't resolve to true.
     Previously in I67b7dd16a94fe60d873c012f6bd246ab24500d5a the None catch was
     removed as this WILL cause a call to glance for missing value and
     optimizations where made to attempt to pre-cache values that can be missing
     in expected ways. However there are still situations that can't be optimized.

    Notably this can occur with disk_format, container_format, name, deleted,
     and checksum.

    Test adapted from I85c42f8351763da201021a22f5ff0ebd62c6b2db by Mike Perez

    Closes-bug #1308058
    Co-Author: Mike Perez <email address hidden>

    Change-Id: Icddead6f746d4d3ba652858eaae425acce8c177d
    (cherry picked from commit ef3f158a5aa1988be8ab2f295252373f3ee85809)