Glance images can contain no data

[Stephen Finucane]

Due to another bug [1], glance was returning None from 'glanceclient.v2.images.Controller.data'. However, the glance documentation states that this is a valid return value. We should handle this. Logs below.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1476448
[2] https://docs.openstack.org/python-glanceclient/latest/reference/api/glanceclient.v2.images.html#glanceclient.v2.images.Controller.data

---

2017-08-15 17:34:01.677 1 ERROR nova.image.glance [req-70546b57-a282-4552-8b9e-65be1871825a bd800a91d263411393899aff269084a0 aaed41f2e25f494c9fadd01c340f25c8 - default default] Error writing to /var/lib/nova/instances/_base/cae3a4306eeb5643cb6caffbe1e3050645f8aee2.part: 'NoneType' object is not iterable
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [req-70546b57-a282-4552-8b9e-65be1871825a bd800a91d263411393899aff269084a0 aaed41f2e25f494c9fadd01c340f25c8 - default default] [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] Instance failed to spawn: TypeError: 'NoneType' object is not iterable
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] Traceback (most recent call last):
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2125, in _build_resources
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] yield resources
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _build_and_run_instance
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] block_device_info=block_device_info)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2793, in spawn
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] block_device_info=block_device_info)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3231, in _create_image
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] fallback_from_host)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3322, in _create_and_inject_local_root
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] instance, size, fallback_from_host)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6968, in _try_fetch_image_cache
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] size=size)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 241, in cache
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] *args, **kwargs)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 595, in create_image
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] prepare_template(target=base, *args, **kwargs)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] return f(*args, **kwargs)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 237, in fetch_func_sync
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] fetch_func(target=target, *args, **kwargs)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 446, in fetch_image
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] images.fetch_to_raw(context, image_id, target)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/images.py", line 134, in fetch_to_raw
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] fetch(context, image_href, path_tmp)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/virt/images.py", line 125, in fetch
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] IMAGE_API.download(context, image_href, dest_path=path)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/image/api.py", line 184, in download
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] dst_path=dest_path)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 369, in download
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] {'path': dst_path, 'exception': ex})
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] self.force_reraise()
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] six.reraise(self.type_, self.value, self.tb)
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] File "/usr/lib/python2.7/site-packages/nova/image/glance.py", line 353, in download
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] for chunk in image_chunks:
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0] TypeError: 'NoneType' object is not iterable
2017-08-15 17:34:01.679 1 ERROR nova.compute.manager [instance: c3fc31f1-28ab-47bf-a08b-65cfc6ab2ce0]

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/526328

[OpenStack Infra (hudson-openstack)]

Change abandoned by Stephen Finucane (<email address hidden>) on branch: master
Review: https://review.openstack.org/526328
Reason: Yeah. Turns out this isn't as necessary for the follow-up patch as I thought it was anyway

[Stephen Finucane (stephenfinucane)]

Turns out this also affect the CLI from python-glanceclient.

---

$ glance image-create --name "test"
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | None |
| container_format | None |
| created_at | 2017-12-08T16:57:11Z |
| disk_format | None |
| id | 0393d2eb-67af-428f-8d09-031b325b2569 |
| min_disk | 0 |
| min_ram | 0 |
| name | test |
| owner | d8e31a82d0a24acfbeca3ddc825e1cab |
| protected | False |
| size | None |
| status | queued |
| tags | [] |
| updated_at | 2017-12-08T16:57:11Z |
| virtual_size | None |
| visibility | shared |
+------------------+--------------------------------------+
$ echo $1
0
$ glance image-download 0393d2eb-67af-428f-8d09-031b325b2569 --file test.out
'NoneType' object is not iterable

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/527370
Committed: https://git.openstack.org/cgit/openstack/python-glanceclient/commit/?id=4dcbc30e317a25495bebc073bb9913d9fd9d43a2
Submitter: Zuul
Branch: master

commit 4dcbc30e317a25495bebc073bb9913d9fd9d43a2
Author: Stephen Finucane <email address hidden>
Date: Tue Dec 12 10:57:24 2017 +0000

    Compare against 'RequestIdProxy.wrapped'

    Due to the 'glanceclient.common.utils.add_req_id_to_object' decorator,
    an instance of 'glanceclient.common.utils.RequestIdProxy' is returned
    for most calls in glanceclient. If we wish to compare to None, we have
    to compare the contents of this wrapper and not the wrapper itself.

    Unit tests are updated to highlight this.

    Change-Id: I7dadf32d37ac2bda33a92c71d5882e9f23e38a82
    Closes-Bug: #1736759

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-glanceclient 2.9.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/526329
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c10a614e92d15280f05574d82bbede6df6aaeec6
Submitter: Zuul
Branch: master

commit c10a614e92d15280f05574d82bbede6df6aaeec6
Author: Stephen Finucane <email address hidden>
Date: Wed Dec 6 17:30:49 2017 +0000

    Handle images with no data

    There isn't really much we can do with these images, which glance tells
    us are possible [1]. Simply raise an exception.

    [1] https://docs.openstack.org/python-glanceclient/latest/reference/api/glanceclient.v2.images.html

    Change-Id: I5f81393a5bb41e6a674369afb899d8a41bb2c3b4
    Closes-Bug: #1736759

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/538939

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/538939
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b72cefee4a14964cddc7a75de6d555f876fc2797
Submitter: Zuul
Branch: stable/pike

commit b72cefee4a14964cddc7a75de6d555f876fc2797
Author: Stephen Finucane <email address hidden>
Date: Wed Dec 6 17:30:49 2017 +0000

    Handle images with no data

    There isn't really much we can do with these images, which glance tells
    us are possible [1]. Simply raise an exception.

    [1] https://docs.openstack.org/python-glanceclient/latest/reference/api/glanceclient.v2.images.html

    Change-Id: I5f81393a5bb41e6a674369afb899d8a41bb2c3b4
    Closes-Bug: #1736759
    (cherry picked from commit c10a614e92d15280f05574d82bbede6df6aaeec6)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 17.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.1.0 release.