Store ID fetched from URI is incorrectly encoded under py27

[Alan Bishop]

Whenever a client updates the locations for an image, glance checks to be sure the locations metadata includes the backend store ID that it parses from the request URI. However, under py27 the parsed store ID will not be properly encoded (it will be a str, but needs to be unicode). This leads to a subsequent failure, when glance checks to verify the value isinstance(val, six.text_type) at [1].

The problem can be triggered by having nova create an image from a running VM. Under these circumstances, the store ID parsed by [2] and added at [3] will be encoded incorrectly.

[1] https://opendev.org/openstack/glance_store/src/branch/master/glance_store/backend.py#L381
[2] https://opendev.org/openstack/glance/src/branch/master/glance/common/store_utils.py#L159
[3] https://opendev.org/openstack/glance/src/branch/master/glance/common/store_utils.py#L195

This leads to the following error:

2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi [req-81008491-28d6-442c-b95e-9b0d2f108213 11d8e44ae836459cbc32cd31d452f068 14efbf135f20447f8f45815b8dcb5249 - default default] Caught error: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.: BackendException: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi Traceback (most recent call last):
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1474, in __call__
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi request, **action_args)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1517, in dispatch
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi return method(*args, **kwargs)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 417, in wrapped
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 256, in update
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi change_method(req, image, change)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 309, in _do_add
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi self._do_add_locations(image, path[1], value)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/images.py", line 565, in _do_add_locations
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi image.locations.insert(pos, value)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 297, in _checker
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi return method(*args, **kwargs)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 292, in insert
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi return self.locations.insert(index, object)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/location.py", line 231, in insert2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi location)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/location.py", line 135, in _check_image_location
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi store_api.check_location_metadata(location['metadata'])
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 375, in check_location_metadata
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi check_location_metadata(val[key], key=key)
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 386, in check_location_metadata
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi % dict(key=key, type=type(val)))
2020-01-30 14:12:54.909 64 ERROR glance.common.wsgi BackendException: The image metadata key store has an invalid type of <type 'str'>. Only dict, list, and unicode are supported.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/705229

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/705229
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=8649fdc2a293009899d2aa918ec360e30adc8e77
Submitter: Zuul
Branch: master

commit 8649fdc2a293009899d2aa918ec360e30adc8e77
Author: Alan Bishop <email address hidden>
Date: Fri Jan 31 06:25:31 2020 -0800

    Ensure store ID parsed from URI is properly encoded

    Ensure the store ID parsed from a request URI is unicode. This fixes
    a py27 issue that allowed an incorrectly encoded value to be added to
    the image's metadata.

    Support for py27 is dropped in Ussuri, but this fix is required on
    earlier releases that do support py27.

    Closes-Bug: #1861501
    Change-Id: I5a1c61430879a910e7b6c79effba538431959d56

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/705272

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/705272
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=c5bdc5536690f5b3ae44c4d9ae2e96470287eb56
Submitter: Zuul
Branch: stable/train

commit c5bdc5536690f5b3ae44c4d9ae2e96470287eb56
Author: Alan Bishop <email address hidden>
Date: Fri Jan 31 06:25:31 2020 -0800

    Ensure store ID parsed from URI is properly encoded

    Ensure the store ID parsed from a request URI is unicode. This fixes
    a py27 issue that allowed an incorrectly encoded value to be added to
    the image's metadata.

    Support for py27 is dropped in Ussuri, but this fix is required on
    earlier releases that do support py27.

    Closes-Bug: #1861501
    Change-Id: I5a1c61430879a910e7b6c79effba538431959d56
    (cherry picked from commit 8649fdc2a293009899d2aa918ec360e30adc8e77)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/glance 20.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/glance 19.0.2 release.