Problem: When trying to create a volume with a name having 'unicode' characters such as : flügel. The operation fails with this error: "Error decoding your request. Either the URL or the request body contained characters that could not be decoded by Cinder."
Traceback:
Flow 'volume_create_manager': UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager Traceback (most recent call last):
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager result = task.execute(**arguments)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 229, in execute
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager volume.refresh()
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 238, in refresh
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager super(CinderCleanableObject, self).refresh()
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/objects/base.py", line 486, in refresh
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager current = self.get_by_id(self._context, self.id)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/objects/base.py", line 352, in get_by_id
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager return cls._from_db_object(context, cls(context), orm_obj, **kargs)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/objects/volume.py", line 262, in _from_db_object
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager if isinstance(field, fields.IntegerField):
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 777, in __setitem__
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager setattr(self, name, value)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 72, in setter
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager field_value = field.coerce(self, name, value)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/fields.py", line 195, in coerce
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager return self._type.coerce(obj, attr, value)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/fields.py", line 262, in coerce
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager return six.text_type(value)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)
2019-05-02 07:29:54.805 9581 ERROR cinder.volume.manager
1) What triggered this problem?
https://github.com/openstack/oslo.db/commit/a6d2cc57d47f149398bf3817ebdfe15ff8d908e2?diff=split
changes in: oslo_db/sqlalchemy/engines.py
Remove convert_unicode flag
2) What is broken?
The fix mentioned above breaks this code.
https://github.com/openstack/oslo.versionedobjects/blob/master/oslo_versionedobjects/fields.py#L275
But, how?
Earlier, oslo_db used to return the unicode strings as unicode objects (due to convert_unicode flag).
Now, the changes mentioned in point-1 opened up scenarios for oslo_db to return the unicode strings as byte strings too.
And https://github.com/openstack/oslo.versionedobjects/blob/master/oslo_versionedobjects/fields.py#L270-L280 is not
handling the byte strings (utf encoded unicode strings).
3) What is the potential solution?
Changes in: oslo_versionedobjects/fields.py
class String(FieldType):
@staticmethod
def coerce(obj, attr, value):
# FIXME(danms): We should really try to avoid the need to do this
accepted_types = six.integer_types + (float, six.string_types,
datetime.datetime)
if isinstance(value, accepted_types):
if isinstance(value, six.binary_type): ---------------------------------- Potential Fix
return value.decode('utf-8')
return six.text_type(value)
else:
raise ValueError(_('A string is required in field %(attr)s, '
'not a %(type)s') %
{'attr': attr, 'type': type(value).__name__})
