Comment 0 for bug 1890477

Revision history for this message
Tristan Zhang (tzmtl) wrote :

After upgrade to Ussuri version, we got an error:

```
 [-] Creating image backup, image_id: 27ac514a-2c60-4f37-9e48-500d78a7946f.
 [-] Protecting image (id: 27ac514a-2c60-4f37-9e48-500d78a7946f) to bank failed.: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 0: invalid continuation byte
 Traceback (most recent call last):
.py", line 199, in _create_backup
     self._data_block_size_bytes
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/protection_plugins/utils.py", line 37, in backup_image_to_bank
     bank_section.update_object("data_" + str(chunks_num), data)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/bank_plugin.py", line 241, in update_object
     context=context
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/bank_plugin.py", line 120, in update_object
     context=context)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/bank_plugins/swift_bank_plugin.py", line 118, in update_object
     value = jsonutils.dumps(value)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_serialization/jsonutils.py", line 201, in dumps
     return json.dumps(obj, default=default, **kwargs)
   File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
     **kw).encode(obj)
   File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
     chunks = self.iterencode(o, _one_shot=True)
   File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
     return _iterencode(o, 0)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_serialization/jsonutils.py", line 106, in to_primitive
     return value.decode(encoding=encoding)
 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 0: invalid continuation byte
```

It seems to be caused by the migration from Python2 to Python3.

In the errors above, we can see that it tried to decode some binary data (one chunk of image fetched from glance). The binary data is bytes type in Python3, but it's not unicode data.

In "karbor/services/protection/bank_plugins/swift_bank_plugin.py", there is a function update_object.

https://github.com/openstack/karbor/blob/0764f5dab12034efdc41d35ccd627f2a6618c459/karbor/services/protection/bank_plugins/swift_bank_plugin.py#L117

```
    def update_object(self, key, value, context=None):
        serialized = False
        try:
            if not isinstance(value, str):
                value = jsonutils.dumps(value)
                serialized = True
            self._put_object(container=self.bank_object_container,
                             obj=key,
                             contents=value,
                             headers={
                                 'x-object-meta-serialized': str(serialized)
                             })
        except SwiftConnectionFailed as err:
            LOG.error("update object failed, err: %s.", err)
            raise exception.BankUpdateObjectFailed(reason=err, key=key)
```

We can see that it tests if "value" is "str" type. If not, it calls jsonutils.dumps to serialize it.

In Python2, binary data is str type. But in Python3, it's "bytes" type, not "str". So in Python3, we can't test if an object is binary in this way.