Type "str" test in Python2 code doesn't work as expected in Python3

Bug #1890477 reported by Tristan Zhang
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Karbor
New
Undecided
Unassigned

Bug Description

After upgraded to Ussuri version, we got an error:

```
 [-] Creating image backup, image_id: 27ac514a-2c60-4f37-9e48-500d78a7946f.
 [-] Protecting image (id: 27ac514a-2c60-4f37-9e48-500d78a7946f) to bank failed.: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 0: invalid continuation byte
 Traceback (most recent call last):
.py", line 199, in _create_backup
     self._data_block_size_bytes
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/protection_plugins/utils.py", line 37, in backup_image_to_bank
     bank_section.update_object("data_" + str(chunks_num), data)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/bank_plugin.py", line 241, in update_object
     context=context
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/bank_plugin.py", line 120, in update_object
     context=context)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/karbor/services/protection/bank_plugins/swift_bank_plugin.py", line 118, in update_object
     value = jsonutils.dumps(value)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_serialization/jsonutils.py", line 201, in dumps
     return json.dumps(obj, default=default, **kwargs)
   File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
     **kw).encode(obj)
   File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
     chunks = self.iterencode(o, _one_shot=True)
   File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
     return _iterencode(o, 0)
   File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_serialization/jsonutils.py", line 106, in to_primitive
     return value.decode(encoding=encoding)
 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 0: invalid continuation byte
```

It seems to be caused by the migration from Python2 to Python3.

In the errors above, we can see that it tried to decode some binary data (one chunk of image fetched from glance). The binary data is bytes type in Python3, but it's not unicode data.

In "karbor/services/protection/bank_plugins/swift_bank_plugin.py", there is a function update_object.

https://github.com/openstack/karbor/blob/0764f5dab12034efdc41d35ccd627f2a6618c459/karbor/services/protection/bank_plugins/swift_bank_plugin.py#L117

```
    def update_object(self, key, value, context=None):
        serialized = False
        try:
            if not isinstance(value, str):
                value = jsonutils.dumps(value)
                serialized = True
            self._put_object(container=self.bank_object_container,
                             obj=key,
                             contents=value,
                             headers={
                                 'x-object-meta-serialized': str(serialized)
                             })
        except SwiftConnectionFailed as err:
            LOG.error("update object failed, err: %s.", err)
            raise exception.BankUpdateObjectFailed(reason=err, key=key)
```

We can see that it tests if "value" is "str" type. If not, it calls jsonutils.dumps to serialize it.

In Python2, binary data is str type. But in Python3, it's "bytes" type, not "str". So in Python3, we can't test if an object is binary in this way.

Tristan Zhang (tzmtl)
description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.