Comment 9 for bug 1389102

Revision history for this message
Qin Zhao (zhaoqin) wrote :

Need to take care of utf-8 byte character size when truncating the byte string. For example, '\xd0\x92' is a 2 bytes character. UnicodeDecodeError will occur, if we truncate it in the middle:

>>> a = '\xd0\x92' * 150
>>> encodeutils.safe_decode(a[:255])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/oslo/utils/encodeutils.py", line 57, in safe_decode
    return text.decode('utf-8', errors)
  File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd0 in position 254: unexpected end of data