UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
402: ordinal not in range(128)
when attempting to log an Instance object with a unicode display name.
This resurfaced relatively recently because of the change in devstack
to use the new OSJournalHandler with use_journal=True which is
suspected of causing some deadlock issues [1] unrelated to this bug.
The problem occurs in code that logs an entire Instance object when
the object has a field with unicode characters in it (display_name).
When the object is sent to logging, the UnicodeDecodeError is raised
while formatting the log record here [2]. This implies an implicit
conversion attempt to unicode at this point.
I found that with the Instance object, the conversion to unicode fails
with the UnicodeDecodeError unless the encoding 'utf-8' is explicitly
specified to six.text_type(). And when specifying an encoding to
six.text_type(), the argument to convert must be a string, not an
Instance object, so this does the conversion in two steps as a utility
function:
1. Get the string representation of the Instance with repr()
2. Call six.text_type(instance_repr, 'utf-8') passing the encoding
if not six.PY3
Reviewed: https:/ /review. openstack. org/461912 /git.openstack. org/cgit/ openstack/ nova/commit/ ?id=564958dba6c 280eb8e11ae9bbc 819c7da6e204bd
Committed: https:/
Submitter: Jenkins
Branch: master
commit 564958dba6c280e b8e11ae9bbc819c 7da6e204bd
Author: melanie witt <email address hidden>
Date: Tue May 2 21:47:12 2017 +0000
Use six.text_type() when logging Instance object
We're seeing a trace in gate jobs, for example:
UnicodeDe codeError: 'ascii' codec can't decode byte 0xc3 in position
402: ordinal not in range(128)
when attempting to log an Instance object with a unicode display name.
This resurfaced relatively recently because of the change in devstack
to use the new OSJournalHandler with use_journal=True which is
suspected of causing some deadlock issues [1] unrelated to this bug.
The problem occurs in code that logs an entire Instance object when
the object has a field with unicode characters in it (display_name).
When the object is sent to logging, the UnicodeDecodeError is raised
while formatting the log record here [2]. This implies an implicit
conversion attempt to unicode at this point.
I found that with the Instance object, the conversion to unicode fails text_type( ), the argument to convert must be a string, not an
with the UnicodeDecodeError unless the encoding 'utf-8' is explicitly
specified to six.text_type(). And when specifying an encoding to
six.
Instance object, so this does the conversion in two steps as a utility
function:
1. Get the string representation of the Instance with repr() type(instance_ repr, 'utf-8') passing the encoding
2. Call six.text_
if not six.PY3
Closes-Bug: #1580728
[1] https:/ /review. openstack. org/#/c/ 462163 /github. com/python/ cpython/ blob/2e576f5/ Lib/logging/ __init_ _.py#L338
[2] https:/
Change-Id: I0fc3ae02cb2e40 1b3240faf0d8b6a a5dc52b91fc