Inconsistent behaviour when unicode args used for logging

[Konstantin]

When i used unicode args for log message, i got UnicodeDecodeError
If replace using oslo_log to logging from std python library, logging works fine

[1] Sample script, logging with oslo_log module http://paste.openstack.org/show/492234/
[2] Sample script, logging with Python logging module http://paste.openstack.org/show/492234/
[3] Sample file with unicode data http://paste.openstack.org/show/492235/

How to reproduce:

1 Save [3] as 'text.txt'
2 Run [1]

Expected result:

File log.txt contains
"жлдоыфв
фыжваофждыов"

Actual result:

Exception "UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)
Logged from file oslo_test.py, line 9"

$ pip freeze|grep oslo.log

oslo.log==3.2.0

[Andrey Volkov (avolkov)]

Can be easily reproduced with:
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
LOG.info("Oslo Logging %s", u'\u2622'.encode('utf8'))

The issue is related to implicit decoding of logging arguments due
_ensure_unicode for message:
https://github.com/openstack/oslo.log/blob/master/oslo_log/log.py#L129

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/323677

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/323677
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=991d8f0a125b7f421c597c0a3af7df9b10c4de19
Submitter: Jenkins
Branch: master

commit 991d8f0a125b7f421c597c0a3af7df9b10c4de19
Author: Andrey Volkov <email address hidden>
Date: Tue May 31 20:14:40 2016 +0300

    Make available to log encoded strings as arguments

    Pure python logging library allows to log encoded strings
    but in oslo_log it causes UnicodeDecodeError for arguments
    which contains unicode symbols.

    Changes add additional check whether all arguments can be converted to
    unicode. If it isn't possible then leave it as is.

    Change-Id: I76e75c35a74b146fae52c0ea60156e1cf78540e7
    Closes-Bug: #1563295

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/oslo.log 3.9.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/371582

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/371582
Committed: https://git.openstack.org/cgit/openstack/oslo.log/commit/?id=a2e7f6597ee7f57936cdf2b760068e556cad9a59
Submitter: Jenkins
Branch: stable/mitaka

commit a2e7f6597ee7f57936cdf2b760068e556cad9a59
Author: Andrey Volkov <email address hidden>
Date: Tue May 31 20:14:40 2016 +0300

    Make available to log encoded strings as arguments

    Pure python logging library allows to log encoded strings
    but in oslo_log it causes UnicodeDecodeError for arguments
    which contains unicode symbols.

    Changes add additional check whether all arguments can be converted to
    unicode. If it isn't possible then leave it as is.

    Change-Id: I76e75c35a74b146fae52c0ea60156e1cf78540e7
    Closes-Bug: #1563295