Exception messages not complete when containing unicode strings

[Sylvain Afchain]

There is an exception during the formatting of exception messages when values contain unicode strings.

ex:

Connection to neutron failed: %(reason)s

Due to this issue, the exception message is the unformatted message.

neutronclient/common/exceptions.py

 37 try:
 38 self._error_string = self.message % kwargs
 39
 40 except Exception:
 41 # at least get the core message out if something happened
 42 self._error_string = self.message

[Akihiro Motoki (amotoki)]

It rarely happens in the current neutronclient code, but it happens.

In [1]: from neutronclient.common import exceptions as nexc

In [2]: unicode_str = 'あああ'

In [3]: unicode_str
Out[4]: '\xe3\x81\x82\xe3\x81\x82\xe3\x81\x82'

In [4]: e = nexc.ConnectionFailed(reason=unicode_str)

In [5]: print e
Connection to neutron failed: %(reason)s

[Akihiro Motoki (amotoki)]

When I changed

    def __init__(self, message=None, **kwargs):
        if message:
            self.message = message
        try:
            self._error_string = self.message % kwargs
        except Exception:
            # at least get the core message out if something happened
            self._error_string = self.message

to

    def __init__(self, message=None, **kwargs):
        if message:
            self.message = message
        self._error_string = self.message % kwargs

I got the following error message:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 0: ordinal not in range(128)

[Akihiro Motoki (amotoki)]

@Mannu

I have a solution on this.
If you haven't figured out how to handle it, I can fix it.

[Mannu (mannu-ray)]

@Akiro

Please go ahead, I was unable to work weekend. So if you have a solution
please fix it.

Regards
Mannu
On 5 Dec 2015 13:50, "Akihiro Motoki" <email address hidden> wrote:

> @Mannu
>
> I have a solution on this.
> If you haven't figured out how to handle it, I can fix it.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1235228
>
> Title:
> Exception messages not complete when containing unicode strings
>
> Status in python-neutronclient:
> Confirmed
>
> Bug description:
> There is an exception during the formatting of exception messages when
> values contain unicode strings.
>
> ex:
>
> Connection to neutron failed: %(reason)s
>
>
> Due to this issue, the exception message is the unformatted message.
>
> neutronclient/common/exceptions.py
>
> 37 try:
> 38 self._error_string = self.message % kwargs
> 39
> 40 except Exception:
> 41 # at least get the core message out if something happened
> 42 self._error_string = self.message
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/python-neutronclient/+bug/1235228/+subscriptions
>

[Akihiro Motoki (amotoki)]

I investigated more detail.

The reported situation only happens when we pass unicode multibyte string without 'u' prefix. ([8][9] in the example below).
The unicode stirng should be prefixed with 'u'. Strictly speaking, it is invalid without 'u'.
Thus this bug is invalid.

On the other hand, [10][11] is a valid case, but we can't print().
print(e) raises an UnicodeEncodeError ([12]).
This means __str__ method in NeutronException class should encode an unicode string.
It needs to be fixed.

In [1]: from neutronclient.common import exceptions as nexc

In [4]: e = nexc.ConnectionFailed(reason='aaa')

In [5]: e._error_string
Out[5]: u'Connection to neutron failed: aaa'

In [6]: e = nexc.ConnectionFailed(reason=u'aaa')

In [7]: e._error_string
Out[7]: u'Connection to neutron failed: aaa'

In [8]: e = nexc.ConnectionFailed(reason='あああ')

In [9]: e._error_string
Out[9]: u'Connection to neutron failed: %(reason)s'

In [10]: e = nexc.ConnectionFailed(reason=u'あああ')

In [11]: e._error_string
Out[11]: u'Connection to neutron failed: \u3042\u3042\u3042'

In [12]: print(e)
---------------------------------------------------------------------------
UnicodeEncodeError Traceback (most recent call last)
<ipython-input-12-bb2ea196f768> in <module>()
----> 1 print(e)

UnicodeEncodeError: 'ascii' codec can't encode characters in position 30-32: ordinal not in range(128)

In [13]: print('%s' % e)
Connection to neutron failed: あああ

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/255140

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/255141

[Armando Migliaccio (armando-migliaccio)]

This bug is > 180 days without activity. We are unsetting assignee and milestone and setting status to Incomplete in order to allow its expiry in 60 days.

If the bug is still valid, then update the bug status.

[Launchpad Janitor (janitor)]

[Expired for python-neutronclient because there has been no activity for 60 days.]

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/255140
Committed: https://git.openstack.org/cgit/openstack/python-neutronclient/commit/?id=d0e50b8b90fa29db04e0c19ea699f6bc7ae16652
Submitter: Jenkins
Branch: master

commit d0e50b8b90fa29db04e0c19ea699f6bc7ae16652
Author: Akihiro Motoki <email address hidden>
Date: Wed Dec 9 06:07:50 2015 +0900

    Use six.python_2_unicode_compatible for NeutronException.__str__

    The definition of __str__ is different between py2 and py3.
    __str__ should return a byte stream to make print(exc) work
    in python2, but __str__ should return string type in python3.
    six.python_2_unicode_compatible sets up __str__ and __unicode__
    approriately.

    Change-Id: I6c85ca8c4e6f86450ee390db81e3aa101293b846
    Related-Bug: #1235228

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/255141
Committed: https://git.openstack.org/cgit/openstack/python-neutronclient/commit/?id=266e6c3c3c3e9a8b856d99238dc5323edeb66955
Submitter: Jenkins
Branch: master

commit 266e6c3c3c3e9a8b856d99238dc5323edeb66955
Author: Akihiro Motoki <email address hidden>
Date: Wed Dec 9 06:22:03 2015 +0900

    Decode exception parameters to expand exception message properly

    If a multibyte string without unicode prefix 'u' is passed to
    NeutronException in python 2, string subsitution will fail.
    In python 2, multibyte string without 'u' prefix is encoded into bytes.
    As a result a string substitution in NeutronException.__init__ will fail.

    Change-Id: I1aa08b69fe119087a7a49fda768a24f85f0e7c76
    Closes-Bug: #1235228

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-neutronclient 6.1.0 release.