We should not bypass bytes decode/encode

[Cedric Brandily]

The commit fcf289797c063088f9003359dfd1c7d4f41ed5ef[1] introduces the pattern:

  if six.PY3:
    if isinstance(line, bytes):
      try:
        line = line.decode(encoding='utf-8')
      except UnicodeError:
        pass
 # concat line with a string

which is not working if a UnicodeError is raised: in such case line is not decoded and line is not converted in a string and the concatenation with the string fails. We should ensure that line is converted to a string

[1] https://github.com/openstack/python-neutronclient/commit/fcf289797c063088f9003359dfd1c7d4f41ed5ef
[2] https://github.com/openstack/python-neutronclient/blob/db7eb557403da7c0e1eca7e12f6ddab6bc0d1fc1/neutronclient/tests/unit/test_cli20.py#L77-L81

[Cedric Brandily (cbrandily)]

The bug is highlighted in https://bugs.launchpad.net/python-neutronclient/+bug/1503415

[Cedric Brandily (cbrandily)]

copy/paste facepalm

The bug is highlighted in http://logs.openstack.org/24/226424/1/gate/gate-python-neutronclient-python34/400ae0d

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/231700

[Cedric Brandily (cbrandily)]

This bug also affects neutron.agent.linux/windows.utils.execute

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/231709

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/231700
Committed: https://git.openstack.org/cgit/openstack/python-neutronclient/commit/?id=88eb5845b77fa06a12c096706093329f82dba54c
Submitter: Jenkins
Branch: master

commit 88eb5845b77fa06a12c096706093329f82dba54c
Author: Cedric Brandily <email address hidden>
Date: Tue Oct 6 22:22:09 2015 +0200

    Ensure to decode bytes or fail

    The commit fcf289797c063088f9003359dfd1c7d4f41ed5ef introduces the
    pattern:

      if isinstance(line, bytes):
        try:
          line = line.decode(encoding='utf-8')
        except UnicodeError:
          pass
      # concat line with a string

    which is not working in PY3K if an UnicodeError is raised because line
    is not decoded and concatened to a string.

    This change delegates decoding to safe_decode[1] which returns a text
    object or raises an error.

    [1] oslo_utils.encodeutils

    Closes-Bug: #1503415
    Related-Bug: #1499004
    Change-Id: I16b8013f33aa3efad65be8040d3210120e047bbd

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/231709
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4e4706e0f5652fde452aca3c69a3a08e62513e2f
Submitter: Jenkins
Branch: master

commit 4e4706e0f5652fde452aca3c69a3a08e62513e2f
Author: Cedric Brandily <email address hidden>
Date: Tue Oct 6 22:58:07 2015 +0200

    Ensure to decode bytes or fail

    The commit 048316e98114145ad19285cdeedd0040b59a66f8 introduces the
    pattern:

      if isinstance(line, bytes):
        try:
          line = line.decode(encoding='utf-8')
        except UnicodeError:
          pass
      # concat line with a string

    which is not working in PY3K if an UnicodeError is raised because line
    is (silently) not decoded and concatened to a string.

    This change ensures to return a text object or to raise an error.

    Closes-Bug: #1503415
    Blueprint: neutron-python3
    Change-Id: I16b8013f33aa3efad65be8040d3210120e047bbd

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/252876

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 8.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/252876
Committed: https://git.openstack.org/cgit/openstack/python-neutronclient/commit/?id=c5e1d2b827e16b761aa9601355fc91d969527d73
Submitter: Jenkins
Branch: stable/liberty

commit c5e1d2b827e16b761aa9601355fc91d969527d73
Author: Cedric Brandily <email address hidden>
Date: Tue Oct 6 22:22:09 2015 +0200

    Ensure to decode bytes or fail

    The commit fcf289797c063088f9003359dfd1c7d4f41ed5ef introduces the
    pattern:

      if isinstance(line, bytes):
        try:
          line = line.decode(encoding='utf-8')
        except UnicodeError:
          pass
      # concat line with a string

    which is not working in PY3K if an UnicodeError is raised because line
    is not decoded and concatened to a string.

    This change delegates decoding to safe_decode[1] which returns a text
    object or raises an error.

    [1] oslo_utils.encodeutils

    Closes-Bug: #1503415
    Related-Bug: #1499004
    Change-Id: I16b8013f33aa3efad65be8040d3210120e047bbd
    (cherry picked from commit 88eb5845b77fa06a12c096706093329f82dba54c)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/python-neutronclient 4.0.0 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/python-neutronclient 3.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-neutronclient 3.1.1 release.