cliutils.print_list mangles output with python3

[Steven Hardy]

Trying to migrate python-heatclient to the latest cliutils, including using the print_list function, and this happens:

(py27)-bash-4.2$ source .tox/py27/bin/activate
(py27)-bash-4.2$ heat stack-list
+--------------------------------------+------------+-----------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+-----------------+----------------------+
| 74261c23-1bc4-4bd6-be29-a7657781c137 | rg1 | CREATE_COMPLETE | 2014-11-07T13:03:44Z |
| 78c6092e-8bd6-4287-a512-ccce02c06598 | t1 | CREATE_FAILED | 2014-11-07T13:10:16Z |
+--------------------------------------+------------+-----------------+----------------------+
(py27)-bash-4.2$ source .tox/py33/bin/activate
(py33)-bash-4.2$ heat stack-list
b'+--------------------------------------+------------+-----------------+----------------------+\n| id | stack_name | stack_status | creation_time |\n+--------------------------------------+------------+-----------------+----------------------+\n| 74261c23-1bc4-4bd6-be29-a7657781c137 | rg1 | CREATE_COMPLETE | 2014-11-07T13:03:44Z |\n| 78c6092e-8bd6-4287-a512-ccce02c06598 | t1 | CREATE_FAILED | 2014-11-07T13:10:16Z |\n+--------------------------------------+------------+-----------------+----------------------+'

It appears that the print(encodeutils.safe_encode(pt.get_string(**kwargs))) mangles the output on python3, because the encodeutils safe_encode encodes the string before printing.

Removing the encodeutils.safe_encode and just printing the pt.get_string works fine, but I assume this isn't acceptable for i18n reasons..

[Steven Hardy (shardy)]

Actually, this looks like it may be an oslo.utils bug where something hasn't made the transition from incubator to the library..

[Steven Hardy (shardy)]

Fix proposed to oslo.utils:

https://review.openstack.org/#/c/133271/

This reinstates the following logic which looks like it got lost in translation between incubator strutils and oslo.utils encodeutils:

https://github.com/openstack/python-heatclient/blob/master/heatclient/openstack/common/strutils.py#L162

[Steven Hardy (shardy)]

Looking at the test failures caused by the oslo.utils patch, it may be that introducing a conditional in print_list may be better, so we do the extra decode there when needed for python3?

[Steven Hardy (shardy)]

patch to cliutils posted instead:

https://review.openstack.org/133290

[Valeriy Ponomaryov (vponomaryov)]

This is duplicate for https://bugs.launchpad.net/oslo-incubator/+bug/1370710 and related commit https://review.openstack.org/#/c/122252

[Steven Hardy (shardy)]

@vponomaryov: Thanks, I hadn't spotted that bug or patch tbh.

AFAICS that is not exactly the same as this (although it would probably resolve the same problem), my fix reinstates the original behavior before the incubator strutils to oslo.utils encodeutils move (just fixing the regression), whereas your patch aims to remove the encoding completely.

It's not clear to me if that will have undesirable consequences related to i18n, presumably the encoding was added for a reason?

Regardless, my patch has landed now which solves my immediate regression problem, but I'll keep an eye on your patch, thanks!