test_ssh_encrypt_decrypt_text fails with UnicodeDecodeError after oslo.concurrency 0.4.0 release

[Matt Riedemann]

http://logs.openstack.org/14/143214/1/gate/gate-nova-python27/bc5d6ce/console.html#_2015-01-13_15_38_09_489

oslo.concurrency 0.4.0 was just released today so there must be something new in there that's breaking nova tests (which could have been doing naughty things anyway):

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiVW5pY29kZURlY29kZUVycm9yOiAndXRmOCcgY29kZWMgY2FuJ3QgZGVjb2RlIGJ5dGUgMHg4YSBpbiBwb3NpdGlvbiA4OiBpbnZhbGlkIHN0YXJ0IGJ5dGVcIiBBTkQgdGFnczpcImNvbnNvbGVcIiBBTkQgYnVpbGRfbmFtZTpcImdhdGUtbm92YS1weXRob24yN1wiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDIxMTY1OTY0NjExfQ==

[Matt Riedemann (mriedem)]

Looks like this is probably causing the failure:

https://github.com/openstack/oslo.concurrency/commit/7c7493feb53429577efca2c4b0380af03ddc149b

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/146929

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/146937

[Matt Riedemann (mriedem)]

Revert proposed for the original breaking change: https://review.openstack.org/#/c/146940/

[Matt Riedemann (mriedem)]

global-requirements block on oslo.concurrency 0.4.0: https://review.openstack.org/#/c/146942/

[Victor Stinner (vstinner)]

History of the change.

My initial goal was to port oslo_concurrency to Python 3.

I had issues with the mask_password() function of oslo_utils.strutils which wants to work on unicode. If the input type is bytes, mask_password() tries to decode the input from the default Python encoding (ex: ascii in Python 2, utf-8 on Python 3), but it keeps the input unchanged on UnicodeDecodeError. I proposed a change to be able to work on bytes, but it was rejected (or at least not accepted):
https://review.openstack.org/#/c/137600/

I chose a different approach: have a reliable behaviour on Python 2 & Python 3, for any kind of output (ascii or non-ascii output): always work on Unicode.

I don't understand the expectations of these functions (execute, ssh_execute). The output (stdout/stderr) should be considered as binary or text? I prefer to consider that it is text because on error was tried to replace a text pattern with a message to hide passwords.

I'm working on a change which would restore the previous behaviour on Python 2, and keep the Python 3 compatibility. Hint: don't decode stdout/stderr on Python 2.

[Victor Stinner (vstinner)]

I proposed this change:
https://review.openstack.org/#/c/146984/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Doug Hellmann (<email address hidden>) on branch: master
Review: https://review.openstack.org/146937
Reason: Reverting the breaking change in oslo.concurrency: https://review.openstack.org/146940

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/146929

[Joe Gordon (jogo)]

not seeing this cause nova breaking change in oslo.concurrency was reverted