Client returns a decode error when secret cannot be formatted correctly

[Christopher Solis]

In addition to original description (below), the same decode error can occur for built-in secret types, like symmetric keys.

openstack secret get -p --payload_content_type 'application/octet-stream' http://10.30.118.79:9311/v1/secrets/bc924dfb-1bae-4657-a91e-49f451809976
'utf8' codec can't decode byte 0x9f in position 0: invalid start byte

The client should have enough information that the secret payload (in this case) is a 256 bit binary value, and not try to utf-8 decode it.

<original description below>

Storing a secret using base64 seems to work fine:

barbican secret store -p '1111222233334444' --payload-content-type 'application/octet-stream' --payload-content-encoding 'base64'

However, retrieving the secret payload fails.

barbican secret get -p http://localhost:9311/v1/secrets/aef819bf-5997-4b02-a8c4-543fe0c735e5 --payload_content_type 'application/octet-stream'
Starting new HTTP connection (1): localhost
Starting new HTTP connection (1): localhost
'utf8' codec can't decode byte 0xd7 in position 0: invalid continuation byte

It looks like the client is trying to use the utf8 character set even though it was not stored in as text/plain content.

[Christopher Solis (cnsolis)]

After investigation the root of this bug is the same as bug 1419166.

I think it's time to deprecate manually setting the content_type and content_encoding parameters. The documentation has it marked as deprecated but it is still possible through the cli and the client to use the parameters.

[Dave McCowan (dave-mccowan)]

For backwards compatibility, I don't think deprecating will be enough.
For encryption keys (like used by Nova/Cinder volume encryption), I can have:

[stack@localhost devstack]$ openstack secret get http://10.30.118.79:9311/v1/secrets/bc924dfb-1bae-4657-a91e-49f451809976
+---------------+--------------------------------------------------------------------------+
| Field | Value |
+---------------+--------------------------------------------------------------------------+
| Secret href | http://10.30.118.79:9311/v1/secrets/bc924dfb-1bae-4657-a91e-49f451809976 |
| Name | Cinder Volume Key |
| Created | 2015-12-03 04:29:04+00:00 |
| Status | ACTIVE |
| Content types | {u'default': u'application/octet-stream'} |
| Algorithm | AES |
| Bit length | 256 |
| Secret type | symmetric |
| Mode | CBC |
| Expiration | None |
+---------------+--------------------------------------------------------------------------+

To get the payload in table format, I get this error:

[stack@localhost devstack]$ openstack secret get -p --payload_content_type 'application/octet-stream' http://10.30.118.79:9311/v1/secrets/bc924dfb-1bae-4657-a91e-49f451809976
'utf8' codec can't decode byte 0x9f in position 0: invalid start byte

If I add "-f value", I can see the payload:

[stack@localhost devstack]$ openstack secret get -p --payload_content_type 'application/octet-stream' -f value http://10.30.118.79:9311/v1/secrets/bc924dfb-1bae-4657-a91e-49f451809976
��]s�v���Ͳ<1ʔ���iu�5��+o_B�

I can decode it manually:

[stack@localhost devstack]$ openstack secret get -p --payload_content_type 'application/octet-stream' -f value http://10.30.118.79:9311/v1/secrets/bc924dfb-1bae-4657-a91e-49f451809976 | od -x
0000000 c69f 735d 7688 e08c cdf2 3cb2 ca31 aa94
0000020 b10f 698d 1c05 ad75 9b35 2b9c 5f6f 8f42
0000040 000a
0000041

At least for secrets with built-in secret types, Barbican client can be smarter about formatting.

[Douglas Mendizábal (dougmendizabal)]

It appears that the decode error is being thrown by Cliff, and not by python-barbicanclient code. I still need to dig into why this is happening, but it's got something to do with the Cliff formatters. :-\

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/283672

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christopher Solis (<email address hidden>) on branch: master
Review: https://review.openstack.org/283672
Reason: It looks like the reason python3.4 is not raising the exception is because it is converting the bytestrings to actual integer values to have a friendly output. Which I think is not good because that basically means what the user puts in might not be what they get back out. At least from a CLI perspective, not the client. Patch is being abandoned in lieu of revamping the CLI, we will address this issue.