Juju cli: consider providing a raw output for binary secrets

[Dima Tisnek]

Created by:

```
> printf '\xED\xA0\x80\xED\xB0\x80\xED\xA0\x81\xED\xB0\x81' > bad-uni
```

Host:

```
> hexdump -C bad-uni
00000000 ed a0 80 ed b0 80 ed a0 81 ed b0 81 |............|
0000000c
```

Added to juju controller by the hook tool using key#file notation.

Juju cli:

```
> juju show-secret --reveal cqc5vlvmp25c77mkspq0 --format json
{"cqc5vlvmp25c77mkspq0":{"revision":1,"owner":"fake-ingress","label":"unicode","created":"2024-07-18T00:20:43Z","updated":"2024-07-18T00:20:43Z","content":{"Data":{"bad-uni":"������������"}}}}

> juju show-secret --reveal cqc5vlvmp25c77mkspq0 --format yaml
cqc5vlvmp25c77mkspq0:
  revision: 1
  owner: fake-ingress
  label: unicode
  created: 2024-07-18T00:20:43Z
  updated: 2024-07-18T00:20:43Z
  content:
    bad-uni: ������������

> juju show-secret --reveal cqc5vlvmp25c77mkspq0 --format yaml | hexdump -C
00000000 63 71 63 35 76 6c 76 6d 70 32 35 63 37 37 6d 6b |cqc5vlvmp25c77mk|
00000010 73 70 71 30 3a 0a 20 20 72 65 76 69 73 69 6f 6e |spq0:. revision|
00000020 3a 20 31 0a 20 20 6f 77 6e 65 72 3a 20 66 61 6b |: 1. owner: fak|
00000030 65 2d 69 6e 67 72 65 73 73 0a 20 20 6c 61 62 65 |e-ingress. labe|
00000040 6c 3a 20 75 6e 69 63 6f 64 65 0a 20 20 63 72 65 |l: unicode. cre|
00000050 61 74 65 64 3a 20 32 30 32 34 2d 30 37 2d 31 38 |ated: 2024-07-18|
00000060 54 30 30 3a 32 30 3a 34 33 5a 0a 20 20 75 70 64 |T00:20:43Z. upd|
00000070 61 74 65 64 3a 20 32 30 32 34 2d 30 37 2d 31 38 |ated: 2024-07-18|
00000080 54 30 30 3a 32 30 3a 34 33 5a 0a 20 20 63 6f 6e |T00:20:43Z. con|
00000090 74 65 6e 74 3a 0a 20 20 20 20 62 61 64 2d 75 6e |tent:. bad-un|
000000a0 69 3a 20 ef bf bd ef bf bd ef bf bd ef bf bd ef |i: .............|
000000b0 bf bd ef bf bd ef bf bd ef bf bd ef bf bd ef bf |................|
000000c0 bd ef bf bd ef bf bd 0a |........|
000000c8
```

When the only output options are json or yaml, the result is that invalid unicode characters are represented using replacement characters (same 3-byte sequence over and over).

This means that secret value cannot be gotten by the operator. The orignal value cannot even be decoded.

[Ian Booth (wallyworld)]

Have you tried the base64 option?

You can get the data and then decode it, eg

data=$(secret-get cqc9fq7mp25c7979s760 foo#base64 | base64 -d)

or

secret-get cqc9fq7mp25c7979s760 foo#base64 | base64 -d | hexdump -C

etc

[Dima Tisnek (dimaqq)]

Good idea, tried this from the charm:

# secret-get cqc5vlvmp25c77mkspq0 bad-uni#base64 | base64 -d | hexdump -C
00000000 ef bf bd ef bf bd ef bf bd ef bf bd ef bf bd ef |................|
00000010 bf bd ef bf bd ef bf bd ef bf bd ef bf bd ef bf |................|
00000020 bd ef bf bd |....|
00000024

So the value cannot be read by the original charm and I imagine by any charm this secret would be shared with.

That still doesn't fully pinpoint where the conversion happened... set hook tool? API call to controller to set the secret? database? API call to get the secret? get hook tool?

[Dima Tisnek (dimaqq)]

P.S. it's relatively low priority, as the ops framework only allows string secret values today, not binary. Which means that no 1p charm can hit this issue, only if someone wrote a charm in bash or rust or something.

[Dima Tisnek (dimaqq)]

P.P.S. we're starting to use #file= notation to set secrets this week, and we're implicitly relying on these secrets to be string-ey and not binary.