Assertions and APIs format hashes differently

[Kyle Nitzsche]

It seems the snap-sha3-384 value returned as a header when downloading a snap from the rest api is not the same value as that in the same snap's snap-revision declaration. But, one needs the snap-sha3-384 to obtain the snap-revision, and this is needed to install a snap asserted.

I have an armhf uc20 pi3 that contains a snap with snapd control.

I am trying to use the snapd rest api to download the current revision of learnit snap, then download its assertions, so that I can ack the assertions followed by installing the snap (asserted). The context is a customer's desire to separate download from actual refresh since the download might be slow (cellular) in order to minimize device service interruption.

learnit snap on armhf is currently rev 24 on latest/stable:

knitzsche@ubuntu:~$ snap info learnit
name: learnit
summary: Learn through flash card tests
publisher: Kyle Nitzsche (knitzsche)
store-url: https://snapcraft.io/learnit
contact: <email address hidden>
license: LGPL-3.0+
description: |
  Learn through flashcards from external card files
snap-id: OHILDGdP7v0yZmkQnvGyXnGD5t2mZZ1g
channels:
  latest/stable: 1 2021-07-22 (24) 1MB -
  latest/candidate: ↑
  latest/beta: ↑
  latest/edge: 1 2021-07-23 (29) 1MB -

After running snap download learnit, here's the key data from the assert file:

knitzsche@ubuntu:~$ grep -A5 "type: snap-revision" learnit_24.assert
type: snap-revision
authority-id: canonical
snap-sha3-384: j73cFx0pIMoX4U2BxGXrFFMcwby4MjAkIxRea8LWx2xilWiR61u-XfgnmFWgkDdF
developer-id: oXBKQ6XsXgTcTNVFH6NlFsTz7Epn2kvJ
snap-id: OHILDGdP7v0yZmkQnvGyXnGD5t2mZZ1g
snap-revision: 24

Note the snap-sha3-384 value.

Now, in sudo snap run --shell of my snapd-control snap, I download learnit blob from latest: note the Snap-sha3-384 is DIFFERENT than above:

knitzsche@ubuntu:~$ sudo snap run --shell test-snapdapi.dev
root@ubuntu:/home/knitzsche# $SNAP/usr/bin/curl -v -sS -X POST -o $SNAP_COMMON/this.snap --unix-socket /run/snapd.socket http://localhost/v2/download --header "Content-Type: application/json" --data '{"snap-name":"learnit", "channel":"stable"}'
* Trying /run/snapd.socket:0...
* Connected to localhost (/run/snapd.socket) port 80 (#0)
> POST /v2/download HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 43
>
} [43 bytes data]
* upload completely sent off: 43 out of 43 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Disposition: attachment; filename=learnit_24.snap
< Content-Length: 1032192
< Content-Type: application/octet-stream
< Snap-Download-Token: eyJzbmFwLW5hbWUiOiJsZWFybml0IiwiZmlsZW5hbWUiOiJsZWFybml0XzI0LnNuYXAiLCJkbC1pbmZvIjp7ImRvd25sb2FkLXVybCI6Imh0dHBzOi8vYXBpLnNuYXBjcmFmdC5pby9hcGkvdjEvc25hcHMvZG93bmxvYWQvT0hJTERHZFA3djB5Wm1rUW52R3lYbkdENXQybVpaMWdfMjQuc25hcCIsInNpemUiOjEwMzIxOTIsInNoYTMtMzg0IjoiOGZiZGRjMTcxZDI5MjBjYTE3ZTE0ZDgxYzQ2NWViMTQ1MzFjYzFiY2I4MzIzMDI0MjMxNDVlNmJjMmQ2Yzc2YzYyOTU2ODkxZWI1YmJlNWRmODI3OTg1NWEwOTAzNzQ1In19BD10zRp1gdq_em45tq6tObJZRVa2Bh06dkmEQv4_5yA
< Snap-Length: 1032192
< Snap-Sha3-384: 8fbddc171d2920ca17e14d81c465eb14531cc1bcb832302423145e6bc2d6c76c62956891eb5bbe5df8279855a0903745
< Date: Thu, 30 Sep 2021 13:58:37 GMT
<
{ [14619 bytes data]
* Connection #0 to host localhost left intact
root@ubuntu:/home/knitzsche#

If I try to use the provided Snap-sha3-384 value to obtain the snap-revision assertion, it is not returned:

root@ubuntu:/home/knitzsche# $SNAP/usr/bin/curl -sS -X GET --unix-socket /run/snapd.socket "http://localhost/v2/assertions/snap-revision?series=16&remote=true&snap-id=OHILDGdP7v0yZmkQnvGyXnGD5t2mZZ1g&snap-sha3-384=8fbddc171d2920ca17e14d81c465eb14531cc1bcb832302423145e6bc2d6c76c62956891eb5bbe5df8279855a0903745&snap-revision=24"
root@ubuntu:/home/knitzsche#

[Kyle Nitzsche (knitzsche)]

comprehensive picture of released learnit snaps: $ snapcraft status learnit
Track Arch Channel Version Revision
latest amd64 stable 1 23
                  candidate 0.2 2
                  beta 0.2 2
                  edge 1 25
         arm64 stable 0.4 7
                  candidate ↑ ↑
                  beta ↑ ↑
                  edge 1 28
         armhf stable 1 24
                  candidate ↑ ↑
                  beta ↑ ↑
                  edge 1 29
         i386 stable - -
                  candidate - -
                  beta - -
                  edge 0.4 9
         ppc64el stable - -
                  candidate - -
                  beta - -
                  edge 1 27
         s390x stable - -
                  candidate - -
                  beta - -
                  edge 1 26

[Kyle Nitzsche (knitzsche)]

arch of system: knitzsche@ubuntu:~$ snap known model | grep arch
architecture: armhf

[Ian Johnson (anonymouse67)]

This is actually not a bug, just a format problem. The hash that is returned in the HTTP response from snapd is actually verbatim copied from the snapstore, which returns the hash as a hex-encoded byte array (the hash of the snap is indeed just a raw byte array so it could contain 0 or NUL which is not a valid ASCII string, hence encoding is necessary). However, snap assertions don't encode hashes as hex-encoded, instead they are URL base64 encoded, so this needs to be converted first before it works.

Not sure a bash + cURL equivalent of doing this conversion, but you can do this conversation with Go using:

b, _ := hex.DecodeString(foo)
assertKey := base64.RawURLEncoding.EncodeToString(b)

And there are of course other options in other programming languages.

[Kyle Nitzsche (knitzsche)]

ACK. It would be a nice touch if an HTTP header returned the value in a format that was directly usable, if possible.

[Kyle Nitzsche (knitzsche)]

So I'd argue it is a bug since the field names are identical but the content is different.
Special additional knowledge is necessary to use it, and this is not documented.

[Samuele Pedroni (pedronis)]

As Ian as explained the store APIs and assertion use sometimes different formatting for what is the same value. The value is the same and converting is easily done programmatically.

Snapd here made the header consistent with the store API instead of the assertion, both approaches are reasonable. The identical naming of the field with the assertion is just a coincidence BTW, as the "Snap-" in the header is just a generic prefix used by snapd, instead of meaning "of the snap".

So while we can document the current formatting situation, we won't change the header formatting because that would make sense only if the store would change its formatting, but both would be be backward incompatible at this point.

So once the documentation has been improved this will be marked as "Won't fix".

[Graham Morrison (morrisong)]

This has now been documented:
https://snapcraft.io/docs/assertions#heading--sha3-384