hash check fails after download (no retry)? with misleading error

[Samuele Pedroni]

We got this kind of errored install sequence. I thought (also from once inspecting the code) that failed delta would end up with a full download, not an error? is something changed/off here?

ErrorMessage
download-snap: Undoing
validate-snap: Done
mount-snap: Undo
copy-snap-data: Undo
setup-profiles: Undo
link-snap: Undo
INFO Requested daemon restart.
set-auto-aliases: Undo
setup-aliases: Undo
start-snap-services: Undone
run-hook: Done
download-snap: Error
ERROR sha3-384 mismatch after patching "libreoffice": got e575a177de30352e07315ee87e266e43496d14c77f47957fca9006872389f4e976eff8718fd06ca541a30f4a4252c627 but expected f6dc17e42e56505037ed286b402aa1182130ab2a685373f5c56aee0f6103cc1575005144818fa7c13ad80853cf293d49
validate-snap: Hold
mount-snap: Hold
copy-snap-data: Hold
setup-profiles: Hold
link-snap: Hold
set-auto-aliases: Hold
setup-aliases: Hold
start-snap-services: Hold
run-hook: Hold
  ProblemType
Snap
  Snap
core

[Michael Nelson (michael.nelson)]

Indeed - there is even a test-case that ensures if any error is raised during the delta download and application, that a the normal download continues.

I don't suppose it's possible to get log output (I'll start now trying to reproduce locally to confirm), but it looks to me like the HashError always uses the error 'sha3-384 mismatch after patching...', even though it's used in contexts outside of deltas - see store.go:1290, which is for the normal download.

That is, it could be that the above error is a sha3-384 mismatch on the full download from line 1290, and has nothing to do with patching? Anyway, I won't assume that and will try to reproduce locally to see the full logs.

[Michael Nelson (michael.nelson)]

Just confirming my comment above. Here's an example of forcing a hash mis-match when applying the delta - which reports the expected HashError, but then successfully does the full download instead:

http://paste.ubuntu.com/24056083/

On the other hand, here's an example of forcing a hash mis-match when checking a full download where deltas are not even used, resulting in the misleading "mismatch after patching..." error message:

http://paste.ubuntu.com/24056108/

I don't have logs of the original libreoffice error, but I'm pretty confident based on the above that the full download had an incorrect hash, and the error message incorrectly mentions "after patching", rather than the fact that it was an error with the full download.

[William Grant (wgrant)]

That's libreoffice (aka. CpUkI0qPIIBVRsjy49adNq4D6Ra72y4v) revision 17. There are three deltas, from revisions 1, 7 and 14. All four blobs match what we expect in our storage, and in at least one CDN POP. All three deltas produce the correct result. I'm inclined to agree with Michael that this is actually unrelated to deltas, and the error message should be generalised.

I also verified that the failing hash isn't a prefix of any of the seven blobs I examined, so it wasn't just a truncated download.

[Michael Vogt (mvo)]

The error message does not involve the words "patching" anymore: `return fmt.Sprintf("sha3-384 mismatch for %q: got %s but expected %s", e.name, e.sha3_384, e.targetSha3_384)`. This was fixed with https://github.com/snapcore/snapd/pull/3152. We also retry now https://github.com/snapcore/snapd/pull/3159/files