juju refresh can fail due to empty charm sha256

[Ian Booth]

It's been observed when running some smoke tests that the unit agent can error when deploying / refreshing a charm - the agent goes to failure status with an error that the charm's sha256 is empty.

It turns out his can happen if the charm is not yet saved to the database when the unit agent makes the api call during deploy / refresh. The root cause here is ultimately the transition to async deploys. There's mistakes in that implementation if the unit agent is allowed to start or upgrade before the charm is ready.

[Ian Booth (wallyworld)]

This PR https://github.com/juju/juju/pull/17134 adds a defensive retry to the unit agent CharmArchiveSha256() method. It will mask the issue until a proper fix can be done.

[Joseph Phillips (manadart)]

Ultimate fix should accommodate https://bugs.launchpad.net/juju/+bug/2058700 too.

[Caner Derici (cderici)]

This is related to https://bugs.launchpad.net/juju/+bug/2053242 (see my comment #3), had a fix up https://github.com/juju/juju/pull/17504 (just realized that it's similar to Ian's PR above), but Heather wanted to take over that one for a more holistic approach (see https://github.com/juju/juju/pull/17504#pullrequestreview-2112189555). I'll ask her if she wants to take over this one as well.