juju isn't consistently showing if a new charm upgrade is available

[Garrett Neugent]

While working to upgrade a customer cloud, I saw that the 'can-upgrade-field' doesn't always populate in "juju status --format yaml", thus falsely showing that a given charm doesn't have newer revisions available. For example:

<user>@<host>$ juju status --format yaml | grep -i 'upgrade' # yields nothing
<user>@<host>$ juju status --format yaml | grep 'cs:aodh'
    charm: cs:aodh-51

if I use charm pull cs:aodh, it correctly shows that the latest revision of aodh for the stable channel is v57:

<user>@<host>$ charm pull cs:aodh
cs:aodh-57

Looking at controller logs, i see the following - https://pastebin.canonical.com/p/JWJ8GpcwsY/

/var/log/juju/machine-0.log:2022-02-10 19:15:40 ERROR juju.worker.dependency engine.go:671 "charm-revision-updater" manifold worker returned unexpected error: while getting latest charm revision info: cannot get metadata from the charm store: unexpected response status from server: 404 NOT FOUND

/var/log/juju/machine-0.log:2022-04-11 17:14:51 ERROR juju.worker.dependency engine.go:671 "charm-revision-updater" manifold worker returned unexpected error: while getting latest charm revision info: cannot get metadata from the charm store: cannot unmarshal response: json: cannot unmarshal array into Go value of type map[string]struct { Meta struct { IdRevision params.IdRevisionResponse "json:\"id-revision\""; Hash256 params.HashResponse "json:\"hash256\"" } }

This is with a model using Juju 2.9.27, and the client being a snap:

$ juju controllers
Use --refresh option with this command to see the latest information.

Controller Model User Access Cloud/Region Models Nodes HA Version
<name>* <model> <user> superuser <cloud>/default 4 91 3 2.9.27

$ juju version
2.9.28-ubuntu-amd64
$ which -a juju
/snap/bin/juju

As a workaround, one can run "charm pull" x on the relevant charm, to verify which is the correct upstream version, and compare it against the version running in the cloud to determine if an upgrade is available.

[Heather Lanigan (hmlanigan)]

The log messages are a result of https://bugs.launchpad.net/snapstore-server/+bug/1967002. The fix went into production on April 12, after the errors logged.

It's unclear why subsequent runs of the charmrevisionupdater worker haven't done their job with this worker. It runs every 24 hours.

Interestingly, a different model on that controller has provided 'can-upgrade-to' results.

[Canonical Juju QA Bot (juju-qa-bot)]

This Medium-priority bug has not been updated in 60 days, so we're marking it Low importance. If you believe this is incorrect, please update the importance.