Juju fails to deploy mysql-k8s charm with its image resource

[Jose C. Massón]

When deploying the mysql-k8s charm in microk8s and run:

$ juju deploy mysql-k8s

I get in juju debug-log:

unit-mysql-k8s-0: 18:39:40 ERROR juju.worker.uniter pebble poll failed for container "mysql": failed to get pebble info: cannot obtain system details: cannot communicate with server: Get "http://localhost/v1/system-info": dial unix /charm/containers/mysql/pebble.socket: connect: no such file or directory

But seems that the real issue here is:

$ microk8s.kubectl describe pod mysql-k8s-0 -n mysql-k8s

  Normal Pulling 19s (x3 over 67s) kubelet Pulling image "registry.jujucharms.com/charm/62ptdfbrjpw3n9tcnswjpart30jauc6wc5wbi/mysql-image@sha256:175204794db040d561902c1cfb702acef21847658520fa9dabd49c3e85394836"
  Warning Failed 16s (x3 over 63s) kubelet Failed to pull image "registry.jujucharms.com/charm/62ptdfbrjpw3n9tcnswjpart30jauc6wc5wbi/mysql-image@sha256:175204794db040d561902c1cfb702acef21847658520fa9dabd49c3e85394836": rpc error: code = Unknown desc = failed to pull and unpack image "registry.jujucharms.com/charm/62ptdfbrjpw3n9tcnswjpart30jauc6wc5wbi/mysql-image@sha256:175204794db040d561902c1cfb702acef21847658520fa9dabd49c3e85394836": failed to resolve reference "registry.jujucharms.com/charm/62ptdfbrjpw3n9tcnswjpart30jauc6wc5wbi/mysql-image@sha256:175204794db040d561902c1cfb702acef21847658520fa9dabd49c3e85394836": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Some minutes before I uploaded the charm and the resource to charmhub:

2021-06-30 18:32:45,293 charmcraft.commands.store INFO Image uploaded, new remote digest: sha256:175204794db040d561902c1cfb702acef21847658520fa9dabd49c3e85394836.
2021-06-30 18:32:45,293 charmcraft.commands.store DEBUG Hitting the store: POST https://api.charmhub.io/v1/charm/mysql-k8s/resources/mysql-image/oci-image/blob {'image-digest': 'sha256:175204794db040d561902c1cfb702acef21847658520fa9dabd49c3e85394836'}

Note the digest is the same: sha256:175204794db040d561902c1cfb702acef21847658520fa9dabd49c3e85394836

and the charm and resource are uploaded:

$ charmcraft revisions mysql-k8s
Revision Version Created at Status
2 2 2021-06-29 released
1 1 2021-06-29 approved

$ charmcraft resources mysql-k8s
Charm Rev Resource Type Optional
2 mysql-image oci-image True
1 mysql-image oci-image True

[Jose C. Massón (jose-masson)]

Attached you'll find the output of:

$ microk8s.kubectl describe pod mysql-k8s-0 -n mysql-k8s > out.txt

[John A Meinel (jameinel)]

I would like to defer 2.9.8 until we know that this is fixed. Charmcraft wants to go stable with publishing to registry.jujucharms.com but we will be broken for those new charms until this is fixed.

[Harry Pidcock (hpidcock)]

https://github.com/juju/juju/pull/13140