Closed/removed track still shows up in charmhub, juju, but not charmcraft status

[Alex Kavanagh]

Context/background:

* charmcraft 1.7.1 and 1.5
* juju 2.9.31
* charm: mysql-innodb-cluster

Originally, there was a 8.0.19 track; a 8.0 track was added, and the 8.0.19 track was requested to be 'removed'. The 8.0 track is the default track.

However, juju and charmhub.io both show the 8.0.19 track; charmcraft does not.

The "https://api.charmhub.io/v2/charms/info/mysql-innodb-cluster?fields=channel-map" endpoint also lists the 8.0.19 track entries.

juju info mysql-innodb-cluster:

name: mysql-innodb-cluster
charm-id: i5HWuKoEat09854Sz0qXHOjmr4wwHLfP
summary: MySQL InnoDB Cluster
publisher: OpenStack Charmers
supports: focal, jammy
tags: cloud, databases
subordinate: false
store-url: https://charmhub.io/mysql-innodb-cluster
description: |
  MySQL InnoDB Cluster Charm deploys and manages the lifecycle of a
  MySQL InnoDB Cluster.
relations:
  provides:
    db-monitor: mysql-monitor
    db-router: mysql-router
    shared-db: mysql-shared
  requires:
    certificates: tls-certificates
channels: |
  8.0/stable: be2f3d1 2022-06-13 (26) 14MB
  8.0/candidate: ↑
  8.0/beta: ↑
  8.0/edge: ↑
  latest/stable: 15 2022-02-09 (15) 13MB
  latest/candidate: ↑
  latest/beta: ↑
  latest/edge: 773bcae 2022-06-01 (27) 14MB
  8.0.19/stable: be2f3d1 2022-05-05 (24) 14MB
  8.0.19/candidate: be2f3d1 2022-04-28 (24) 14MB
  8.0.19/beta: ↑
  8.0.19/edge: be2f3d1 2022-04-27 (24) 14MB

And "charmcraft status mysql-innodb-cluster":

Track Base Channel Version Revision Resources
latest ubuntu 19.10 (all) stable 3 3 mysql-shell (r0)
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 20.04 (all) stable 15 15 mysql-shell (r0)
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 20.04 (amd64) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
         ubuntu 20.04 (arm64) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
         ubuntu 20.04 (ppc64el) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
         ubuntu 20.04 (s390x) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
         ubuntu 20.10 (all) stable 15 15 mysql-shell (r0)
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 21.04 (all) stable 15 15 mysql-shell (r0)
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 21.10 (all) stable 15 15 mysql-shell (r0)
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 22.04 (amd64) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
         ubuntu 22.04 (arm64) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
         ubuntu 22.04 (ppc64el) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
         ubuntu 22.04 (s390x) stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge 773bcae 27 -
5.7 - stable - - -
                                 candidate - - -
                                 beta - - -
                                 edge - - -
8.0 ubuntu 20.04 (amd64) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 20.04 (arm64) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 20.04 (ppc64el) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 20.04 (s390x) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 22.04 (amd64) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 22.04 (arm64) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 22.04 (ppc64el) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑
         ubuntu 22.04 (s390x) stable be2f3d1 26 -
                                 candidate ↑ ↑ ↑
                                 beta ↑ ↑ ↑
                                 edge ↑ ↑ ↑

When charmcraft tries to close a risk on the 8.0.19 track:

charmcraft close mysql-innodb-cluster 8.0.19/candidate
Store operation failed:
- None: Invalid track: 8.0.19
Full execution log: '/home/alex/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220615-120556.034291.log'

2022-06-15 12:05:56.034 Starting charmcraft version 1.7.1
2022-06-15 12:05:56.034 Raw pre-parsed sysargs: args={'help': False, 'verbose': False, 'quiet': False, 'trace': False, 'project_dir': None} filtered=['close', 'mysql-innodb-cluster', '8.0.19/candidate']
2022-06-15 12:05:56.034 General parsed sysargs: command='close' args=['mysql-innodb-cluster', '8.0.19/candidate']
2022-06-15 12:05:56.034 Couldn't find config file '/home/alex/Projects/Canonical/git/github.com/openstack-charmers/release-tools/charmcraft.yaml'
2022-06-15 12:05:56.035 Command parsed sysargs: Namespace(channel='8.0.19/candidate', name='mysql-innodb-cluster')
2022-06-15 12:05:56.035 System details: OSPlatform(system='ubuntu', release='20.04', machine='x86_64'); Environment: DESKTOP_SESSION='ubuntu', XDG_CURRENT_DESKTOP='ubuntu:GNOME'
2022-06-15 12:05:56.110 Retrieving credentials for 'charmcraft' on 'api.charmhub.io' from keyring 'SecretService Keyring'.
2022-06-15 12:05:56.145 HTTP 'POST' for 'https://api.charmhub.io/v1/charm/mysql-innodb-cluster/releases' with params None and headers {'Authorization': '<macaroon>', 'User-Agent': 'charmcraft/1.7.1 ubuntu/20.04 (x86_64) python/3.8.10'}
2022-06-15 12:05:56.294 Store operation failed:
- None: Invalid track: 8.0.19
2022-06-15 12:05:56.294 Traceback (most recent call last):
2022-06-15 12:05:56.294 File "/snap/charmcraft/978/lib/charmcraft/commands/store/store.py", line 140, in error_decorator
2022-06-15 12:05:56.294 return method(self, *args, **kwargs)
2022-06-15 12:05:56.294 File "/snap/charmcraft/978/lib/charmcraft/commands/store/store.py", line 322, in release
2022-06-15 12:05:56.294 self._client.request_urlpath_json("POST", endpoint, json=items)
2022-06-15 12:05:56.294 File "/snap/charmcraft/978/lib/charmcraft/commands/store/client.py", line 91, in request_urlpath_json
2022-06-15 12:05:56.294 response = super().request(method, self.api_base_url + urlpath, *args, **kwargs)
2022-06-15 12:05:56.294 File "/snap/charmcraft/978/lib/craft_store/base_client.py", line 174, in request
2022-06-15 12:05:56.294 return self.http_client.request(
2022-06-15 12:05:56.294 File "/snap/charmcraft/978/lib/craft_store/http_client.py", line 166, in request
2022-06-15 12:05:56.294 raise errors.StoreServerError(response)
2022-06-15 12:05:56.294 craft_store.errors.StoreServerError: Store operation failed:
2022-06-15 12:05:56.294 - None: Invalid track: 8.0.19
2022-06-15 12:05:56.294 Full execution log: '/home/alex/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220615-120556.034291.log'

Juju downloading various versions:

$ juju download mysql-innodb-cluster
Fetching charm "mysql-innodb-cluster" using "stable" channel and base "amd64/ubuntu/20.04"
Install the "mysql-innodb-cluster" charm with:
    juju deploy ./mysql-innodb-cluster_55f085e.charm

$ juju download mysql-innodb-cluster --channel=8.0/stable
Fetching charm "mysql-innodb-cluster" using "8.0/stable" channel and base "amd64/ubuntu/20.04"
Install the "mysql-innodb-cluster" charm with:
    juju deploy ./mysql-innodb-cluster_55f085e.charm

So the default track is 8.0/stable.

juju download mysql-innodb-cluster --channel=8.0.19/stable
Fetching charm "mysql-innodb-cluster" using "8.0.19/stable" channel and base "amd64/ubuntu/20.04"
Install the "mysql-innodb-cluster" charm with:
    juju deploy ./mysql-innodb-cluster_d307f37.charm

So juju can download the 8.0.19 charm.

The issue:

juju and charmhub.io can 'see' and access the 8.0.19 track. charmcraft can't, and also won't allow the track/risk to be manipulated/closed. There's an inconsistency amongst the applications that access the charmhub.

Also raised on charmcraft: https://github.com/canonical/charmcraft/issues/785

[Juan M. Tirado (tiradojm)]

Juju fully depends on charmhub.io to deploy charms. There are no special tracks for Juju to access removed data/charms. This is probably and ongoing issue with Charmhub.io rather than Juju itself.

[Daniel Manrique (roadmr)]

CHarmhub-side it's not really a bug because the way we "deleted" the track caused an inconsistency in the data model; the "fix" is to not delete tracks from a particular table :)