Comment 6 for bug 2060943

Thanks for the follow up!

I did some more investigation:

# ---- Comparing the charms

I have downloaded the charm from CH and compared it to a locally built one, but it seems like there is not a significant difference. I just see some packages are updated (setuptools, pip and wheel). Other than that they seem pretty much equal.

# ---- juju debug-logs

I have noticed though that I am getting this message every time I deploy from CH:

application-kubeflow-volumes: 12:50:37 WARNING juju.worker.caasoperator operator "kubeflow-volumes" is shutting down, err: failed to initialize caasoperator for "kubeflow-volumes": failed to
 download charm "ch:amd64/focal/kubeflow-volumes-260" from API server: Get https://10.152.183.207:17070/model/3477bc3a-fc6a-4714-82fa-ec3383e85505/charms?file=%2A&url=ch%3Aamd64%2Ffocal%2Fku
beflow-volumes-260: cannot retrieve charm: ch:amd64/focal/kubeflow-volumes-260
application-kubeflow-volumes: 12:50:37 INFO juju.worker.caasoperator.runner runner is dying
application-kubeflow-volumes: 12:50:37 ERROR juju.worker.dependency "operator" manifold worker returned unexpected error: failed to initialize caasoperator for "kubeflow-volumes": failed to
download charm "ch:amd64/focal/kubeflow-volumes-260" from API server: Get https://10.152.183.207:17070/model/3477bc3a-fc6a-4714-82fa-ec3383e85505/charms?file=%2A&url=ch%3Aamd64%2Ffocal%2Fkub
eflow-volumes-260: cannot retrieve charm: ch:amd64/focal/kubeflow-volumes-260

I'm not sure if that is an actual issue as after that, I get the following lines:

application-kubeflow-volumes: 12:50:40 INFO juju.worker.caasoperator.charm downloading ch:amd64/focal/kubeflow-volumes-260 from API server
application-kubeflow-volumes: 12:50:40 INFO juju.downloader downloading from ch:amd64/focal/kubeflow-volumes-260
application-kubeflow-volumes: 12:50:40 INFO juju.downloader download complete ("ch:amd64/focal/kubeflow-volumes-260")
application-kubeflow-volumes: 12:50:40 INFO juju.downloader download verified ("ch:amd64/focal/kubeflow-volumes-260")
application-kubeflow-volumes: 12:50:44 INFO juju.worker.caasoperator operator "kubeflow-volumes" started
application-kubeflow-volumes: 12:50:44 INFO juju.worker.caasoperator.runner start "kubeflow-volumes/5"
application-kubeflow-volumes: 12:50:44 INFO juju.worker.leadership kubeflow-volumes/5 promoted to leadership of kubeflow-volumes
application-kubeflow-volumes: 12:50:44 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-kubeflow-volumes-5
application-kubeflow-volumes: 12:50:44 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/5 unit "kubeflow-volumes/5" started
application-kubeflow-volumes: 12:50:44 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/5 resuming charm install

This does not seem to be the case when I build locally and deploy:

application-kubeflow-volumes: 12:49:00 INFO juju.worker.caasoperator.charm downloading local:focal/kubeflow-volumes-1 from API server
application-kubeflow-volumes: 12:49:00 INFO juju.downloader downloading from local:focal/kubeflow-volumes-1
application-kubeflow-volumes: 12:49:00 INFO juju.downloader download complete ("local:focal/kubeflow-volumes-1")
application-kubeflow-volumes: 12:49:00 INFO juju.downloader download verified ("local:focal/kubeflow-volumes-1")
application-kubeflow-volumes: 12:49:04 INFO juju.worker.caasoperator operator "kubeflow-volumes" started
application-kubeflow-volumes: 12:49:04 INFO juju.worker.caasoperator.runner start "kubeflow-volumes/4"
application-kubeflow-volumes: 12:49:04 INFO juju.worker.leadership kubeflow-volumes/4 promoted to leadership of kubeflow-volumes
application-kubeflow-volumes: 12:49:04 INFO juju.agent.tools ensure jujuc symlinks in /var/lib/juju/tools/unit-kubeflow-volumes-4
application-kubeflow-volumes: 12:49:04 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/4 unit "kubeflow-volumes/4" started
application-kubeflow-volumes: 12:49:04 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/4 resuming charm install
application-kubeflow-volumes: 12:49:04 INFO juju.worker.caasoperator.uniter.kubeflow-volumes/4.charm downloading local:focal/kubeflow-volumes-1 from API server
application-kubeflow-volumes: 12:49:04 INFO juju.downloader downloading from local:focal/kubeflow-volumes-1
application-kubeflow-volumes: 12:49:04 INFO juju.downloader download complete ("local:focal/kubeflow-volumes-1")