juju doesn't accept it's own generated cloud.yaml for update-cloud

[Alexander Litvinov]

I can't figure a way to do show-cloud, then modify a part of it, and update-cloud because juju doesn't like the file it generates for me:

$ juju show-cloud microk8s-local --controller localhost-localhost --output cloud.yaml --format yaml

$ cat cloud.yaml
name: microk8s-local
summary: Cloud "microk8s-local" from controller "localhost-localhost"
defined: public
type: k8s
auth-types: [certificate, clientcertificate, oauth2, oauth2withcert, userpass]
endpoint: https://172.31.86.143:16443
regions:
  localhost: {}
users:
  admin:
    display-name: admin
    access: admin

$ juju update-cloud microk8s-local --controller localhost-localhost -f cloud.yaml
ERROR could not read cloud definition from provided file: Invalid cloud metadata map[auth-types:[certificate clientcertificate oauth2 oauth2withcert userpass] defined:public endpoint:https://172.31.86.143:16443 name:microk8s-local regions:map[localhost:map[]] summary:Cloud "microk8s-local" from controller "localhost-localhost" type:k8s users:map[admin:map[access:admin display-name:admin]]]

Going with json doesn't seem to work as well:

$ juju show-cloud microk8s-local --controller localhost-localhost --output cloud.json --format json

$ cat cloud.json
[{"name":"microk8s-local","summary":"Cloud \"microk8s-local\" from controller \"localhost-localhost\"","defined":"public","type":"k8s","auth-types":["certificate","clientcertificate","oauth2","oauth2withcert","userpass"],"endpoint":"https://172.31.86.143:16443","regions":{"localhost":{}},"users":{"admin":{"display-name":"admin","access":"admin"}}}]

$ juju update-cloud microk8s-local --controller localhost-localhost -f cloud.json
ERROR could not read cloud definition from provided file: cannot unmarshal yaml cloud metadata: yaml: unmarshal errors:
  line 1: cannot unmarshal !!seq into map[string]interface {}

Expected result:
Files generated with show-cloud --output (either yaml or json) are accepted back with update-cloud without any modifications of the file.

$ juju --version
3.3.1-genericlinux-amd64
Juju controller running on LXD and cloud is microk8s.
env is on public cloud vm, available to access if needed to troubleshoot.

[Alexander Litvinov (alitvinov)]

trying to adjust the file manually to what i assume it expects, but no luck yet, sometimes getting a different error.
Might be a separate issue, or same as the line seems to be the same

$ juju clouds --controller localhost-localhost

Clouds available on the controller:
Cloud Regions Default Type
localhost 1 localhost lxd
microk8s-local 1 localhost k8s

ubuntu@ip-172-31-82-122:~$ cat cloud-fixed.yaml
clouds:
  microk8s-local:
    type: k8s
    auth-types: [certificate, clientcertificate, oauth2, oauth2withcert, userpass]
    endpoint: https://172.31.86.143:16443
    regions:
      localhost: {}

$ juju update-cloud microk8s-local --controller localhost-localhost -f cloud-fixed.yaml
ERROR could not read cloud definition from provided file: no registered provider for "k8s"
no registered provider for "k8s"
github.com/juju/juju/cmd/juju/cloud.(*CloudFileReader).ReadCloudFromFile:747:
github.com/juju/juju/cmd/juju/cloud.(*updateCloudCommand).Run:138: could not read cloud definition from provided file

[Alexander Litvinov (alitvinov)]

According to the example from the docs (https://juju.is/docs/juju/cloud#heading--file-clouds-yaml) it should work I believe

$ cat example.yaml
clouds:
  microk8s-local:
    type: k8s
    auth-types:
      - certificate
      - clientcertificate
      - oauth2
      - oauth2withcert
      - userpass
    endpoint: https://172.31.86.143:16443
    regions:
      localhost:
        endpoint: https://172.31.86.143:16443

$ juju update-cloud microk8s-local --controller localhost-localhost -f example.yaml
ERROR could not read cloud definition from provided file: no registered provider for "k8s"

[Ian Booth (wallyworld)]

The cloud type needs to be "kubernetes". If you make that change the yaml in comment #2 will work.

juju show-cloud is meant to be for humans to read - it is not designed to be ingested by add/update cloud.
The add/update commands as you have found expect a file containing a list of clouds under a top level "clouds" key. Humans just want to see a single cloud printed.

We could make add/update cloud more flexible to read the human readable format as well as the more generic format.

[Alexander Litvinov (alitvinov)]

Ok indeed changing type to "kubernetes" works.
Then I believe second part of this (I believe bug) is
after I did update-cloud, show-cloud shows type as "k8s" again however updade-cloud expects "kubernetes" only.
There should be only 1 of them or both accepted otherwise it's confusing for user.

$ cat example.yaml
clouds:
  microk8s-local:
    type: kubernetes
...

$ juju update-cloud microk8s-local --controller localhost-localhost -f example.yaml
OK

$ juju show-cloud microk8s-local --controller localhost-localhost
Cloud "microk8s-local" from controller "localhost-localhost":
defined: public
type: k8s

[Alexander Litvinov (alitvinov)]

Also, regarding the initial issue,
>juju show-cloud is meant to be for humans to read

one option is to make one of the possible values of --output for show-cloud command a machine readable one.

[Ian Booth (wallyworld)]

juju displays k8s as "k8s" because it was considered easier to read for humans than "kubernetes".
juju show-cloud is for human consumption.

We could look at a cloud type alias - we have the same issue with lxd vs localhost. I can't recall exactly off hand if we allow those to be interchanged. I think we have a similar issue in place places, for things like migration and credential parsing. So it's not exactly super trivial to solve.

[Simon Richardson (simonrichardson)]

> we have the same issue with lxd vs localhost. I can't recall exactly off hand if we allow those to be interchanged.

Yes with great pain. I think most, if not all, is solved now.