juju deploy fails with remote error tls: handshake failure

[codingfreak]

Hi All,

I am not sure if this is an issue or something wrong on my end.

I am following instructions in given link to deploy kubeflow in a new ubuntu 20.04 vm instance

https://charmed-kubeflow.io/docs/get-started-with-charmed-kubeflow#heading--part-i-access-charmed-kubeflow

Juju controller installation was successful and when I try to deploy kubeflow charm as shown below it just throws TLS handshake failure error

$ juju controllers
Use --refresh option with this command to see the latest information.

Controller Model User Access Cloud/Region Models Nodes HA Version
microk8s-localhost* kubeflow admin superuser microk8s/localhost 2 - - 2.9.42

$ juju deploy kubeflow --trust --channel=1.7/stable
ERROR resolving with preferred channel: Post "https://api.charmhub.io/v2/charms/refresh": remote error: tls: handshake failure

$ juju deploy kubeflow
ERROR resolving with preferred channel: Post "https://api.charmhub.io/v2/charms/refresh": remote error: tls: handshake failure

$ juju version
2.9.42-ubuntu-amd64

$ juju deploy kubeflow --trust --channel=1.7/stable --debug
20:24:25 INFO juju.cmd supercommand.go:56 running juju [2.9.42 7b871e782195bdac9c90f8a8f01723cc3e08ab92 gc go1.18.10]
20:24:25 DEBUG juju.cmd supercommand.go:57 args: []string{"/snap/juju/22345/bin/juju", "deploy", "kubeflow", "--trust", "--channel=1.7/stable", "--debug"}
20:24:25 DEBUG juju.jujuclient proxy.go:65 unmarshalled proxy config for "kubernetes-port-forward"
20:24:25 INFO juju.juju api.go:86 connecting to API addresses: [10.152.183.138:17070]
20:24:25 DEBUG juju.api apiclient.go:625 starting proxier for connection
20:24:25 DEBUG juju.api apiclient.go:629 tunnel proxy in use at localhost on port 43975
20:24:25 DEBUG juju.api apiclient.go:1152 successfully dialed "wss://localhost:43975/api"
20:24:25 INFO juju.api apiclient.go:1054 cannot resolve "localhost": lookup localhost: operation was canceled
20:24:25 INFO juju.api apiclient.go:687 connection established to "wss://localhost:43975/api"
20:24:25 DEBUG juju.jujuclient proxy.go:65 unmarshalled proxy config for "kubernetes-port-forward"
20:24:25 INFO juju.juju api.go:86 connecting to API addresses: [10.152.183.138:17070]
20:24:25 DEBUG juju.api apiclient.go:625 starting proxier for connection
20:24:25 DEBUG juju.api apiclient.go:629 tunnel proxy in use at localhost on port 38075
20:24:25 DEBUG juju.api apiclient.go:1152 successfully dialed "wss://localhost:38075/model/4a2ce290-1cdb-489c-800b-bf0414c8cbef/api"
20:24:25 INFO juju.api apiclient.go:1054 cannot resolve "localhost": lookup localhost: operation was canceled
20:24:25 INFO juju.api apiclient.go:687 connection established to "wss://localhost:38075/model/4a2ce290-1cdb-489c-800b-bf0414c8cbef/api"
20:24:25 DEBUG juju.cmd.juju.application.deployer deployer.go:396 cannot interpret as local charm: file does not exist
20:24:25 DEBUG juju.cmd.juju.application.deployer deployer.go:208 cannot interpret as a redeployment of a local charm from the controller
20:24:26 DEBUG juju.api monitor.go:35 RPC connection died
20:24:26 DEBUG juju.api monitor.go:35 RPC connection died
ERROR resolving with preferred channel: Post "https://api.charmhub.io/v2/charms/refresh": remote error: tls: handshake failure
20:24:26 DEBUG cmd supercommand.go:537 error stack:
resolving with preferred channel: Post "https://api.charmhub.io/v2/charms/refresh": remote error: tls: handshake failure
github.com/juju/juju/cmd/juju/application/store.(*CharmAdaptor).ResolveCharm:100:
github.com/juju/juju/cmd/juju/application/store.(*CharmAdaptor).ResolveBundleURL:135:
github.com/juju/juju/cmd/juju/application/deployer.(*factory).maybeReadRepositoryBundle:464:
github.com/juju/juju/cmd/juju/application/deployer.(*factory).GetDeployer:71:
github.com/juju/juju/cmd/juju/application.(*DeployCommand).Run:909:

[codingfreak (codingfreak)]

mm411:~$ juju deploy minio --channel ckf-1.7/stable --show-log --logging-config='<root>=TRACE'
06:38:10 INFO juju.cmd supercommand.go:56 running juju [2.9.42 7b871e782195bdac9c90f8a8f01723cc3e08ab92 gc go1.18.10]
06:38:10 DEBUG juju.cmd supercommand.go:57 args: []string{"/snap/juju/22345/bin/juju", "deploy", "minio", "--channel", "ckf-1.7/stable", "--show-log", "--logging-config=<root>=TRACE"}
06:38:10 DEBUG juju.jujuclient proxy.go:65 unmarshalled proxy config for "kubernetes-port-forward"
06:38:10 INFO juju.juju api.go:86 connecting to API addresses: [10.152.183.70:17070]
06:38:10 DEBUG juju.api apiclient.go:625 starting proxier for connection
06:38:10 DEBUG juju.api apiclient.go:629 tunnel proxy in use at localhost on port 38553
06:38:10 TRACE juju.api certpool.go:48 cert dir "/etc/juju/certs.d" does not exist
06:38:10 TRACE juju.api apiclient.go:1149 dialing: "wss://localhost:38553/api" 127.0.0.1:38553
06:38:10 DEBUG juju.api apiclient.go:1152 successfully dialed "wss://localhost:38553/api"
06:38:10 INFO juju.api apiclient.go:1054 cannot resolve "localhost": lookup localhost: operation was canceled
06:38:10 INFO juju.api apiclient.go:687 connection established to "wss://localhost:38553/api"
06:38:10 TRACE juju.rpc.jsoncodec codec.go:227 -> {"request-id":1,"type":"Admin","version":3,"request":"Login","params":{"auth-tag":"user-admin","credentials":"57dffeb21b6b27d532a2ec6876ed8e0c","nonce":"","macaroons":null,"bakery-version":3,"cli-args":"/snap/juju/22345/bin/juju deploy minio --channel ckf-1.7/stable --show-log --logging-config=\u003croot\u003e=TRACE","user-data":"","client-version":"2.9.42"}}
06:38:10 TRACE juju.rpc.jsoncodec codec.go:121 <- {"request-id":1,"response":{"servers":[[{"value":"10.152.183.70","type":"ipv4","scope":"local-cloud","port":17070}]],"controller-tag":"controller-521d948c-0683-449c-8e13-62d700a3317a","user-info":{"display-name":"","identity":"user-admin","controller-access":"superuser","model-access":""},"facades":[{"name":"AllModelWatcher","versions":[2,3,4]},{"name":"ApplicationOffers","versions":[1,2,3,4]},{"name":"Bundle","versions":[1,2,3,4,5,6]},{"name":"Cloud","versions":[1,2,3,4,5,6,7]},{"name":"Controller","versions":[3,4,5,6,7,8,9,10,11]},{"name":"CrossController","versions":[1]},{"name":"HighAvailability","versions":[2]},{"name":"MigrationTarget","versions":[1]},{"name":"ModelConfig","versions":[1,2,3]},{"name":"ModelManager","versions":[2,3,4,5,6,7,8,9]},{"name":"ModelSummaryWatcher","versions":[1]},{"name":"ModelUpgrader","versions":[1]},{"name":"NotifyWatcher","versions":[1]},{"name":"Pinger","versions":[1]},{"name":"UserManager","versions":[1,2,3]}],"server-version":"2.9.42"}}
06:38:10 TRACE juju.rpc.jsoncodec codec.go:227 -> {"request-id":2,"type":"Controller","version":11,"request":"ControllerConfig","params":{}}
06:38:10 TRACE juju.rpc.jsoncodec codec.go:121 <- {"request-id":2,"response":{"config":{"agent-logfile-max-backups":2,"agent-logfile-max-size":"100M","api-port":17070,"api-port-open-delay":"2s","audit-log-capture-args":false,"audit-log-exclude-methods":["ReadOnlyMethods"],"audit-log-max-backups":10,"audit-log-max-size":"300M","auditing-enabled":true,"batch-raft-fsm":false,"ca-cert":"...

[Juan M. Tirado (tiradojm)]

I think this is a problem on your end. I just tried to deploy minio with the same commands you provided in your last message and everything seems to be up and running. No errors in the traces from the debug-log output.

Model Controller Cloud/Region Version SLA Timestamp
minio k8s2942 microk8s/localhost 2.9.43 unsupported 12:27:39+02:00

App Version Status Scale Charm Channel Rev Address Exposed Message
minio res:oci-image@1755999 waiting 1 minio ckf-1.7/stable 186 10.152.183.228 no

Unit Workload Agent Address Ports Message
minio/0* waiting idle 10.1.250.234 9000/TCP,9001/TCP waiting for container

[Jeff Diademi (jeffdiademi)]

This issue is described here: https://bugs.launchpad.net/juju/+bug/2073408

[Ian Booth (wallyworld)]

Are you sure that's the correct bug link?