units stay blocked with "Connect a container runtime."

[Jason Hobbs]

Starting this afternoon, when deploying kubernetes-worker, my units go into blocked state and stay there with this message:

"Connect a container runtime."

It seems like the charm was changed and that a container runtime now has to be connected, but I don't see anything about that in the charm docs or release notes.

Was there a charm release?

[Mike Wilson (knobby)]

There was a release today to support 1.15. The release notes[1] on ubuntu.com have information about this. Where did you look so we can update those docs to point to the proper place?

[1] https://ubuntu.com/kubernetes/docs/release-notes

[Jason Hobbs (jason-hobbs)]

You're right, and Tim sent me an email earlier today telling me that and I
didn't see it.

I went to https://ubuntu.com/kubernetes/docs/release-notes and didn't see
anything about having to make the new relation. I didn't look under
upgrades because I'm not upgrading.

Maybe it would be good to mention the need for the relation on the
k8s-master/worker charm doc pages on jaas.ai

On Fri, Jun 28, 2019 at 10:20 PM Mike Wilson <email address hidden> wrote:

> There was a release today to support 1.15. The release notes[1] on
> ubuntu.com have information about this. Where did you look so we can
> update those docs to point to the proper place?
>
> [1] https://ubuntu.com/kubernetes/docs/release-notes
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1834722
>
> Title:
> units stay blocked with "Connect a container runtime."
>
> Status in Kubernetes Worker Charm:
> New
>
> Bug description:
> Starting this afternoon, when deploying kubernetes-worker, my units go
> into blocked state and stay there with this message:
>
> "Connect a container runtime."
>
> It seems like the charm was changed and that a container runtime now
> has to be connected, but I don't see anything about that in the charm
> docs or release notes.
>
> Was there a charm release?
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/charm-kubernetes-worker/+bug/1834722/+subscriptions
>

[Mike Wilson (knobby)]

Added these two PRs to try and improve this experience.

https://github.com/charmed-kubernetes/charm-kubernetes-master/pull/35
https://github.com/charmed-kubernetes/charm-kubernetes-worker/pull/23

[George Kraft (cynerva)]

Cherry-picked to stable branches:

https://github.com/charmed-kubernetes/charm-kubernetes-master/commit/1d95f1f3131460a346afb6e01de128720759207b
https://github.com/charmed-kubernetes/charm-kubernetes-worker/commit/b23cb1796e171e918e59051aba12cd1e123a368e

[i will (ianwww)]

I think this is related? When I try to create a cluster using ONLY docker as subordinate charm in my bundle.yaml and no containerd i'm getting the Container runtime not available prompt on the docker charm itself. Can anyone confirm this?

[Tim Van Steenburgh (tvansteenburgh)]

@ianwww Please post your bundle.yaml.

[i will (ianwww)]

thanks for the response @tim

[i will (ianwww)]

Ultimately I was doing this is a workaround because I could not configure containerd to access my insecure registries hosted locally in artifactory in my kubernetes cluster. (with that I was getting x509 errors, despite providing the u/p in the plugin toml config file) I hoped that I would be able to solve my issue easier with my familiarity with docker vs containerd+docker plugin.

[Tim Van Steenburgh (tvansteenburgh)]

I tweaked your bundle a little and everything came up fine for me. My edits are at http://paste.ubuntu.com/p/Hx8VZd3Wdb/.

1. I removed spaces/zones stuff for simplicity
2. I bumped the revision on a couple charms
3. I added the 'channel' config back to etcd. You need this. Otherwise you'll get a really old version of etcd that won't work.

See if that works for you.

[i will (ianwww)]

thanks for help @tvansteenburgh. It did work! I tried to identify which of the changes you made it was. From my testing it was commenting this out in the docker charm config. Does this look wrong?
options:
      #docker-opts: "--insecure-registry "

[George Kraft (cynerva)]

The --insecure-registry option requires an argument. See https://docs.docker.com/engine/reference/commandline/dockerd/#insecure-registries