Not able to create TLS_DISABLED k8s cluster

Bug #1714880 reported by yatin on 2017-09-04
22
This bug affects 5 people
Affects Status Importance Assigned to Milestone
Magnum
New
Undecided
Unassigned

Bug Description

Setup magnum with devstack:-
My env:-
[centos@devstack-1 magnum]$ magnum-api --version
5.0.1.dev15
[centos@devstack-1 magnum]$ magnum --version
2.7.0

Create TLS_DISABLED k8s cluster.

Login to k8s master node:-
kube api server failed to start:-

[fedora@k8s-cluster-ecked4vw4tnl-master-0 ~]$ sudo atomic containers list
   CONTAINER ID IMAGE COMMAND CREATED STATE BACKEND RUNTIME
   kubelet docker.io/openstackm /usr/bin/kubelet-doc 2017-09-04 05:11 running ostree runc
   kube-proxy docker.io/openstackm /usr/bin/kube-proxy- 2017-09-04 05:11 running ostree runc
   kube-apiserv docker.io/openstackm /usr/bin/kube-apiser 2017-09-04 05:12 failed ostree runc
   kube-control docker.io/openstackm /usr/bin/kube-contro 2017-09-04 05:12 running ostree runc
   kube-schedul docker.io/openstackm /usr/bin/kube-schedu 2017-09-04 05:12 running ostree runc

Following Error is seen in apiserver logs: Sep 04 06:09:54 k8s-cluster-ecked4vw4tnl-master-0.novalocal runc[7368]: I0904 06:09:54.619508 1 server.go:112] Version: v1.7.4
Sep 04 06:09:54 k8s-cluster-ecked4vw4tnl-master-0.novalocal runc[7368]: error creating self-signed certificates: open /var/run/kubernetes/apiserver.crt: read-only file system

/var/run/kubernetes is the default cirt-dir for apiserver certs.

It looks like atomic container(k8s apiserver 1.7.4) is trying to create certs even for TLS_DISABLED cluster.

Spyros Trigazis (strigazi) wrote :

If we mount /var/run/kubernetes does it work? I mean if it can write in /var/run/kubernetes

yatin (yatinkarel) wrote :

Haven't tried this, will try and update.

Spyros Trigazis (strigazi) wrote :

We can try to pass this parameter to the apiserver --cert-dir
when tls-disbled.

from the docs [1]:

      --cert-dir string The directory where the TLS certs
                               are located. If --tls-cert-file and
                               --tls-private-key-file are provided,
                               this flag will be ignored. (default
                               "/var/run/kubernetes")

with /etc/kubernetes/certs/ (make sure that the dir exists in the non-tls case)

[1] https://kubernetes.io/docs/admin/kube-apiserver/

Spyros Trigazis (strigazi) wrote :

It doesn't work, we need to have a writable directory for the kubernetes-apiserver system container OR create certs for the apiserver. I'll open an issue in atomic-system-containers

Lingxian Kong (kong) wrote :

Hi, Spyros, how we fix this in magnum ourselves?

Spyros Trigazis (strigazi) wrote :

In the v1.7.7 container images the problem is fixed (in pike). I'm not sure how useful is a non tls cluster though. With RBAC enabled it can not work. AFAIK there is no option for RBAC enabled clusters to work without tls. If there is a requirement to have non-tls cluster we can do the following separation.

Generate certs and enable rbac and expose the api at 6443 with tls, AND expose the api at 8080 as well.

Spyros Trigazis (strigazi) wrote :

In any case you need the fixed images with 1.7.7 or higher to have the proper mounts and permissions.

Rick Cano (ledsole) wrote :

I am still experiencing this issue with the latest Fedora Atomic (Fedora-Atomic-27-20180326.1.x86_64.qcow2 as of this writing) on Queens. Do I need to build my own atomic image with disk image-builder for this to work?

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.