k8s_atomic: Remove kubelet from master nodes

[Spyros Trigazis]

Currently we start kubelet as unscheduled. Before containerizing kube [1]. we needed to run kubelet before as unscheduled to run the controller-manager, scheduler and kube-proxy as static pods. There is no such need anymore.

Let's remove kubelet and kube-proxy completely from the masters to run only the control plane on them.

[1] https://blueprints.launchpad.net/magnum/+spec/run-kube-as-container

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/514604

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/514604
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=2f69309eca5b25a2b0cde32a2291a377f9f64506
Submitter: Zuul
Branch: master

commit 2f69309eca5b25a2b0cde32a2291a377f9f64506
Author: Spyros Trigazis <email address hidden>
Date: Tue Oct 24 10:04:43 2017 +0000

    k8s_atomic: Remove kubelet and kube-proxy from master

    Currently we start kubelet as unscheduled. Before
    containerizing kube. we needed to run kubelet before as
    unscheduled to run the controller-manager, scheduler and
    kube-proxy as static pods. There is no such need anymore.

    Change-Id: I0e36606427530756d8084b643ba43880541bbe44
    Partially-Implements: blueprint run-kube-as-container
    Closes-Bug: #1726482

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/533593

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/magnum 6.0.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/542742

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/533593
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=2329cb7fb4d197e49d6c07d37b2f7ec14a11c880
Submitter: Zuul
Branch: master

commit 2329cb7fb4d197e49d6c07d37b2f7ec14a11c880
Author: Spyros Trigazis <email address hidden>
Date: Mon Jan 15 11:16:02 2018 +0100

    k8s: Fix kubelet, add RBAC and pass e2e tests

    Due to a few several small connected patches for the
    fedora atomic driver, this patch includes 4 smaller patches.

    Patch 1:
    k8s: Do not start kubelet and kube-proxy on master

    Patch [1], misses the removal of kubelet and kube-proxy from
    enable-services-master.sh and therefore they are started if they
    exist in the image or the script will fail.

    https://review.openstack.org/#/c/533593/
    Closes-Bug: #1726482

    Patch 2:
    k8s: Set require-kubeconfig when needed

    From kubernetes 1.8 [1] --require-kubeconfig is deprecated and
    in kubernetes 1.9 it is removed.

    Add --require-kubeconfig only for k8s <= 1.8.

    [1] https://github.com/kubernetes/kubernetes/issues/36745

    Closes-Bug: #1718926

    https://review.openstack.org/#/c/534309/

    Patch 3:
    k8s_fedora: Add RBAC configuration

    * Make certificates and kubeconfigs compatible
      with NodeAuthorizer [1].
    * Add CoreDNS roles and rolebindings.
    * Create the system:kube-apiserver-to-kubelet ClusterRole.
    * Bind the system:kube-apiserver-to-kubelet ClusterRole to
      the kubernetes user.
    * remove creation of kube-system namespaces, it is created
      by default
    * update client cert generation in the conductor with
      kubernetes' requirements
    * Add --insecure-bind-address=127.0.0.1 to work on
      multi-master too. The controller manager on each
      node needs to contact the apiserver (on the same node)
      on 127.0.0.1:8080

    [1] https://kubernetes.io/docs/admin/authorization/node/

    Closes-Bug: #1742420
    Depends-On: If43c3d0a0d83c42ff1fceffe4bcc333b31dbdaab
    https://review.openstack.org/#/c/527103/

    Patch 4:
    k8s_fedora: Update coredns config to pass e2e

    To pass the e2e conformance tests, coredns needs to
    be configured with POD-MODE verified. Otherwise, pods
    won't be resolvable [1].

    [1] https://github.com/coredns/coredns/tree/master/plugin/kubernetes

    https://review.openstack.org/#/c/528566/
    Closes-Bug: #1738633

    Change-Id: Ibd5245ca0f5a11e1d67a2514cebb2ffe8aa5e7de

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/542742
Committed: https://git.openstack.org/cgit/openstack/magnum/commit/?id=eb92701e05bb57e4d608e5bc66a69ed33c82c76e
Submitter: Zuul
Branch: stable/queens

commit eb92701e05bb57e4d608e5bc66a69ed33c82c76e
Author: Spyros Trigazis <email address hidden>
Date: Mon Jan 15 11:16:02 2018 +0100

    k8s: Fix kubelet, add RBAC and pass e2e tests

    Due to a few several small connected patches for the
    fedora atomic driver, this patch includes 4 smaller patches.

    Patch 1:
    k8s: Do not start kubelet and kube-proxy on master

    Patch [1], misses the removal of kubelet and kube-proxy from
    enable-services-master.sh and therefore they are started if they
    exist in the image or the script will fail.

    https://review.openstack.org/#/c/533593/
    Closes-Bug: #1726482

    Patch 2:
    k8s: Set require-kubeconfig when needed

    From kubernetes 1.8 [1] --require-kubeconfig is deprecated and
    in kubernetes 1.9 it is removed.

    Add --require-kubeconfig only for k8s <= 1.8.

    [1] https://github.com/kubernetes/kubernetes/issues/36745

    Closes-Bug: #1718926

    https://review.openstack.org/#/c/534309/

    Patch 3:
    k8s_fedora: Add RBAC configuration

    * Make certificates and kubeconfigs compatible
      with NodeAuthorizer [1].
    * Add CoreDNS roles and rolebindings.
    * Create the system:kube-apiserver-to-kubelet ClusterRole.
    * Bind the system:kube-apiserver-to-kubelet ClusterRole to
      the kubernetes user.
    * remove creation of kube-system namespaces, it is created
      by default
    * update client cert generation in the conductor with
      kubernetes' requirements
    * Add --insecure-bind-address=127.0.0.1 to work on
      multi-master too. The controller manager on each
      node needs to contact the apiserver (on the same node)
      on 127.0.0.1:8080

    [1] https://kubernetes.io/docs/admin/authorization/node/

    Closes-Bug: #1742420
    Depends-On: If43c3d0a0d83c42ff1fceffe4bcc333b31dbdaab
    https://review.openstack.org/#/c/527103/

    Patch 4:
    k8s_fedora: Update coredns config to pass e2e

    To pass the e2e conformance tests, coredns needs to
    be configured with POD-MODE verified. Otherwise, pods
    won't be resolvable [1].

    [1] https://github.com/coredns/coredns/tree/master/plugin/kubernetes

    https://review.openstack.org/#/c/528566/
    Closes-Bug: #1738633

    Change-Id: Ibd5245ca0f5a11e1d67a2514cebb2ffe8aa5e7de

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/magnum 6.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/magnum 7.0.0 release.