Default CNI plugins are not available

[Brent Rowsell]

Plugins dhcp and static are not available on StarlingX.
The following changes are required:

1) Package containernetworking-plugins need to be upgraded to containernetworking-plugins-0.8.1-1.el7.centos.x86_64.rpm for the static plugin. This is available in the centos mirror.
This will install the default plugins to /usr/libexec/cni

2) Calico currently installs to /opt/cni/bin. Calico also installs a subset of the default plugins.
The calico install, https://github.com/projectcalico/cni-plugin/blob/master/k8s-install/scripts/install-cni.sh, will need to be updates to reference /usr/libexec/cni and not overwrite the default plugins.

3) Multus and sriov install will also need to be updated as it also reference /opt/cni/bin

4) Update kubelet option,--cni-bin-dir, to point to /usr/libexec/cni

[Ghada Khalil (gkhalil)]

Marking for stx.3.0 - missing functionality due to old package in stx

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/680408

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/680409

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/680410

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/680408
Committed: https://git.openstack.org/cgit/starlingx/tools/commit/?id=29cac7d1ebc2435c40e0373b1aee84e28dac5037
Submitter: Zuul
Branch: master

commit 29cac7d1ebc2435c40e0373b1aee84e28dac5037
Author: Steven Webster <email address hidden>
Date: Tue Sep 3 08:27:44 2019 -0500

    Add the containernetworking-plugins package

    This commit adds the containernetworking-plugins package, which
    brings in a number of CNI plugins including static and dhcp.

    The containernetworking-cni package is removed, as it used to
    contain the actual plugins before a new separate package /
    project was created.

    Change-Id: Ia6db0103fff8edadef0ec1c881b766bf7e5f661a
    Closes-Bug: #1840391
    Signed-off-by: Steven Webster <email address hidden>

[Ghada Khalil (gkhalil)]

Re-opening as more commits still need to merge before this launchpad is fully addressed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/684173

[OpenStack Infra (hudson-openstack)]

Change abandoned by Steven Webster (<email address hidden>) on branch: master
Review: https://review.opendev.org/680410
Reason: Abandoning as these files have moved to stx-puppet. Will create a new review and remove the obsoleted conf.erb files (replaced with ansible equivalents)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/685082

[OpenStack Infra (hudson-openstack)]

Change abandoned by Steven Webster (<email address hidden>) on branch: master
Review: https://review.opendev.org/685082

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/680409
Committed: https://git.openstack.org/cgit/starlingx/ansible-playbooks/commit/?id=aa6c5763bd808d1062070dac1717dc2f571ce471
Submitter: Zuul
Branch: master

commit aa6c5763bd808d1062070dac1717dc2f571ce471
Author: Steven Webster <email address hidden>
Date: Fri Aug 30 14:08:23 2019 -0500

    Enable default CNI plugins

    Currently, only a subset of the default CNI plugins are enabled in
    StarlingX. These plugins are provided by Calico and placed in the
    /opt/cni/bin directory.

    On CentOS, the full set of CNI plugins that are distributed by the
    containernetworking-plugins package are installed under the
    default directory for system daemons and utilities at
    /usr/libexec/cni

    This means that some of the default plugins such as static ipam and
    dhcp are not available to be used, as kublet looks at the
    /opt/cni/bin directory by default.

    This commit changes the cni-bin volume of the Calico, Multus, and
    SRIOV CNI daemonsets to refer to the plugin executable path used by
    the containernetworking-plugins package. As such, it is also
    required that the --cni-bin-dir option now be passed to kubelet.

    Depends-on: https://review.opendev.org/680408
    Closes-Bug: #1840391
    Change-Id: I9a0cbd63012d9a748826f24ae935823693048044
    Signed-off-by: Steven Webster <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/684173
Committed: https://git.openstack.org/cgit/starlingx/stx-puppet/commit/?id=295410ed3dfce8485ee1ef9fa70f635049d0c9c4
Submitter: Zuul
Branch: master

commit 295410ed3dfce8485ee1ef9fa70f635049d0c9c4
Author: Steven Webster <email address hidden>
Date: Tue Sep 17 10:06:36 2019 -0500

    Enable default CNI plugins

    Currently, only a subset of the default CNI plugins are enabled in
    StarlingX. These plugins are provided by Calico and placed in the
    /opt/cni/bin directory.

    On CentOS, the full set of CNI plugins that are distributed by the
    containernetworking-plugins package are installed under the
    default directory for system daemons and utilities at
    /usr/libexec/cni

    This means that some of the default plugins such as static ipam and
    dhcp are not available to be used, as kublet looks at the
    /opt/cni/bin directory by default.

    This commit changes the cni-bin volume of the Calico, Multus, and
    SRIOV CNI daemonsets to refer to the plugin executable path used by
    the containernetworking-plugins package. As such, it is also
    required that the --cni-bin-dir option now be passed to kubelet.

    In addition, since the config_controller --force option has been
    removed, we can safely remove the calico, multus, and SRIOV puppet
    templates. They have already been replaced by the equivalent
    ansible bootstrap templates.

    Depends-on: https://review.opendev.org/680409
    Change-Id: I5a21ccc7cb06ab8f50df19b513335cd4a16211f5
    Partial-Bug: #1840391
    Signed-off-by: Steven Webster <email address hidden>