direct ports are in DOWN state after attaching to PODs

[Danil Golov]

https://review.openstack.org/#/c/512281/
https://review.openstack.org/#/c/512280/
These two patches implement a SR-IOV functionality for kuryr-k8s. It allow to use direct neutron ports for PODs.

Problem:
After POD is in Running state, direct neutron ports remain in DOWN state, that is wrong behaviour. However it doesn't causes any problems with functionality of these ports.

Decision:
There are some ways to fix this problem:
1. Make some changes in sriov-nic-agent that should be installed in all worker nodes that use SR-IOV functionality to handle all used VFs (both attached to VMs and PODs). It allows to change status of ports to BUILD.
2. Kuryr-k8s-controller should have information about PCI devices from worker nodes to make neutron update this information in port DB. This could be done with a few different ways:
    2.1. Use CRD or annotations to pass information from worker nodes to master
    2.2. Change device plugins to annotate master

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/642703
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=b0ce30142ececa9ad8d5508e9ed54930c63d3817
Submitter: Zuul
Branch: master

commit b0ce30142ececa9ad8d5508e9ed54930c63d3817
Author: Danil Golov <email address hidden>
Date: Mon Mar 11 15:15:26 2019 +0300

    Annotate nodes with pci info for direct ports

    This commit allows to add pci information for direct
    neutron ports attached to PODs into nodes annotations.
    It happens on binding stage when pci information can
    be requested.

    Also this commit allows to delete annotations for such
    ports when apropriate POD is deleted and VFs are returned
    into host's network namespace.

    For future commits: it is necessary to update neutron
    ports with pci info when POD is in Running state.

    Change-Id: I5ea8da6bb3143fd689e701f5e503488d4a6c9b33
    Closes-Bug: 1818606
    Signed-off-by: Danil Golov <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/642704
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=14a685d9e143c6a6235d0656e31472637986ebee
Submitter: Zuul
Branch: master

commit 14a685d9e143c6a6235d0656e31472637986ebee
Author: Danil Golov <email address hidden>
Date: Tue Mar 12 11:28:02 2019 +0300

    Update sriov neutron ports with pci info

    This commit updates neutron sriov ports when
    POD that contains it is in Running state.
    It makes request to neutron API to update ports
    with binding:profile information.
    Due to this it is required to make actions with
    privileged user having admin rights.

    Change-Id: Iade446cefd78b494291729939db7db75692392c9
    Closes-Bug: 1818606
    Signed-off-by: Danil Golov <email address hidden>

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kuryr-kubernetes 1.1.0 release.