populate eswitch fdb of SR-IOV-enabled NICs via l2 agent FDB extension

[Dmitrii Shcherbakov]

[Summary]
The neutron-api charm needs to expose a new fdb extension configuration option to support population of SR-IOV MAC addresses into OVS.

[Original Bug Report]
In order for instances using SR-IOV to communicate with instances using OVS or linux bridge on the same hypervisor host, the embedded switch (eswitch) forwarding database (FDB) of an SR-IOV NIC must be updated with a MAC address of an OVS/linux bridge port. This is done by running `bridge fdb add <vm_ovs_port_mac> <pf_interface_plugged_into_ovs>`.

where vm_ovs_port_mac is a mac address of a VM port plugged directly into ovs (provider networks case without vrouters). DVR is not handled in the Neutron codebase currently (*) because device owner types are different and proxy-arp handling needs to be done properly (i.e. fdb add <fg_device_in_fip_namespace_mac> <pf_interface_plugged_into_ovs>). The service subnets + DVR case is not handled at all and is not possible to implement because with L3 agent port and floating IP subnet separation VFs and floating ip ports do not share a L2 and hence adding a MAC to eswitch fdb is irrelevant (the traffic will always be forwarded to the next hop from a VF to reach a FIP).

From the charm perspective, ML2 extension should be added using the same mechanism as the QoS extension: settings from neutron-api need to be propagated to neutron-gateway and neutron-openvswitch and the shared_physical_device_mappings config should be added to neutron-gateway and neutron-openvswitch config.

https://docs.openstack.org/neutron/queens/admin/config-sriov.html#optional-fdb-l2-agent-extension

l2 extension commit commit:
https://github.com/openstack/neutron/commit/2c8f61b816bf531a17a7b45d35a5388e8a2f607a
neutron.agent.l2.extensions.fdb_population:FdbPopulationAgentExtension

# this scenario is listed on slide 33
#https://events.static.linuxfound.org/sites/events/files/slides/LinuxConJapan2014_makita_0.pdf
# bridge fdb add <ovs-port-mac> <pf_interface_name>
bridge fdb add de:ad:be:e:f:00 <pf_interface_name>

# this invocation in the code
https://github.com/openstack/neutron/blob/stable/queens/neutron/agent/linux/bridge_lib.py#L117-L119

https://github.com/openstack/neutron/blob/stable/queens/neutron/agent/l2/extensions/fdb_population.py#L81-L102
        def update_port(self, device, port_id, mac):
# ...
                bridge_lib.FdbInterface.add(mac, device)
# ...

config:
https://github.com/openstack/neutron/blob/stable/queens/neutron/conf/agent/l2_ext_fdb_population.py
shared_physical_device_mappings
                help=_("Comma-separated list of <physical_network>:<network_device> tuples mapping physical network names to the agent's node-specific shared physical network device between SR-IOV and OVS or SR-IOV and linux bridge"))

This functionality will not work with DVR though for several reasons:

1) only compute:nova and network:router_interface interfaces

(*) DVR is not handled judging by the code and PERMITTED_DEVICE_OWNERS
https://github.com/openstack/neutron/commit/2c8f61b816bf531a17a7b45d35a5388e8a2f607a#diff-05571ca75c330859ba720dc1e0737896R53
    # FDB udpates are triggered for ports with a certain device_owner only:
    # - device owner "compute": updates the FDB with normal port instances,
    # required in order to enable communication between
    # SR-IOV direct port instances and normal port instance.
    # - device owner "router_interface": updates the FDB woth OVS/LB ports,
    # required in order to enable communication for SR-IOV instances
    # with floating ip that are located with the network node.
    # - device owner "DHCP": not required because messages are broadcast.
    PERMITTED_DEVICE_OWNERS = {'compute', 'network:router_interface'}