[RFE] Add RPC query API to l2pop for FDB resync

[ChenjieXu]

Problem Description
=======================================
Layer 2 population (l2pop) provides L2populationRpcCallBackMixin(in /neutron/plugins/ml2/drivers/l2pop/rpc_manager/l2population_rpc.py) as a general mixin class of l2pop RPC callback. The following methods are called through RPC: add_fdb_entries(), remove_fdb_entries(), update_fdb_entries(). And the following methods are used in an agent as internal methods: fdb_add(), fdb_remove(), fdb_update().

For now L2 agent (OVS agent and Linuxbridge agent) inherits the class L2populationRpcCallBackMixin and implement their own fdb_add(), fdb_remove() and fdb_update(). However, some agents in other subprojects (e.g. dynamic-routing agent) can also inherit the class L2populationRpcCallBackMixin to enable l2pop in their networks. These agents only need to register the l2pop related topics and implement the add_fdb_entries(), remove_fdb_entries(), update_fdb_entries(), fdb_add(), fdb_remove() and fdb_update().

However, in some scenarios, the agent needs to get the full FDB from Neutron. For example: agent restart and adding a new agent. To address the above scenarios, l2pop mechanism driver will check the port number on the agent or agent uptime after updating a port. If the number equals 1 or agent uptime less than agent_boot_time, l2pop mechanism driver will send the full FDB to the agent. This mechanism works fine with l2 agent. However, if other agents which try to implement l2pop don’t have ports, then the l2pop mechanism driver won’t be triggered to check the port number on the agent or agent uptime. As a result of that, no FDB will be sent to the agent in the scenarios that the agent restarts and adding a new agent.

Based on the above analysis, we proposed to add a new mechanism to cover the scenario (agent restart and adding new agent) for other agents that may implement l2pop.

Proposed Change
=======================================
The idea is to extend the l2pop RPC handlers to allow an agent to query the full FDB for a list of network_id values. Class L2populationRpcQueryMixin should be used by the agent to query the full FDB. The agent should inherit the class L2populationRpcQueryMixin and calling the following method:
      get_fdb_entries(self, context, network_ids, host=None, source=None)
This method will send an RPC request to topic “q-plugin-l2population-query”. This method should be used to retrieve the FDB entries required to populate the FDB on a given host. If “source” is specified as a requestor then no records from that source are included in the results. If host is not supplied then the full set of FDB entries for the network is returned.

Listeners should be created in l2pop mechanism driver to consume the topic “q-plugin-l2population-query”. The RPC request to the topic “q-plugin-l2population-query” should be processed by these listeners. The following method in the mech_driver.py will be called:
      get_fdb_entries(self, context, network_ids, host=None, source=None)
Finally full FDB entries can be returned to the agent.

All changes can be viewed through the link below:
https://review.openstack.org/#/c/620929/

Data Model Impact
=======================================
None

REST API Impact
=======================================
None

Command Line Client Impact
=======================================
None

Other Impact
=======================================
None

Other Deployer Impact
=======================================
None

Performance Impact
=======================================
None

Implementation
=======================================
Assignee(s)

Work Items
=======================================
• Add class L2populationRpcCallBackMixin to neutron\plugins\ml2\drivers\l2pop\rpc_manager\l2population_rpc.py.
• Add class L2populationServerQueryAPI to neutron\plugins\ml2\drivers\l2pop\rpc.py
• Add L2populationServer, method get_fdb_entries and _get_fdb_entries to L2populationMechanismDriver in neutron\plugins\ml2\drivers\l2pop\mech_driver.py
• Add related tests.

Dependencies
=======================================
None

Testing
=======================================
Unit tests are necessary.

Documentation Impact
=======================================
None.

References
=======================================
None

[Bence Romsics (bence-romsics)]

Thank you for your RFE report.

I see that a related RFE (https://bugs.launchpad.net/neutron/+bug/1793653) was already triaged. Unless this one was discussed already please bring this similarly to the neutron-drivers meeting (every Friday at 14:00 UTC) the next time you can:

http://eavesdrop.openstack.org/#Neutron_drivers_Meeting

[ChenjieXu (midone)]

Hi Bence,

Thank you for your comment! I will propose to discuss this RFE at the neutron-drivers meeting as you said.

[ChenjieXu (midone)]

I sent an email to Ryan Tidwell to request him to review this RFE. The RPC API proposed by this RFE can be used by BGP Dynamic Routing agent. The project stx-neutron-dynamic-routing is a fork project for neutron-dynamic-routing. The project stx-neutron-dynamic-routing implements the l2pop for BGP Dynamic Routing agent:
https://github.com/starlingx-staging/stx-neutron-dynamic-routing/blob/master/neutron_dynamic_routing/services/bgp/agent/bgp_dragent.py#L1053
https://github.com/starlingx-staging/stx-neutron-dynamic-routing/blob/master/neutron_dynamic_routing/services/bgp/agent/bgp_dragent.py#L1180

The plan is to upstream these changes to neutron-dynamic-routing. However, in the short term this is not being prioritized nor has any attempt been made to approach the individual project teams about getting this accepted. So I ask Ryan to give us some inputs on the potential use.

[ChenjieXu (midone)]

This RFE is specific to the Neutron BGP-EVPN use-case which is currently not being pursued for upstreaming from stx-neutron-dynamic-routing. Therefore, until this feature is required, and an attempt is made to have it accepted by the OpenStack community, this feature cannot be used as a justification for this RFE. This RFE is being abandoned and can be revived if the BGP-EVPN feature becomes a priority.

[Miguel Lavalle (minsel)]

Thanks for the update and submission

[OpenStack Infra (hudson-openstack)]

Change abandoned by ChenjieXu (<email address hidden>) on branch: master
Review: https://review.openstack.org/620929
Reason: The related RFE has been abandoned and can be revived if the BGP-EVPN feature becomes a priority.