[RFE] Openflow-based DVR L3

[Yi Yang]

This will use openflow to implement neutron l3 agent for DVR mode.

It will implement the below functions:

1. ARP Reply for local VMs.
2. ARP Reply for qrouter port.
3. ICMPv4 & v6 Reply for qrouter port.
4. Routing between VMs.
5. ARP request for remote FIP or FIP gateway triggered by packet_in2.
6. Enhance os_ken to handle advanced packet_in (https://review.opendev.org/c/openstack/os-ken/+/795963)
7. Openflow-based FIP (without iptables/conntrack involved)
8. Keep namespace based implementation there for smooth transition and debug (you still can use namespaces to connect to VMs)
9. Openflow-based FIP port forwarding.

All these functions are implemented by openflow in br-int. neutron ovs agent and neutron l3 agent and os_ken will be changed to support this. neutron AIPs won't have any change.

[LIU Yulong (dragon889)]

There was a BP [1][2] which had WIP works [3] for it. But the Neutron team decided not to continue the work for it since they assume that OVN is an alternative to achive flow based DVR. I don't know if the Neutron team can accept this again. But if you want to take over this, maybe re-open that BP can be enough.

[1] https://blueprints.launchpad.net/neutron/+spec/openflow-based-dvr
[2] https://bugs.launchpad.net/neutron/+bug/1705536
[3] https://review.opendev.org/q/topic:%2522openflow-based-dvr%2522

[Lajos Katona (lajos-katona)]

Thanks for propsing this RFE.
On a following Neutron Drivers meeting ([1]) the drivers team will discuss it, please check the mails on <email address hidden> when this RFE will be topic on the meeting (subject like: [neutron] Drivers meeting - agenda....)

[0] https://meetings.opendev.org/#Neutron_drivers_Meeting

[Yi Yang (yangyi01)]

Although OVN is default option since Victoria, OVN isn't best choice in many scenarios, especially in old deployed openstack versions. Moreover non-OVN options are still there, most users are still using them.

My proposal used different way to implement Openflow-based DVR, it is totally different from https://blueprints.launchpad.net/neutron/+spec/openflow-based-dvr, I have implemented it in our neutron, just hope that more users can use it if they're interested in it.

[Yi Yang (yangyi01)]

Thanks Lajos, I'm preparing spec, once it is submitted, we can discuss it in https://meetings.opendev.org/#Neutron_drivers_Meeting

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/796746

[Yi Yang (yangyi01)]

Also added a blueprint

https://blueprints.launchpad.net/neutron/+spec/openflow-based-dvr-l3

[shihanzhang (shihanzhang)]

hi Yi Yang, do you plan to implement push-notification for L3 agent, i think the performance of l3 agent rpc is serious than data plane.

[Yi Yang (yangyi01)]

I don't think RPC can impact on data plane performance. Can you tell me how often RPCs you mentioned are called, in what cases they will be called? In my perspective, they are called only if operator does some changes (such as bind a FIP, add a subnet to router, etc.)

[shihanzhang (shihanzhang)]

yes, RPC issues dose not impact on data plane performance, it's control plane issues, when there are many routers bonding to a l3 agent, the RPC call always time out if l3 agent restart.

[Yi Yang (yangyi01)]

Then, the same issue will still be there even if you use notification to replace rpc for this case unless a notification can include all the routers' information. This is a rabbitmq issue actually.

[shihanzhang (shihanzhang)]

i think rabbitmq is just a message channel, the problem is the RPC call method, maybe it's better to use RPC cast to notify router's information with OVO object like current push-notification mechanism.

[Slawek Kaplonski (slaweq)]

We discussed that RFE on the today's drivers meeting https://meetings.opendev.org/meetings/neutron_drivers/2021/neutron_drivers.2021-06-18-14.00.log.html#l-19 and finally we decided to approve it. But it should be kept in mind that final goal of this rfe should be to replace currently DVR implementation with new, OF based one.
Also we will need detailed spec with details about how OF rules will be created, where and how all that will work. It will also need to be included in the documentation which should be part of the implementation patch(es).

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/797221

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/797252

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Yi Yang <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/797252
Reason: Wrong Change-Id

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/797221
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Yi Yang (yangyi01)]

I have been working on this and will submit our implementation code to help reviewers understand it better and also hope we can get more feed backs.

For this spec, I'll add ascii-based graph after submitting code.

By the way, I can't see restore button, can you folks help reactivate it?

[Yi Yang (yangyi01)]

Sorry, please ignore the above comment, it should go to https://review.opendev.org/c/openstack/neutron/+/797221.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/os-ken/+/795963
Committed: https://opendev.org/openstack/os-ken/commit/bf25f07861a93c1cfc1592e181baebc8a161b2aa
Submitter: "Zuul (22348)"
Branch: master

commit bf25f07861a93c1cfc1592e181baebc8a161b2aa
Author: Yi Yang <email address hidden>
Date: Fri Jun 11 04:10:59 2021 -0400

    Enhance os-ken to support Nicira PacketIn2

    neutron ovs agent has to handle advanced packet_in
    with custom userdata which is necessary to openflow-based
    floating IP implementation, but current os-ken doesn't
    have such infrastructure.

    This story is to enhance os-ken in order that neutron ovs
    agent can act as an advanced SDN controller to handle
    advanced packet_in.

    Story: #2008964
    Task: #42600
    Related-Bug: #1931953

    Signed-off-by: Yi Yang <email address hidden>

    Change-Id: I4f993be5dd4a5f48396c4be152da5b61ea92392f

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/804832

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/805930

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/804832
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/797221
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/805930
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/796746
Reason: Superseded by https://review.opendev.org/c/openstack/neutron-specs/+/797252

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/805930
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron-specs/+/797252
Reason: Abandoned because of lack of activity. Please feel free to restore it and address the comments.