[RFE] Floating IP Subnets on Routed Provider Networks

[John Davidge]

This RFE proposes an enhancement to the Routed Provider Networks[1][2] feature to allow the creation of Floating IP subnets. The planned implementation will leverage the BGP Dynamic Routing[3] and Service Subnets[4] features to achieve this.

Problem Description
===================

Currently, all subnets on a Routed Network are required to belong to a segment, which typically corresponds to the physical rack on which all VMs for the subnet will be provisioned. This doesn't make sense for a Floating IP subnet, which will not be associated with a neutron segment as its address range will span more than a single rack. Additionally, no mechanism currently exists to provide routing from a floating IP to a specific Routed Network Segment.

Proposed Solution
=================

To solve the first part of the problem, Service Subnets[4] will be used to allow a Floating IP subnet to be created on a Routed Network without the need for subnet/segment association. Specifically, a subnet with service_type "network:floating_ip" will not be subject to the segment association check.

To solve the second part, BGP Dynamic Routing[3] will be used to advertise the ToR for the relevant segment as the next-hop for assigned Floating IPs. This will require enhancing the BGP Agent to be segment-aware, rather than just network-aware.

[1] https://specs.openstack.org/openstack/neutron-specs/specs/newton/routed-networks.html
[2] https://docs.openstack.org/newton/networking-guide/config-routed-networks.html
[3] https://docs.openstack.org/newton/networking-guide/config-bgp-dynamic-routing.html
[4] https://docs.openstack.org/newton/networking-guide/config-service-subnets.html

[Miguel Lavalle (minsel)]

As discussed today in the drivers meeting, next step is a spec and a blue print (e.g. https://blueprints.launchpad.net/neutron/+spec/enginefacade-switch)

[Javier Diaz Jr (javierdiazcharles)]

Any progress or ETAs on this RFE?

[Ryan Tidwell (ryan-tidwell)]

See https://review.opendev.org/#/c/486450/ for design details that are being hashed out. I'm in the process of proposing an initial patch for this, look for it shortly. This initial patch will demonstrate what I'm proposing in the design spec. Reviews on https://review.opendev.org/#/c/486450/ would be greatly appreciated.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/669395

[Takashi Sogabe (sogabe)]

Are there any blockers around the RFE?
I evaluated the configuration by using devstack with PoC code. It seems work well and should be a little impact for upstream code.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/741429

[OpenStack Infra (hudson-openstack)]

Change abandoned by Thomas Goirand (<email address hidden>) on branch: master
Review: https://review.opendev.org/741429

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/741607
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=b64d4e2423da8b1664ec632a849b4d8c0892e20e
Submitter: Zuul
Branch: master

commit b64d4e2423da8b1664ec632a849b4d8c0892e20e
Author: Thomas Goirand <email address hidden>
Date: Fri Jul 17 11:38:37 2020 +0200

    Add DEVICE_OWNER_ROUTED constant

    This patch adds the DEVICE_OWNER_ROUTED constant.

    Related-Bug: #1667329
    Change-Id: Ibde33bdacba6bd1e9c41cc69d0054bf55e1e6454

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/486450
Committed: https://git.openstack.org/cgit/openstack/neutron-specs/commit/?id=3f584dc9b72fb883d82296dad7c413e51fd45b15
Submitter: Zuul
Branch: master

commit 3f584dc9b72fb883d82296dad7c413e51fd45b15
Author: Miguel Lavalle <email address hidden>
Date: Sun Jul 23 19:38:13 2017 -0500

    Floating IP's for routed networks

    This specification proposes to add functionality that allows users to
    associate floating IPs to ports of routed networks

    Partial-Bug #1667329

    Change-Id: Id07f52da9fff321827aff8be11614a347030bb04

[Brian Haley (brian-haley)]

Was this completed in https://review.opendev.org/c/openstack/neutron/+/669395 ? Is there any more work to do? Thanks.

[Dmitrii Shcherbakov (dmitriis)]

Brian, I believe the original post's note about requiring changes to neutron-dynamic-routing ("This will require enhancing the BGP Agent to be segment-aware, rather than just network-aware.") hasn't been done.

At least, I haven't seen any changes or mention of multiple segments in NDR or mentions of DEVICE_OWNER_ROUTED.

The next-hop lookup for advertised routes hasn't had any changes since the new service type was added to Neutron either:

https://github.com/openstack/neutron-dynamic-routing/blob/15d30d02631ccd92a25a3e45827c664a7eda8149/neutron_dynamic_routing/db/bgp_db.py#L1100-L1100
https://github.com/openstack/neutron-dynamic-routing/blob/15d30d02631ccd92a25a3e45827c664a7eda8149/neutron_dynamic_routing/db/bgp_db.py#L1133-L1142

Based on the change you linked and the doc it introduced (https://docs.openstack.org/neutron/latest/admin/config-bgp-floating-ip-over-l2-segmented-network.html) it looks like the L3 agent-based DVR case may already be working:

1) subnets with the service type "network:floatingip_agent_gateway" are created with a segment associated - so ports in fip-<ext-net-id> namespaces get IPs allocated from a subnet of a relevant segment.
2) a subnet with service types "network:routed" (which makes Neutron alter segment validation), "network:floatingip" and "network:router_gateway" is created without an association to a segment.

Provided that the next-hop lookup logic above returns the right next-hops, it might work. It would be good to get a comment from someone who did the lab testing of the feature and wrote the doc.

--
There is a significant change still needed at the OVN integration side: DVR is implemented for OVN such that there aren't any "network:floatingip_agent_gateway" ports created per node per provider network.

Without that, NDR's queries for next-hops of DVR FIPs will not not have any results as the ports with "network:floatingip_agent_gateway" device owner aren't there.

https://github.com/openstack/neutron-lib/blob/de6852c60d4dd8d6a33385be686ccc8833608565/neutron_lib/constants.py#L60-L61

It probably deserves a spec of itself to resolve, but I would at least have the OVN-related limitations documented in the scope of this RFE.