[ovn] l3ha and distributed router extra attributes do not reflect OVN state

[Dmitrii Shcherbakov]

With https://bugs.launchpad.net/neutron/+bug/1995974 fixed and https://review.opendev.org/c/openstack/neutron/+/864051 merged extra attributes such as `distributed` and `ha` are now created for OVN routers as well.

Their default values are taken from the global configuration options more relevant for default L3 service plugin implementation based on Linux network namespaces

https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/db/l3_attrs_db.py#L24-L27
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_hamode_db.py#L21
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_dvr_db.py#L19-L27

as opposed to relying on the OVN-specific options. For instance, it order to enable the support for distributed floating IPs there is an OVN-specific global option that enables this mode for all OVN routers:

https://github.com/openstack/neutron/blob/598fcb437a0ad3d564435799c70f38429ab4f0eb/neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py#L133-L140

As a result, OVN routers now have the `distributed` property set to `False` by default (unless the global ML2/ovs-specific default is changed) and it does not reflect the state of the `ovn/enable_distributed_floating_ip` option. It can also be changed via the API on the router without any apparent effect.

The ML2/ovs and ML2/ovn comparison docs still refer to OVN-based router having no `l3ha` or `distributed` attributes whereas this is not the case anymore:
https://github.com/openstack/neutron/blame/cd66232c2b26cb4141c2e9426ce2dec0f38c364c/doc/source/ovn/faq/index.rst#L16-L29

One place where it becomes relevant is the neutron-dynamic-routing project which relies on the `distributed` property to determine whether to add /32 routes with next-hops set to a router gateway port IP (centralized FIPs case) or not (distributed FIPs case).

https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L564
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L567-L580

For distributed routers the logic is such that IP addresses of ports with a device owner set to `network:floatingip_agent_gateway` are used as a next hop for /32 routes, however, the OVN-based L3 service plugin implementation (OVNL3RouterPlugin) does not create those on a per external network basis much like the core L3RouterPlugin-based implementation does with DVR.

As a result, if an operator uses distributed FIPs with OVN with the router attribute `distributed == False`, neutron-dynamic-routing will advertise /32 routes with the centralized FIP logic (the southbound traffic would go via the router gateway port).

On the other hand, if an operator uses distributed FIPs with OVN with the router attribute `distributed == True`, neutron-dynamic-routing will not advertise anything because the centralized routes will not be added as the router seems to be distributed whereas there are no `network:floatingip_agent_gateway` ports created with OVN.

There are at least two outputs to expect as a fix:

1) Make sure the distributed state is reflected correctly for OVN routers based on the OVN-specific config option;
2) Fix neutron-dynamic routing to still create centralized /32 routes if there are not any `network:floatingip_agent_gateway` ports
OR change the OVN implementation to create those for the purposes of direct southbound routing purposes.

A similar approach can be done for the `ha` attribute but based on OVN's approach to it that does not rely on VRRP.

=====

A note on `network:floatingip_agent_gateway` ports with ML2/ovs:

They were originally added to implement the "fast exit" RFE: https://bugs.launchpad.net/neutron/+bug/1577488
https://review.opendev.org/c/openstack/neutron/+/283757

One *other* reason for having floatingip_agent_gateway ports is that Neutron allows for multiple subnets per network with different service types. So floating IPs can be allocated from a subnet (with "network:floatingip" service type https://github.com/openstack/neutron-lib/blob/fd0468f79343f47410a7932115acf7c52ceeb1c3/neutron_lib/constants.py#L55) different from the subnet used for other purposes like allocation of gateway IPs ("network:router_gateway" service type)

https://docs.openstack.org/neutron/latest/admin/config-service-subnets.html

In that case, the network fabric will need a route with a next hop at the OpenStack side through which a FIP will be reachable. And this is despite the fact an external network is a single-segment one (shared L2).

NDR currently tries to look up a next hop based on ports with `network:floatingip_agent_gateway` device owner for a floating IP associated with a `distributed == True` router.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Patch: https://review.opendev.org/c/openstack/neutron/+/886992

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Slawek Kaplonski <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/886992
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/886992
Committed: https://opendev.org/openstack/neutron/commit/1f1824397dd4e2cfd2327b2d9fec7f57453b45d2
Submitter: "Zuul (22348)"
Branch: master

commit 1f1824397dd4e2cfd2327b2d9fec7f57453b45d2
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Jun 26 16:50:49 2023 +0000

    [OVN] Populate the "router.distributed" flag in ML2/OVN

    In ML2/OVN there is a static common configuration parameter to define
    if the routers (more in particular the floating IPs) can be distributed
    or centralized:
      [ovn] enable_distributed_floating_ip

    This patch writes this value on each new router created. It also
    implements a maintenance method to populate this flag when the
    Neutron API is restarted and the value changed.

    Closes-Bug: #2022058
    Change-Id: Ib109b09fde4db8738c1d0b3e394c201492d210c6

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 24.0.0.0b1 development milestone.