| 2023-06-01 11:37:45 |
Dmitrii Shcherbakov |
bug |
|
|
added bug |
| 2023-06-01 13:44:43 |
Dmitrii Shcherbakov |
description |
With https://bugs.launchpad.net/neutron/+bug/1995974 fixed and https://review.opendev.org/c/openstack/neutron/+/864051 merged extra attributes such as `distributed` and `ha` are now created for OVN routers as well.
Their default values are taken from the global configuration options more relevant for default L3 service plugin implementation based on Linux network namespaces
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/db/l3_attrs_db.py#L24-L27
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_hamode_db.py#L21
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_dvr_db.py#L19-L27
as opposed to relying on the OVN-specific options. For instance, it order to enable the support for distributed floating IPs there is an OVN-specific global option that enables this mode for all OVN routers:
https://github.com/openstack/neutron/blob/598fcb437a0ad3d564435799c70f38429ab4f0eb/neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py#L133-L140
As a result, OVN routers now have the `distributed` property set to `False` by default (unless the global ML2/ovs-specific default is changed) and it does not reflect the state of the `ovn/enable_distributed_floating_ip` option. It can also be changed via the API on the router without any apparent effect.
The ML2/ovs and ML2/ovn comparison docs still refer to OVN-based router having no `l3ha` or `distributed` attributes whereas this is not the case anymore:
https://github.com/openstack/neutron/blame/cd66232c2b26cb4141c2e9426ce2dec0f38c364c/doc/source/ovn/faq/index.rst#L16-L29
One place where it becomes relevant is the neutron-dynamic-routing project which relies on the `distributed` property to determine whether to add /32 routes with next-hops set to a router gateway port IP (centralized FIPs case) or not (distributed FIPs case).
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L564
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L567-L580
For distributed routers the logic is such that IP addresses of ports with a device owner set to `floatingip_agent_gateway` are used as a next hop for /32 routes, however, the OVN-based L3 service plugin implementation (OVNL3RouterPlugin) does not create those on a per external network bases much like the core L3RouterPlugin-based implementation does with DVR.
As a result, if an operator uses distributed FIPs with OVN with the router attribute `distributed == False`, neutron-dynamic-routing will advertise /32 routes with the centralized FIP logic (the southbound traffic would go via the router gateway port).
On the other hand, if an operator uses distributed FIPs with OVN with the router attribute `distributed == True`, neutron-dynamic-routing will not advertise anything because the centralized routes will not be added as the router seems to be distributed whereas there are no `floatingip_agent_gateway` ports created with OVN.
There are at least two outputs to expect as a fix:
1) Make sure the distributed state is reflected correctly for OVN routers based on the OVN-specific config option;
2) Fix neutron-dynamic routing to still create centralized /32 routes if there are not any `floatingip_agent_gateway` ports
OR change the OVN implementation to create those for the purposes of direct southbound routing purposes. |
With https://bugs.launchpad.net/neutron/+bug/1995974 fixed and https://review.opendev.org/c/openstack/neutron/+/864051 merged extra attributes such as `distributed` and `ha` are now created for OVN routers as well.
Their default values are taken from the global configuration options more relevant for default L3 service plugin implementation based on Linux network namespaces
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/db/l3_attrs_db.py#L24-L27
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_hamode_db.py#L21
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_dvr_db.py#L19-L27
as opposed to relying on the OVN-specific options. For instance, it order to enable the support for distributed floating IPs there is an OVN-specific global option that enables this mode for all OVN routers:
https://github.com/openstack/neutron/blob/598fcb437a0ad3d564435799c70f38429ab4f0eb/neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py#L133-L140
As a result, OVN routers now have the `distributed` property set to `False` by default (unless the global ML2/ovs-specific default is changed) and it does not reflect the state of the `ovn/enable_distributed_floating_ip` option. It can also be changed via the API on the router without any apparent effect.
The ML2/ovs and ML2/ovn comparison docs still refer to OVN-based router having no `l3ha` or `distributed` attributes whereas this is not the case anymore:
https://github.com/openstack/neutron/blame/cd66232c2b26cb4141c2e9426ce2dec0f38c364c/doc/source/ovn/faq/index.rst#L16-L29
One place where it becomes relevant is the neutron-dynamic-routing project which relies on the `distributed` property to determine whether to add /32 routes with next-hops set to a router gateway port IP (centralized FIPs case) or not (distributed FIPs case).
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L564
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L567-L580
For distributed routers the logic is such that IP addresses of ports with a device owner set to `floatingip_agent_gateway` are used as a next hop for /32 routes, however, the OVN-based L3 service plugin implementation (OVNL3RouterPlugin) does not create those on a per external network basis much like the core L3RouterPlugin-based implementation does with DVR.
As a result, if an operator uses distributed FIPs with OVN with the router attribute `distributed == False`, neutron-dynamic-routing will advertise /32 routes with the centralized FIP logic (the southbound traffic would go via the router gateway port).
On the other hand, if an operator uses distributed FIPs with OVN with the router attribute `distributed == True`, neutron-dynamic-routing will not advertise anything because the centralized routes will not be added as the router seems to be distributed whereas there are no `floatingip_agent_gateway` ports created with OVN.
There are at least two outputs to expect as a fix:
1) Make sure the distributed state is reflected correctly for OVN routers based on the OVN-specific config option;
2) Fix neutron-dynamic routing to still create centralized /32 routes if there are not any `floatingip_agent_gateway` ports
OR change the OVN implementation to create those for the purposes of direct southbound routing purposes. |
|
| 2023-06-01 13:46:16 |
Dmitrii Shcherbakov |
description |
With https://bugs.launchpad.net/neutron/+bug/1995974 fixed and https://review.opendev.org/c/openstack/neutron/+/864051 merged extra attributes such as `distributed` and `ha` are now created for OVN routers as well.
Their default values are taken from the global configuration options more relevant for default L3 service plugin implementation based on Linux network namespaces
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/db/l3_attrs_db.py#L24-L27
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_hamode_db.py#L21
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_dvr_db.py#L19-L27
as opposed to relying on the OVN-specific options. For instance, it order to enable the support for distributed floating IPs there is an OVN-specific global option that enables this mode for all OVN routers:
https://github.com/openstack/neutron/blob/598fcb437a0ad3d564435799c70f38429ab4f0eb/neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py#L133-L140
As a result, OVN routers now have the `distributed` property set to `False` by default (unless the global ML2/ovs-specific default is changed) and it does not reflect the state of the `ovn/enable_distributed_floating_ip` option. It can also be changed via the API on the router without any apparent effect.
The ML2/ovs and ML2/ovn comparison docs still refer to OVN-based router having no `l3ha` or `distributed` attributes whereas this is not the case anymore:
https://github.com/openstack/neutron/blame/cd66232c2b26cb4141c2e9426ce2dec0f38c364c/doc/source/ovn/faq/index.rst#L16-L29
One place where it becomes relevant is the neutron-dynamic-routing project which relies on the `distributed` property to determine whether to add /32 routes with next-hops set to a router gateway port IP (centralized FIPs case) or not (distributed FIPs case).
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L564
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L567-L580
For distributed routers the logic is such that IP addresses of ports with a device owner set to `floatingip_agent_gateway` are used as a next hop for /32 routes, however, the OVN-based L3 service plugin implementation (OVNL3RouterPlugin) does not create those on a per external network basis much like the core L3RouterPlugin-based implementation does with DVR.
As a result, if an operator uses distributed FIPs with OVN with the router attribute `distributed == False`, neutron-dynamic-routing will advertise /32 routes with the centralized FIP logic (the southbound traffic would go via the router gateway port).
On the other hand, if an operator uses distributed FIPs with OVN with the router attribute `distributed == True`, neutron-dynamic-routing will not advertise anything because the centralized routes will not be added as the router seems to be distributed whereas there are no `floatingip_agent_gateway` ports created with OVN.
There are at least two outputs to expect as a fix:
1) Make sure the distributed state is reflected correctly for OVN routers based on the OVN-specific config option;
2) Fix neutron-dynamic routing to still create centralized /32 routes if there are not any `floatingip_agent_gateway` ports
OR change the OVN implementation to create those for the purposes of direct southbound routing purposes. |
With https://bugs.launchpad.net/neutron/+bug/1995974 fixed and https://review.opendev.org/c/openstack/neutron/+/864051 merged extra attributes such as `distributed` and `ha` are now created for OVN routers as well.
Their default values are taken from the global configuration options more relevant for default L3 service plugin implementation based on Linux network namespaces
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/db/l3_attrs_db.py#L24-L27
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_hamode_db.py#L21
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_dvr_db.py#L19-L27
as opposed to relying on the OVN-specific options. For instance, it order to enable the support for distributed floating IPs there is an OVN-specific global option that enables this mode for all OVN routers:
https://github.com/openstack/neutron/blob/598fcb437a0ad3d564435799c70f38429ab4f0eb/neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py#L133-L140
As a result, OVN routers now have the `distributed` property set to `False` by default (unless the global ML2/ovs-specific default is changed) and it does not reflect the state of the `ovn/enable_distributed_floating_ip` option. It can also be changed via the API on the router without any apparent effect.
The ML2/ovs and ML2/ovn comparison docs still refer to OVN-based router having no `l3ha` or `distributed` attributes whereas this is not the case anymore:
https://github.com/openstack/neutron/blame/cd66232c2b26cb4141c2e9426ce2dec0f38c364c/doc/source/ovn/faq/index.rst#L16-L29
One place where it becomes relevant is the neutron-dynamic-routing project which relies on the `distributed` property to determine whether to add /32 routes with next-hops set to a router gateway port IP (centralized FIPs case) or not (distributed FIPs case).
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L564
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L567-L580
For distributed routers the logic is such that IP addresses of ports with a device owner set to `floatingip_agent_gateway` are used as a next hop for /32 routes, however, the OVN-based L3 service plugin implementation (OVNL3RouterPlugin) does not create those on a per external network basis much like the core L3RouterPlugin-based implementation does with DVR.
As a result, if an operator uses distributed FIPs with OVN with the router attribute `distributed == False`, neutron-dynamic-routing will advertise /32 routes with the centralized FIP logic (the southbound traffic would go via the router gateway port).
On the other hand, if an operator uses distributed FIPs with OVN with the router attribute `distributed == True`, neutron-dynamic-routing will not advertise anything because the centralized routes will not be added as the router seems to be distributed whereas there are no `floatingip_agent_gateway` ports created with OVN.
There are at least two outputs to expect as a fix:
1) Make sure the distributed state is reflected correctly for OVN routers based on the OVN-specific config option;
2) Fix neutron-dynamic routing to still create centralized /32 routes if there are not any `floatingip_agent_gateway` ports
OR change the OVN implementation to create those for the purposes of direct southbound routing purposes.
A similar approach can be done for the `ha` attribute but based on OVN's approach to it that does not rely on VRRP. |
|
| 2023-06-01 14:44:11 |
Dmitrii Shcherbakov |
summary |
[ovn] l3ha and disitributed router extra attributes do not reflect OVN state |
[ovn] l3ha and distributed router extra attributes do not reflect OVN state |
|
| 2023-06-01 16:41:19 |
Brian Haley |
bug |
|
|
added subscriber Brian Haley |
| 2023-06-01 16:43:59 |
Brian Haley |
tags |
bgp ndr neutron-dynamic-routing ovn |
bgp l3-bgp neutron-dynamic-routing ovn |
|
| 2023-06-06 17:21:30 |
Dmitrii Shcherbakov |
description |
With https://bugs.launchpad.net/neutron/+bug/1995974 fixed and https://review.opendev.org/c/openstack/neutron/+/864051 merged extra attributes such as `distributed` and `ha` are now created for OVN routers as well.
Their default values are taken from the global configuration options more relevant for default L3 service plugin implementation based on Linux network namespaces
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/db/l3_attrs_db.py#L24-L27
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_hamode_db.py#L21
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_dvr_db.py#L19-L27
as opposed to relying on the OVN-specific options. For instance, it order to enable the support for distributed floating IPs there is an OVN-specific global option that enables this mode for all OVN routers:
https://github.com/openstack/neutron/blob/598fcb437a0ad3d564435799c70f38429ab4f0eb/neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py#L133-L140
As a result, OVN routers now have the `distributed` property set to `False` by default (unless the global ML2/ovs-specific default is changed) and it does not reflect the state of the `ovn/enable_distributed_floating_ip` option. It can also be changed via the API on the router without any apparent effect.
The ML2/ovs and ML2/ovn comparison docs still refer to OVN-based router having no `l3ha` or `distributed` attributes whereas this is not the case anymore:
https://github.com/openstack/neutron/blame/cd66232c2b26cb4141c2e9426ce2dec0f38c364c/doc/source/ovn/faq/index.rst#L16-L29
One place where it becomes relevant is the neutron-dynamic-routing project which relies on the `distributed` property to determine whether to add /32 routes with next-hops set to a router gateway port IP (centralized FIPs case) or not (distributed FIPs case).
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L564
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L567-L580
For distributed routers the logic is such that IP addresses of ports with a device owner set to `floatingip_agent_gateway` are used as a next hop for /32 routes, however, the OVN-based L3 service plugin implementation (OVNL3RouterPlugin) does not create those on a per external network basis much like the core L3RouterPlugin-based implementation does with DVR.
As a result, if an operator uses distributed FIPs with OVN with the router attribute `distributed == False`, neutron-dynamic-routing will advertise /32 routes with the centralized FIP logic (the southbound traffic would go via the router gateway port).
On the other hand, if an operator uses distributed FIPs with OVN with the router attribute `distributed == True`, neutron-dynamic-routing will not advertise anything because the centralized routes will not be added as the router seems to be distributed whereas there are no `floatingip_agent_gateway` ports created with OVN.
There are at least two outputs to expect as a fix:
1) Make sure the distributed state is reflected correctly for OVN routers based on the OVN-specific config option;
2) Fix neutron-dynamic routing to still create centralized /32 routes if there are not any `floatingip_agent_gateway` ports
OR change the OVN implementation to create those for the purposes of direct southbound routing purposes.
A similar approach can be done for the `ha` attribute but based on OVN's approach to it that does not rely on VRRP. |
With https://bugs.launchpad.net/neutron/+bug/1995974 fixed and https://review.opendev.org/c/openstack/neutron/+/864051 merged extra attributes such as `distributed` and `ha` are now created for OVN routers as well.
Their default values are taken from the global configuration options more relevant for default L3 service plugin implementation based on Linux network namespaces
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/db/l3_attrs_db.py#L24-L27
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_hamode_db.py#L21
https://github.com/openstack/neutron/blob/0de6a4d620f1cb780c6a3635e10406b0db97762a/neutron/conf/db/l3_dvr_db.py#L19-L27
as opposed to relying on the OVN-specific options. For instance, it order to enable the support for distributed floating IPs there is an OVN-specific global option that enables this mode for all OVN routers:
https://github.com/openstack/neutron/blob/598fcb437a0ad3d564435799c70f38429ab4f0eb/neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py#L133-L140
As a result, OVN routers now have the `distributed` property set to `False` by default (unless the global ML2/ovs-specific default is changed) and it does not reflect the state of the `ovn/enable_distributed_floating_ip` option. It can also be changed via the API on the router without any apparent effect.
The ML2/ovs and ML2/ovn comparison docs still refer to OVN-based router having no `l3ha` or `distributed` attributes whereas this is not the case anymore:
https://github.com/openstack/neutron/blame/cd66232c2b26cb4141c2e9426ce2dec0f38c364c/doc/source/ovn/faq/index.rst#L16-L29
One place where it becomes relevant is the neutron-dynamic-routing project which relies on the `distributed` property to determine whether to add /32 routes with next-hops set to a router gateway port IP (centralized FIPs case) or not (distributed FIPs case).
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L564
https://github.com/openstack/neutron-dynamic-routing/blob/513ea649be9fd652b0c5b391167f851bc3d653bb/neutron_dynamic_routing/db/bgp_db.py#L567-L580
For distributed routers the logic is such that IP addresses of ports with a device owner set to `network:floatingip_agent_gateway` are used as a next hop for /32 routes, however, the OVN-based L3 service plugin implementation (OVNL3RouterPlugin) does not create those on a per external network basis much like the core L3RouterPlugin-based implementation does with DVR.
As a result, if an operator uses distributed FIPs with OVN with the router attribute `distributed == False`, neutron-dynamic-routing will advertise /32 routes with the centralized FIP logic (the southbound traffic would go via the router gateway port).
On the other hand, if an operator uses distributed FIPs with OVN with the router attribute `distributed == True`, neutron-dynamic-routing will not advertise anything because the centralized routes will not be added as the router seems to be distributed whereas there are no `network:floatingip_agent_gateway` ports created with OVN.
There are at least two outputs to expect as a fix:
1) Make sure the distributed state is reflected correctly for OVN routers based on the OVN-specific config option;
2) Fix neutron-dynamic routing to still create centralized /32 routes if there are not any `network:floatingip_agent_gateway` ports
OR change the OVN implementation to create those for the purposes of direct southbound routing purposes.
A similar approach can be done for the `ha` attribute but based on OVN's approach to it that does not rely on VRRP.
=====
A note on `network:floatingip_agent_gateway` ports with ML2/ovs:
They were originally added to implement the "fast exit" RFE: https://bugs.launchpad.net/neutron/+bug/1577488
https://review.opendev.org/c/openstack/neutron/+/283757
One *other* reason for having floatingip_agent_gateway ports is that Neutron allows for multiple subnets per network with different service types. So floating IPs can be allocated from a subnet (with "network:floatingip" service type https://github.com/openstack/neutron-lib/blob/fd0468f79343f47410a7932115acf7c52ceeb1c3/neutron_lib/constants.py#L55) different from the subnet used for other purposes like allocation of gateway IPs ("network:router_gateway" service type)
https://docs.openstack.org/neutron/latest/admin/config-service-subnets.html
In that case, the network fabric will need a route with a next hop at the OpenStack side through which a FIP will be reachable. And this is despite the fact an external network is a single-segment one (shared L2).
NDR currently tries to look up a next hop based on ports with `network:floatingip_agent_gateway` device owner for a floating IP associated with a `distributed == True` router. |
|
| 2023-06-26 16:57:09 |
Rodolfo Alonso |
neutron: assignee |
|
Rodolfo Alonso (rodolfo-alonso-hernandez) |
|
| 2023-06-26 16:57:11 |
Rodolfo Alonso |
neutron: importance |
Undecided |
Medium |
|
| 2023-06-26 16:57:40 |
Rodolfo Alonso |
neutron: status |
New |
In Progress |
|
| 2023-10-30 15:05:12 |
OpenStack Infra |
neutron: status |
In Progress |
Fix Released |
|
