a public network with multi subnets , but based on ovn, They can't communicate with each other

[李亚冲]

a public network with multi subnets , but based on ovn, They can't communicate with each other

Hi, I think it's not a bug, or it's a logical bug, when I change my mechanism_driver from openvswitch to ovn, I've been troubled by this matter for a long time.

ok, my scenario: I applied a subnets from ISP provider, let's call it subnet1 of public_network , After a period of time, the subnet IP is not enough, So I applied for another subnets, let's call it subnet2 of public_network. until now, the public_network will have two subnet.

But FIPs based on two subnets cannot communicate with each other, The FIP of the subnet2 cannot communicate with itself either.

I found some bug reports : <https://bugzilla.redhat.com/show_bug.cgi?id=1920025><https://access.redhat.com/errata/RHBA-2022:9044> <https://bugzilla.redhat.com/show_bug.cgi?id=1929901> , but those think it's a ovn bug.

And in my test, I' found it's a neutron logical bug, yes or no, it's a bug, ha

In neutron ovn_client.py, the function _update_lrouter_port , it's will be called by create or update or delete. sub function get_nets_and_ipv6_ra_confs_for_router_port will get all subnet of network. but but, if a subnet not a fixed ip in the router, it's can't be get.

```
    def _update_lrouter_port(self, context, port, if_exists=False, txn=None):
        """Update a logical router port."""
        networks, ipv6_ra_configs = (
            self._get_nets_and_ipv6_ra_confs_for_router_port(context, port))

        lsp_address = ovn_const.DEFAULT_ADDR_FOR_LSP_WITH_PEER
        lrp_name = utils.ovn_lrouter_port_name(port['id'])
        update = {'networks': networks, 'ipv6_ra_configs': ipv6_ra_configs}
        is_gw_port = const.DEVICE_OWNER_ROUTER_GW == port.get(
            'device_owner')
        commands = [
            self._nb_idl.update_lrouter_port(
                name=lrp_name,
                external_ids=self._gen_router_port_ext_ids(port),
                options=self._gen_router_port_options(port),
                if_exists=if_exists,
                **update),
            self._nb_idl.set_lrouter_port_in_lswitch_port(
                port['id'], lrp_name, is_gw_port=is_gw_port,
                lsp_address=lsp_address)]

        self._transaction(commands, txn=txn)

    def _get_nets_and_ipv6_ra_confs_for_router_port(self, context, port):
        port_fixed_ips = port['fixed_ips']
        networks = set()
        ipv6_ra_configs = {}
        ipv6_ra_configs_supported = self._nb_idl.is_col_present(
            'Logical_Router_Port', 'ipv6_ra_configs')
        is_gw_port = const.DEVICE_OWNER_ROUTER_GW == port.get(
            'device_owner')

        # here, it's port_fixed_ips, so subnets2 no fixed ip belong to router external_port
        for fixed_ip in port_fixed_ips:
            subnet_id = fixed_ip['subnet_id']
            subnet = self._plugin.get_subnet(context, subnet_id)
            cidr = netaddr.IPNetwork(subnet['cidr'])
            networks.add("%s/%s" % (fixed_ip['ip_address'],
                                    str(cidr.prefixlen)))

```

ok, fine, that's ok, it can be resolved, set router port multiple fixed ip. But there is no documentation or tutorial telling openstack administrators to do this. no one.

```
# openstack port show b64eb3b2-d60b-4054-a4ea-adf926efe824 -f json
{
  ...
  "fixed_ips": [
    {
      "subnet_id": "d7241630-a850-45c8-9b72-7ff20d3a00e3",
      "ip_address": "192.168.0.100"
    },
    {
      "subnet_id": "e8cd9b0b-6d66-462a-aaaa-dd5867368f85",
      "ip_address": "192.168.1.100"
    }
  ],
  ...
}

```

When mechanism_driver is openvswitch, Routing is implemented by iptables, which does not require multiple fixed IPs to be set on the routing port. but ovn not, it need.

So my opinion is
plan1: traverse all subnet in _get_nets_and_ipv6_ra_confs_for_router_port

or plan2: tell us need add multiple fixed ip to router external_port when have multiple subnets

[李亚冲 (879228763-6)]

This is a very complex logic, including knowledge of ovn, openvswitch, neutorn, and even router. I can't explain it clearly in this limited content, but I think someone should know about it.

[Lajos Katona (lajos-katona)]

Hi, thanks for reporting and checking this issue.
Do you think that you can summarize this issue in the OVN gaps document:
https://opendev.org/openstack/neutron/src/branch/master/doc/source/ovn/gaps.rst

I think this finding has a good place there. In the meantime I check if this logic can be changed.

[Lajos Katona (lajos-katona)]

I tested this today in OVS and OVN based deployment, and my VMs were able to ping each others FIP in both environment without any extra fixed IP on any router port.

[李亚冲 (879228763-6)]

@lajos-katona

need multiple subnet on one public_network, it's like this :

https://photos.app.goo.gl/rzP6xpbqrVGGw5Vf8

because openvswitch driver use iptables impl l3_router, in my test, it's ok. so, in my bug report , I've been troubled by this matter for a long time.

[Lajos Katona (lajos-katona)]

Hi, I added 2 subnets to my public network, so I believe that my environment is the same as yours

[李亚冲 (879228763-6)]

ok fine, I wonder if there is any configuration that can prevent this from happening

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

I think what you are describing is https://bugs.launchpad.net/neutron/+bug/2035281. Please check if you have this code in your deployment.