NAT reflection with OVN on xena not working

[Eugen Mayer]

---- Problem
When a VM-A located in network 'INTRANET' tries to reach a VM-B also located in network 'INTRANET', but using a FIP with port forwarding, the connection does not work.

So if on of my FIPs has the IP 1.1.1.1 and that fip forwards port 25 to VM-B i do this on VM-A

telnet 1.1.1.1 25

The connection stalls.

---- Setup

I have the following setup
 - Xena
 - OVN
 - DVR disabled
 - Multiple FIPs
 - multiple computes, one controller
 - Since non DVR, all FIPs are routed through the controller

The topology looks like this
 - 'INTRANET' network
 - 'K8S' network

---- Additional observation

- It does not matter if VM-A and VM-B are located on the same or different computes
- A VM-C located in network K8S can reach VM-B
- A VM-D located in network K8S can reach VM-C located in K8S using a non-port forward FIP
- Connection to 1.1.1.1 from the outer world are working without any issues

[Eugen Mayer (eugenmayer)]

Thank you for working on this is yatin!

[yatin (yatinkarel)]

So i investigated this issue and below are the results:-

- Issue is not seen in an OVS deployment
- Issue is reproducable with a devstack OVN master setup
- Got a workaround(Add lb to logical_switch) to clear this issue but that doesn't look ideal as would need duplicating it on both switch and router(switch is already connected to router) and may need more work on neutron side to get it applied so created a OVN BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2043543 for further investigation.

Let's see if we can get some better solutions from OVN side itself(may be it's already supported in some newver ovn version, /me not checked that yet) and can avoid these workarounds.

[Eugen Mayer (eugenmayer)]

Thank you for working and confirming thiss @yatin!

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/833620

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/833620
Committed: https://opendev.org/openstack/neutron/commit/2492cf2e0703dbe6fcfe26e199505b7b43a65f50
Submitter: "Zuul (22348)"
Branch: master

commit 2492cf2e0703dbe6fcfe26e199505b7b43a65f50
Author: yatinkarel <email address hidden>
Date: Mon Mar 14 17:49:51 2022 +0530

    [OVN][port-forwarding] Add lb also to logical_switches

    Currently we are adding lbs to logical_routers, but
    to get the lbs accessible from the vms we need to
    add lbs also to all the logical_switches connected
    to the logical router as suggested in the Related
    Bug rhbz#2043543.
    If in future ovn automatically handles addition of
    lbs to logical_switches if they exist on associated
    logical_router, then we can remove this handling
    from port-forwarding service plugin.

    Also subscribed to router_interface's after create
    and delete events to handle the case when logical
    switches are added/removed from router after intial
    port forwarding create.

    Related-Bug: #1957185
    Related-Bug: rhbz#2043543
    Change-Id: I0c4d492887216cad7a8155dceb738389f2886376

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/839201

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/839182

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/839183

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/839201
Committed: https://opendev.org/openstack/neutron/commit/82671b55024ddff8b9593ca9a76898c67468fcea
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 82671b55024ddff8b9593ca9a76898c67468fcea
Author: yatinkarel <email address hidden>
Date: Mon Mar 14 17:49:51 2022 +0530

    [OVN][port-forwarding] Add lb also to logical_switches

    Currently we are adding lbs to logical_routers, but
    to get the lbs accessible from the vms we need to
    add lbs also to all the logical_switches connected
    to the logical router as suggested in the Related
    Bug rhbz#2043543.
    If in future ovn automatically handles addition of
    lbs to logical_switches if they exist on associated
    logical_router, then we can remove this handling
    from port-forwarding service plugin.

    Also subscribed to router_interface's after create
    and delete events to handle the case when logical
    switches are added/removed from router after intial
    port forwarding create.

    Related-Bug: #1957185
    Related-Bug: rhbz#2043543
    Change-Id: I0c4d492887216cad7a8155dceb738389f2886376
    (cherry picked from commit 2492cf2e0703dbe6fcfe26e199505b7b43a65f50)
    Conflicts: neutron/services/portforwarding/drivers/ovn/driver.py

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/839182
Committed: https://opendev.org/openstack/neutron/commit/02fd530f9cb6128a65992a974ddf4f1c2667039e
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 02fd530f9cb6128a65992a974ddf4f1c2667039e
Author: yatinkarel <email address hidden>
Date: Mon Mar 14 17:49:51 2022 +0530

    [OVN][port-forwarding] Add lb also to logical_switches

    Currently we are adding lbs to logical_routers, but
    to get the lbs accessible from the vms we need to
    add lbs also to all the logical_switches connected
    to the logical router as suggested in the Related
    Bug rhbz#2043543.
    If in future ovn automatically handles addition of
    lbs to logical_switches if they exist on associated
    logical_router, then we can remove this handling
    from port-forwarding service plugin.

    Also subscribed to router_interface's after create
    and delete events to handle the case when logical
    switches are added/removed from router after intial
    port forwarding create.

    Related-Bug: #1957185
    Related-Bug: rhbz#2043543
    Change-Id: I0c4d492887216cad7a8155dceb738389f2886376
    (cherry picked from commit 2492cf2e0703dbe6fcfe26e199505b7b43a65f50)
    Conflicts: neutron/services/portforwarding/drivers/ovn/driver.py

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/839183
Committed: https://opendev.org/openstack/neutron/commit/87200a271851b55d3992c8ae98e702f5323a61dd
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 87200a271851b55d3992c8ae98e702f5323a61dd
Author: yatinkarel <email address hidden>
Date: Mon Mar 14 17:49:51 2022 +0530

    [OVN][port-forwarding] Add lb also to logical_switches

    Currently we are adding lbs to logical_routers, but
    to get the lbs accessible from the vms we need to
    add lbs also to all the logical_switches connected
    to the logical router as suggested in the Related
    Bug rhbz#2043543.
    If in future ovn automatically handles addition of
    lbs to logical_switches if they exist on associated
    logical_router, then we can remove this handling
    from port-forwarding service plugin.

    Also subscribed to router_interface's after create
    and delete events to handle the case when logical
    switches are added/removed from router after intial
    port forwarding create.

    Related-Bug: #1957185
    Related-Bug: rhbz#2043543
    Change-Id: I0c4d492887216cad7a8155dceb738389f2886376
    (cherry picked from commit 2492cf2e0703dbe6fcfe26e199505b7b43a65f50)
    Conflicts: neutron/services/portforwarding/drivers/ovn/driver.py

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/841602

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/841602
Committed: https://opendev.org/openstack/neutron/commit/e59cf95dc017a45bda9da01e5feb401fd3b3fbde
Submitter: "Zuul (22348)"
Branch: stable/xena

commit e59cf95dc017a45bda9da01e5feb401fd3b3fbde
Author: yatinkarel <email address hidden>
Date: Thu May 12 19:43:27 2022 +0530

    [Xena Only] Use _ovn instead of _nb_ovn for l3_plugin

    This was introduced in [1], Switch to _nb_ovn is only
    available since Yoga[2]. In wallaby it's already fixed
    as part of other fix[3].

    [1] https://review.opendev.org/q/I0c4d492887216cad7a8155dceb738389f2886376
    [2] https://github.com/openstack/neutron/commit/92b0811
    [3] https://review.opendev.org/c/openstack/neutron/+/841204

    Related-Bug: #1957185
    Change-Id: I91ef163083678dab5229946c4cf1f09b9885ff6d

[Brian Haley (brian-haley)]

Will close this as patches seemed to have all merged, please re-open if necessary.