Wrong addition of VIPs to all logical router pods leading to triggering GARP on different locations

Bug #1964901 reported by Luis Tomas Bolivar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Undecided
Unassigned

Bug Description

When a loadbalancer is created in an OSP tenant network (VIP and members), and that tenant networks is connected to a router, which in turns is connected to the provider network, the ovn loadbalancer gets associated to the ovn logical router. This includes also the cr-lrp port (patch port connecting the router and the provider network, in OSP world, the router gateway port), and it can be seen by the entry on the nat_address of that port, which includes the VIP of the loadbalalancer.

This may cause problems as that means ovn-controller will send GARPs for that (internal, tenant network) IP. There is nothing blocking different tenants in OSP to create a subnet with the same CIDR and then a loadbalancer with the same VIP. If that is the case, there may be several ovn-controllers generating GARPs on the provider network for the same IP, each one with the MAC of the logical router port belonging to each user. This could be a problem for the physical network infrastructure.

Steps to Reproduce:
1. Create a router in OSP and attach it to the provider network
2. Create a tenant network/subnet and connect it to the router
3. Create a Load Balancer in OSP, with the VIP in that tenant network

Actual results:
Check the VIP of the loadbalancer is on the OVN SB Port_Binding table, at the nat_addresses of the patch port connecting the router to the provider network:

datapath : e3a0a334-9a02-41c7-a64d-6ea747839808
external_ids : {"neutron:cidrs"="172.24.100.181/24 2001:db8::f816:3eff:fe77:7f9c/64", "neutron:device_id"="335cd008-216f-4571-a685-b0de5a7ffe50", "neutron:device_owner"="network:router_gateway", "neutron:network_name"=neutron-d923b3db-500d-4241-95be-c3869c72b36a, "neutron:port_name"="", "neutron:project_id"="", "neutron:revision_number"="6", "neutron:security_group_ids"=""}
logical_port : "add962d2-21ab-4733-b6ef-35538eff25a8"
mac : [router]
nat_addresses : ["fa:16:3e:77:7f:9c 172.24.100.181 is_chassis_resident(\"cr-lrp-add962d2-21ab-4733-b6ef-35538eff25a8\")", "fa:16:3e:77:7f:9c 172.24.100.229 *20.0.0.98* 172.24.100.112 is_chassis_resident(\"cr-lrp-add962d2-21ab-4733-b6ef-35538eff25a8\")"]
options : {peer=lrp-add962d2-21ab-4733-b6ef-35538eff25a8}
parent_port : []
tag : []
tunnel_key : 4
type : patch
up : false
virtual_parent : []

Expected results:
In the example, ip 20.0.0.98 should not be there as that belongs to an IP in a tenant network that should not be advertized (GARP) in the provider network.

Revision history for this message
Luis Tomas Bolivar (ltomasbo) wrote :
Changed in neutron:
assignee: nobody → Luis Tomas Bolivar (ltomasbo)
assignee: Luis Tomas Bolivar (ltomasbo) → nobody
status: New → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/833730

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/833731

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/833732

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/833733

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/833734

Revision history for this message
Akihiro Motoki (amotoki) wrote :

The patch in the master branch has been merged. We usually mark it as "Fix Released" when a fix in the master branch is merged.

tags: added: ovn
Changed in neutron:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/833731
Committed: https://opendev.org/openstack/neutron/commit/391af6c158b9e99f2a47cfb2dd8b5b04edc859f7
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 391af6c158b9e99f2a47cfb2dd8b5b04edc859f7
Author: Luis Tomas Bolivar <email address hidden>
Date: Tue Mar 8 13:04:58 2022 +0100

    Ensure no GARPs are sent for Load Balancer VIPs on tenant networks

    When a loadbalancer is created in tenant network (VIP and members),
    and that tenant network is connected to a router, which is connected
    to the provider network, the ovn loadbalancer gets
    associated to the ovn logical router. If the "router" option is used for
    "nat-addresses" (as currently done), ovn-controller will send GARPs for
    that VIP.

    As there is nothing blocking different tenants in openstack to create a
    subnet with the same CIDR and then a loadbalancer with the same VIP,
    there may be several ovn-controllers generating GARPs on the provider
    network for the same IP, each one with the MAC of the logical router
    port belonging to each tenant.
    This could be a problem for the physical network infrastructure.

    This patch fixes it by setting a new option added in OVN[1], named
    "exclude-lb-vips-from-garp", on the router gateway port, ensuring
    no GARPs are sent for the load balancer VIPs.

    [1] https://patchwork.ozlabs.org/project/ovn/patch/183edfc446633c5c38d7d7361089d34432c527dd<email address hidden>/

    Closes-Bug: #1964901

    Change-Id: Ifc2e5b2cc64c0b3beafd0a2e6eb9b9f248970cc5
    (cherry picked from commit 9d4e6edd5b4c92c88fbe8d06f6a176fae1b35f25)

tags: added: in-stable-xena
tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/833730
Committed: https://opendev.org/openstack/neutron/commit/e61b117b3319188089e0e38727d6eb49c2ad5fec
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit e61b117b3319188089e0e38727d6eb49c2ad5fec
Author: Luis Tomas Bolivar <email address hidden>
Date: Tue Mar 8 13:04:58 2022 +0100

    Ensure no GARPs are sent for Load Balancer VIPs on tenant networks

    When a loadbalancer is created in tenant network (VIP and members),
    and that tenant network is connected to a router, which is connected
    to the provider network, the ovn loadbalancer gets
    associated to the ovn logical router. If the "router" option is used for
    "nat-addresses" (as currently done), ovn-controller will send GARPs for
    that VIP.

    As there is nothing blocking different tenants in openstack to create a
    subnet with the same CIDR and then a loadbalancer with the same VIP,
    there may be several ovn-controllers generating GARPs on the provider
    network for the same IP, each one with the MAC of the logical router
    port belonging to each tenant.
    This could be a problem for the physical network infrastructure.

    This patch fixes it by setting a new option added in OVN[1], named
    "exclude-lb-vips-from-garp", on the router gateway port, ensuring
    no GARPs are sent for the load balancer VIPs.

    [1] https://patchwork.ozlabs.org/project/ovn/patch/183edfc446633c5c38d7d7361089d34432c527dd<email address hidden>/

    Closes-Bug: #1964901

    Change-Id: Ifc2e5b2cc64c0b3beafd0a2e6eb9b9f248970cc5
    (cherry picked from commit 9d4e6edd5b4c92c88fbe8d06f6a176fae1b35f25)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/833734
Committed: https://opendev.org/openstack/neutron/commit/2896ffc0bd2132268d5f2815674d5f906b431589
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 2896ffc0bd2132268d5f2815674d5f906b431589
Author: Luis Tomas Bolivar <email address hidden>
Date: Tue Mar 8 13:04:58 2022 +0100

    Ensure no GARPs are sent for Load Balancer VIPs on tenant networks

    When a loadbalancer is created in tenant network (VIP and members),
    and that tenant network is connected to a router, which is connected
    to the provider network, the ovn loadbalancer gets
    associated to the ovn logical router. If the "router" option is used for
    "nat-addresses" (as currently done), ovn-controller will send GARPs for
    that VIP.

    As there is nothing blocking different tenants in openstack to create a
    subnet with the same CIDR and then a loadbalancer with the same VIP,
    there may be several ovn-controllers generating GARPs on the provider
    network for the same IP, each one with the MAC of the logical router
    port belonging to each tenant.
    This could be a problem for the physical network infrastructure.

    This patch fixes it by setting a new option added in OVN[1], named
    "exclude-lb-vips-from-garp", on the router gateway port, ensuring
    no GARPs are sent for the load balancer VIPs.

    [1] https://patchwork.ozlabs.org/project/ovn/patch/183edfc446633c5c38d7d7361089d34432c527dd<email address hidden>/

    Closes-Bug: #1964901

    Change-Id: Ifc2e5b2cc64c0b3beafd0a2e6eb9b9f248970cc5
    (cherry picked from commit 9d4e6edd5b4c92c88fbe8d06f6a176fae1b35f25)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/833732
Committed: https://opendev.org/openstack/neutron/commit/ca458d9eb9eb7a5550a40a28a88f0f3d30bb52c4
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit ca458d9eb9eb7a5550a40a28a88f0f3d30bb52c4
Author: Luis Tomas Bolivar <email address hidden>
Date: Tue Mar 8 13:04:58 2022 +0100

    Ensure no GARPs are sent for Load Balancer VIPs on tenant networks

    When a loadbalancer is created in tenant network (VIP and members),
    and that tenant network is connected to a router, which is connected
    to the provider network, the ovn loadbalancer gets
    associated to the ovn logical router. If the "router" option is used for
    "nat-addresses" (as currently done), ovn-controller will send GARPs for
    that VIP.

    As there is nothing blocking different tenants in openstack to create a
    subnet with the same CIDR and then a loadbalancer with the same VIP,
    there may be several ovn-controllers generating GARPs on the provider
    network for the same IP, each one with the MAC of the logical router
    port belonging to each tenant.
    This could be a problem for the physical network infrastructure.

    This patch fixes it by setting a new option added in OVN[1], named
    "exclude-lb-vips-from-garp", on the router gateway port, ensuring
    no GARPs are sent for the load balancer VIPs.

    [1] https://patchwork.ozlabs.org/project/ovn/patch/183edfc446633c5c38d7d7361089d34432c527dd<email address hidden>/

    Closes-Bug: #1964901

    Change-Id: Ifc2e5b2cc64c0b3beafd0a2e6eb9b9f248970cc5
    (cherry picked from commit 9d4e6edd5b4c92c88fbe8d06f6a176fae1b35f25)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/833733
Committed: https://opendev.org/openstack/neutron/commit/4627955f973e295dc9f6fb7ff7712f4444216db9
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 4627955f973e295dc9f6fb7ff7712f4444216db9
Author: Luis Tomas Bolivar <email address hidden>
Date: Tue Mar 8 13:04:58 2022 +0100

    Ensure no GARPs are sent for Load Balancer VIPs on tenant networks

    When a loadbalancer is created in tenant network (VIP and members),
    and that tenant network is connected to a router, which is connected
    to the provider network, the ovn loadbalancer gets
    associated to the ovn logical router. If the "router" option is used for
    "nat-addresses" (as currently done), ovn-controller will send GARPs for
    that VIP.

    As there is nothing blocking different tenants in openstack to create a
    subnet with the same CIDR and then a loadbalancer with the same VIP,
    there may be several ovn-controllers generating GARPs on the provider
    network for the same IP, each one with the MAC of the logical router
    port belonging to each tenant.
    This could be a problem for the physical network infrastructure.

    This patch fixes it by setting a new option added in OVN[1], named
    "exclude-lb-vips-from-garp", on the router gateway port, ensuring
    no GARPs are sent for the load balancer VIPs.

    [1] https://patchwork.ozlabs.org/project/ovn/patch/183edfc446633c5c38d7d7361089d34432c527dd<email address hidden>/

    Closes-Bug: #1964901

    Change-Id: Ifc2e5b2cc64c0b3beafd0a2e6eb9b9f248970cc5
    (cherry picked from commit 9d4e6edd5b4c92c88fbe8d06f6a176fae1b35f25)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.0.0.0rc2

This issue was fixed in the openstack/neutron 20.0.0.0rc2 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 17.4.0

This issue was fixed in the openstack/neutron 17.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.3.0

This issue was fixed in the openstack/neutron 18.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.2.0

This issue was fixed in the openstack/neutron 19.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn train-eol

This issue was fixed in the openstack/networking-ovn train-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron ussuri-eol

This issue was fixed in the openstack/neutron ussuri-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.