[DVR] Neutron doesn't configure multiple external subnets for one network properly

Bug #1805456 reported by Slawek Kaplonski
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Rodolfo Alonso

Bug Description

Bug originally reported for RHOSP: https://bugzilla.redhat.com/show_bug.cgi?id=1651647 by Alex Stupnikov.
Original description:

It is possible to create two subnets for single external network and it looks like we support such kind of setup. However, it doesn't work properly: every DVR will have a single on-link route for ONLY ONE external subnet and will not have on-link route for another one.

As a result, there are two scenarios when routing doesn't work as it should:

- two instances with FIPs from different external subnets will have asymmetric traffic flows: if there is an on-link route for DST packet, it will be routed directly to another DVR. If not, packet will be routed to external router. As a result, if there is a stateful firewall on external router, this communication will be blocked.
- two instances with FIPs from the same external subnet could have suboptimal traffic flows if there are no on-link routes for this subnet. As a result, traffic will always go through external router, which could cause issues if router has protection against such kind of flows.

Here is an example output of routing table in fip-* namespace on compute node where single external network has two subnets: 10.0.0.0/24 and 10.0.1.0/24:

default via 10.0.0.1 dev fg-83ec5f16-be table 2852022899
10.0.0.0/24 dev fg-83ec5f16-be proto kernel scope link src 10.0.0.225
10.0.0.218 via 169.254.106.114 dev fpr-789f245b-1
10.0.0.219 via 169.254.106.114 dev fpr-789f245b-1
10.0.1.15 via 169.254.106.114 dev fpr-789f245b-1

As we can see, there are host routes for local FIPs, a single on-link route for 10.0.0.0/24 subnet and default route.

Additional information:

An upstream bug for legacy routers [1] was solved long time ago.

[1] https://bugs.launchpad.net/neutron/+bug/1312467

Changed in neutron:
assignee: Slawek Kaplonski (slaweq) → Rodolfo Alonso (rodolfo-alonso-hernandez)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/622449

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/622449
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=97c98a1c6dac5788690504463a5ca33f4b181d6a
Submitter: Zuul
Branch: master

commit 97c98a1c6dac5788690504463a5ca33f4b181d6a
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Dec 3 18:11:26 2018 +0000

    [DVR] Allow multiple subnets per external network

    An external network can have more than one subnet. Currently only the
    first subnet is added to the FIP namespace routing table. Packets for
    FIPs with addresses in other subnets can't pass through the external
    port because there is no route for those FIP CIDRs.

    This change adds routes for those CIDRs via the external port IP and
    interface.

    These routes doesn't collide with the existing ones, added to provide
    a back path for the packets with a destination IP matching a FIP.

    E.g.:
    $ ip netns exec fip-e1ec0f98-b593-4514-ae08-f1c5cf1c2788 ip route
      (1) 169.254.106.114/31 dev fpr-3937f879-d proto kernel scope link \
          src 169.254.106.115
      (2) 192.168.20.250 via 169.254.106.114 dev fpr-3937f879-d
      (3) 192.168.30.0/24 dev fg-bee060f1-dd proto kernel scope link \
          src 192.168.30.129
      (4) 192.168.20.0/24 via 192.168.30.129 dev fg-bee060f1-dd scope link

    Rule (2) is added when a FIP is assigned. This rule permits ingress
    packets going into the router namespace. This FIP belongs to the second
    subnet of the external network (note the external port CIDR is not the
    same). Rule (4), added by this patch, allows egress packets to exit
    the FIP namespace through the external port. Rule (2), because of the
    prefix length (32), has more priority than rule (4).

    Change-Id: I4d476b47e89fa5709dca2f66ffae72a27d88340a
    Closes-Bug: #1805456

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 14.0.0.0b1

This issue was fixed in the openstack/neutron 14.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/631719

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/631724

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/queens)

Reviewed: https://review.openstack.org/631724
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8d99593adb780ed16bd1aa8412b4b4ccf5d20f31
Submitter: Zuul
Branch: stable/queens

commit 8d99593adb780ed16bd1aa8412b4b4ccf5d20f31
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Dec 3 18:11:26 2018 +0000

    [DVR] Allow multiple subnets per external network

    An external network can have more than one subnet. Currently only the
    first subnet is added to the FIP namespace routing table. Packets for
    FIPs with addresses in other subnets can't pass through the external
    port because there is no route for those FIP CIDRs.

    This change adds routes for those CIDRs via the external port IP and
    interface.

    These routes doesn't collide with the existing ones, added to provide
    a back path for the packets with a destination IP matching a FIP.

    E.g.:
    $ ip netns exec fip-e1ec0f98-b593-4514-ae08-f1c5cf1c2788 ip route
      (1) 169.254.106.114/31 dev fpr-3937f879-d proto kernel scope link \
          src 169.254.106.115
      (2) 192.168.20.250 via 169.254.106.114 dev fpr-3937f879-d
      (3) 192.168.30.0/24 dev fg-bee060f1-dd proto kernel scope link \
          src 192.168.30.129
      (4) 192.168.20.0/24 via 192.168.30.129 dev fg-bee060f1-dd scope link

    Rule (2) is added when a FIP is assigned. This rule permits ingress
    packets going into the router namespace. This FIP belongs to the second
    subnet of the external network (note the external port CIDR is not the
    same). Rule (4), added by this patch, allows egress packets to exit
    the FIP namespace through the external port. Rule (2), because of the
    prefix length (32), has more priority than rule (4).

    Change-Id: I4d476b47e89fa5709dca2f66ffae72a27d88340a
    Closes-Bug: #1805456

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/rocky)

Reviewed: https://review.openstack.org/631719
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=886782c1777d8f104b5c57ffd4eff7680056602c
Submitter: Zuul
Branch: stable/rocky

commit 886782c1777d8f104b5c57ffd4eff7680056602c
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Mon Dec 3 18:11:26 2018 +0000

    [DVR] Allow multiple subnets per external network

    An external network can have more than one subnet. Currently only the
    first subnet is added to the FIP namespace routing table. Packets for
    FIPs with addresses in other subnets can't pass through the external
    port because there is no route for those FIP CIDRs.

    This change adds routes for those CIDRs via the external port IP and
    interface.

    These routes doesn't collide with the existing ones, added to provide
    a back path for the packets with a destination IP matching a FIP.

    E.g.:
    $ ip netns exec fip-e1ec0f98-b593-4514-ae08-f1c5cf1c2788 ip route
      (1) 169.254.106.114/31 dev fpr-3937f879-d proto kernel scope link \
          src 169.254.106.115
      (2) 192.168.20.250 via 169.254.106.114 dev fpr-3937f879-d
      (3) 192.168.30.0/24 dev fg-bee060f1-dd proto kernel scope link \
          src 192.168.30.129
      (4) 192.168.20.0/24 via 192.168.30.129 dev fg-bee060f1-dd scope link

    Rule (2) is added when a FIP is assigned. This rule permits ingress
    packets going into the router namespace. This FIP belongs to the second
    subnet of the external network (note the external port CIDR is not the
    same). Rule (4), added by this patch, allows egress packets to exit
    the FIP namespace through the external port. Rule (2), because of the
    prefix length (32), has more priority than rule (4).

    Change-Id: I4d476b47e89fa5709dca2f66ffae72a27d88340a
    Closes-Bug: #1805456
    (cherry picked from commit 97c98a1c6dac5788690504463a5ca33f4b181d6a)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 13.0.3

This issue was fixed in the openstack/neutron 13.0.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 12.0.6

This issue was fixed in the openstack/neutron 12.0.6 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.