[l3][dvr] with openflow security group east-west traffic between different vlan networks is broken

[LIU Yulong]

ENV: stable/queens & master

This is a long story long time ago [1] [2] [3]. But we recently meet such issue, when dvr router is connected to two different vlan networks, the east-west traffic is not reachable.

# ovs-ofctl show br-int

 1(int-br-ex): addr:22:32:17:d4:08:6a
 2(int-br-vlan): addr:76:ed:47:bf:21:ec
 3(patch-tun): addr:9a:56:bf:23:ac:37
 ...
 ...
 255(tap321a4669-c2): addr:fe:16:3e:93:31:67
 LOCAL(br-int): addr:7a:ae:b6:87:7b:4d

# ovs-ofctl dump-flows br-int
 # this will be applied aways, since it has higher priority, fa:16:3f:93:05:7d is the dvr host mac from request VM's hypervisor
 cookie=0xb27e128dd9a83dfc, duration=6408639.091s, table=0, n_packets=22187, n_bytes=30725358, idle_age=860, hard_age=65534, priority=4,in_port=2,dl_src=fa:16:3f:93:05:7d actions=resubmit(,2)
 # this will not get matched
 cookie=0xb27e128dd9a83dfc, duration=116506.106s, table=0, n_packets=60698, n_bytes=80563747, idle_age=825, hard_age=65534, priority=3,in_port=2,dl_vlan=587 actions=mod_vlan_vid:45,resubmit(,60)
 cookie=0xb27e128dd9a83dfc, duration=167233.168s, table=2, n_packets=22177, n_bytes=30724518, idle_age=51621, hard_age=65534, priority=4,dl_vlan=587,dl_dst=fa:16:3e:93:31:67 actions=mod_dl_src:fa:16:3e:ca:bf:28,resubmit(,60)
 cookie=0xb27e128dd9a83dfc, duration=167719.120s, table=60, n_packets=22257, n_bytes=30732678, idle_age=4, hard_age=65534, priority=4,dl_vlan=587,dl_dst=fa:16:3e:93:31:67 actions=strip_vlan,output:255

Since the request packet never go into conntrack table, so the reply packets will be dropped.

[1] https://specs.openstack.org/openstack/neutron-specs/specs/kilo/neutron-ovs-dvr-vlan.html
[2] https://blueprints.launchpad.net/neutron/+spec/neutron-ovs-dvr-vlan
[3] https://review.opendev.org/#/q/topic:bp/neutron-ovs-dvr-vlan

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/662925

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/663008

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/665517

[OpenStack Infra (hudson-openstack)]

Change abandoned by LIU Yulong (<email address hidden>) on branch: master
Review: https://review.opendev.org/662925
Reason: This is not achievable for upstream CI now.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/665517
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=aa58542e823d23d233524cd5639c7ec4bb757769
Submitter: Zuul
Branch: master

commit aa58542e823d23d233524cd5639c7ec4bb757769
Author: LIU Yulong <email address hidden>
Date: Sat Jun 15 22:48:53 2019 +0800

    Add VLAN type conntrack direct flow

    For vlan type network, we add a segment match flow
    to the openflow security group ingress table. Then
    the packets will be recorded in conntrack table, and
    the reply packets can be processed properly.

    Change-Id: Ieded0654d0ad16235ec923b822dcd842bd7735e5
    Closes-Bug: #1831534

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/709409

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/710181

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/710182

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/709409
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d5e168b281e704d5954cd06bcf941b183eb7c8d7
Submitter: Zuul
Branch: stable/queens

commit d5e168b281e704d5954cd06bcf941b183eb7c8d7
Author: LIU Yulong <email address hidden>
Date: Sat Jun 15 22:48:53 2019 +0800

    Add VLAN type conntrack direct flow

    For vlan type network, we add a segment match flow
    to the openflow security group ingress table. Then
    the packets will be recorded in conntrack table, and
    the reply packets can be processed properly.

    Conflicts:
     doc/source/contributor/internals/openvswitch_firewall.rst

    Change-Id: Ieded0654d0ad16235ec923b822dcd842bd7735e5
    Closes-Bug: #1831534
    (cherry picked from commit aa58542e823d23d233524cd5639c7ec4bb757769)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/710181
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=eec11491c37d0a1890b07c68b269762ac4ed5b5d
Submitter: Zuul
Branch: stable/stein

commit eec11491c37d0a1890b07c68b269762ac4ed5b5d
Author: LIU Yulong <email address hidden>
Date: Sat Jun 15 22:48:53 2019 +0800

    Add VLAN type conntrack direct flow

    For vlan type network, we add a segment match flow
    to the openflow security group ingress table. Then
    the packets will be recorded in conntrack table, and
    the reply packets can be processed properly.

    Change-Id: Ieded0654d0ad16235ec923b822dcd842bd7735e5
    Closes-Bug: #1831534
    (cherry picked from commit aa58542e823d23d233524cd5639c7ec4bb757769)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/710182
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=dabb77fcbc481e3b57f223b0c55c888dcfbe1410
Submitter: Zuul
Branch: stable/rocky

commit dabb77fcbc481e3b57f223b0c55c888dcfbe1410
Author: LIU Yulong <email address hidden>
Date: Sat Jun 15 22:48:53 2019 +0800

    Add VLAN type conntrack direct flow

    For vlan type network, we add a segment match flow
    to the openflow security group ingress table. Then
    the packets will be recorded in conntrack table, and
    the reply packets can be processed properly.

    Conflicts:
        doc/source/contributor/internals/openvswitch_firewall.rst

    Change-Id: Ieded0654d0ad16235ec923b822dcd842bd7735e5
    Closes-Bug: #1831534
    (cherry picked from commit aa58542e823d23d233524cd5639c7ec4bb757769)

[OpenStack Infra (hudson-openstack)]

Change abandoned by "liuyulong <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/663008
Reason: Plz restore if this still needed.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron queens-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron rocky-eol release.