Connectivity issues due to skb marks on the encapsulating packet

[Oleg Bondarev]

Looks like by default OVS tunnels inherit skb marks from tunneled packets.
As a result Neutron IPTables marks set in qrouter namespace are inherited by VXLAN encapsulating packets.
These marks may conflict with marks used by underlying networking (like Calico) and lead to VXLAN
tunneled packets being dropped.

The proposal is to set 'egress_pkt_mark = 0' explicitly for tunnel ports. The option was added in OVS 2.8.0 (https://www.openvswitch.org/releases/NEWS-2.8.0.txt)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/675054

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/675054
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=762773525234814c1c47b5d21e072a30a94ff9e6
Submitter: Zuul
Branch: master

commit 762773525234814c1c47b5d21e072a30a94ff9e6
Author: Oleg Bondarev <email address hidden>
Date: Wed Aug 7 12:14:18 2019 +0400

    Clear skb mark on encapsulating packets

    Looks like by default OVS tunnels inherit skb marks from
    tunneled packets. As a result Neutron IPTables marks set in
    qrouter namespace are inherited by VXLAN encapsulating packets.
    These marks may conflict with marks used by underlying networking
    (like Calico) and lead to VXLAN tunneled packets being dropped.

    This patch ensures that skb marks are cleared by OVS before entering
    a tunnel to avoid conflicts with IPTables rules in default namespace.

    Closes-Bug: #1839252
    Change-Id: Id029be51bffe4188dd7f2155db16b21d19da1698

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/675728

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/675729

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/675730

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.opendev.org/675731

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/675729
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d581cf07cdb06065689a901c20a3d887f2b0c8b6
Submitter: Zuul
Branch: stable/rocky

commit d581cf07cdb06065689a901c20a3d887f2b0c8b6
Author: Oleg Bondarev <email address hidden>
Date: Wed Aug 7 12:14:18 2019 +0400

    Clear skb mark on encapsulating packets

    Looks like by default OVS tunnels inherit skb marks from
    tunneled packets. As a result Neutron IPTables marks set in
    qrouter namespace are inherited by VXLAN encapsulating packets.
    These marks may conflict with marks used by underlying networking
    (like Calico) and lead to VXLAN tunneled packets being dropped.

    This patch ensures that skb marks are cleared by OVS before entering
    a tunnel to avoid conflicts with IPTables rules in default namespace.

    Closes-Bug: #1839252
    Change-Id: Id029be51bffe4188dd7f2155db16b21d19da1698
    (cherry picked from commit 762773525234814c1c47b5d21e072a30a94ff9e6)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/675730
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e4b8795c23d24f180e23456683a2e2796d0247b7
Submitter: Zuul
Branch: stable/queens

commit e4b8795c23d24f180e23456683a2e2796d0247b7
Author: Oleg Bondarev <email address hidden>
Date: Wed Aug 7 12:14:18 2019 +0400

    Clear skb mark on encapsulating packets

    Looks like by default OVS tunnels inherit skb marks from
    tunneled packets. As a result Neutron IPTables marks set in
    qrouter namespace are inherited by VXLAN encapsulating packets.
    These marks may conflict with marks used by underlying networking
    (like Calico) and lead to VXLAN tunneled packets being dropped.

    This patch ensures that skb marks are cleared by OVS before entering
    a tunnel to avoid conflicts with IPTables rules in default namespace.

    Closes-Bug: #1839252
    Change-Id: Id029be51bffe4188dd7f2155db16b21d19da1698
    (cherry picked from commit 762773525234814c1c47b5d21e072a30a94ff9e6)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/675731
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=62f26a24e3d08321252a1144d76666b63cab6a50
Submitter: Zuul
Branch: stable/pike

commit 62f26a24e3d08321252a1144d76666b63cab6a50
Author: Oleg Bondarev <email address hidden>
Date: Wed Aug 7 12:14:18 2019 +0400

    Clear skb mark on encapsulating packets

    Looks like by default OVS tunnels inherit skb marks from
    tunneled packets. As a result Neutron IPTables marks set in
    qrouter namespace are inherited by VXLAN encapsulating packets.
    These marks may conflict with marks used by underlying networking
    (like Calico) and lead to VXLAN tunneled packets being dropped.

    This patch ensures that skb marks are cleared by OVS before entering
    a tunnel to avoid conflicts with IPTables rules in default namespace.

    Closes-Bug: #1839252
    Change-Id: Id029be51bffe4188dd7f2155db16b21d19da1698
    (cherry picked from commit 762773525234814c1c47b5d21e072a30a94ff9e6)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/675728
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9f6d8c383f4d7f5ea6e4c157fe7eb176042960cf
Submitter: Zuul
Branch: stable/stein

commit 9f6d8c383f4d7f5ea6e4c157fe7eb176042960cf
Author: Oleg Bondarev <email address hidden>
Date: Wed Aug 7 12:14:18 2019 +0400

    Clear skb mark on encapsulating packets

    Looks like by default OVS tunnels inherit skb marks from
    tunneled packets. As a result Neutron IPTables marks set in
    qrouter namespace are inherited by VXLAN encapsulating packets.
    These marks may conflict with marks used by underlying networking
    (like Calico) and lead to VXLAN tunneled packets being dropped.

    This patch ensures that skb marks are cleared by OVS before entering
    a tunnel to avoid conflicts with IPTables rules in default namespace.

    Closes-Bug: #1839252
    Change-Id: Id029be51bffe4188dd7f2155db16b21d19da1698
    (cherry picked from commit 762773525234814c1c47b5d21e072a30a94ff9e6)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.5 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron pike-eol release.