[ovn] Geneve traffic should not create conntrack entries

Bug #1885551 reported by Daniel Alvarez
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Michele Baldessari

Bug Description

Geneve UDP traffic right now creates conntrack entries whenever there's traffic between different hypervisors:

udp 17 26 src=172.17.2.55 dst=172.17.2.75 sport=8543 dport=6081 [UNREPLIED] src=172.17.2.75 dst=172.17.2.55 sport=6081 dport=8543 mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
udp 17 26 src=172.17.2.75 dst=172.17.2.55 sport=6901 dport=6081 [UNREPLIED] src=172.17.2.55 dst=172.17.2.75 sport=6081 dport=6901 mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1

These are not needed and can impact performance. From TripleO we can prevent this from happening by adding the following iptables rules wherever ovn-controller runs (ie. tunnel endpoints).

iptables -t raw -A PREROUTING -p udp --dport 6081 -j NOTRACK
iptables -t raw -A OUTPUT -p udp --dport 6081 -j NOTRACK

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.opendev.org/738419

Changed in tripleo:
assignee: nobody → Daniel Alvarez (dalvarezs)
status: New → In Progress
Changed in tripleo:
importance: Undecided → High
milestone: none → victoria-1
Changed in tripleo:
assignee: Daniel Alvarez (dalvarezs) → Michele Baldessari (michele)
Changed in tripleo:
assignee: Michele Baldessari (michele) → Daniel Alvarez (dalvarezs)
Changed in tripleo:
assignee: Daniel Alvarez (dalvarezs) → Michele Baldessari (michele)
Changed in tripleo:
milestone: victoria-1 → victoria-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/738419
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=a9e95b26bc87c35f8efe54243fee8dcb4089f9b2
Submitter: Zuul
Branch: master

commit a9e95b26bc87c35f8efe54243fee8dcb4089f9b2
Author: Daniel Alvarez <email address hidden>
Date: Mon Jun 29 13:23:20 2020 +0200

    [ovn] Don't add conntrack entries for Geneve

    As Geneve UDP traffic is allowed, there's no reason to create
    conntrack entries as it may result in a performance hit.

    This patch is preventing Geneve traffic to be sent to conntrack.

    Closes-Bug: #1885551
    Change-Id: I1eb6c77ea3cbdfaaa2b2a3fec0e6b8d2a71aae95
    Signed-off-by: Daniel Alvarez <email address hidden>

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/848907
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/359e6b066115c43853af8e9ac2399c5e49118480
Submitter: "Zuul (22348)"
Branch: stable/train

commit 359e6b066115c43853af8e9ac2399c5e49118480
Author: Daniel Alvarez <email address hidden>
Date: Mon Jun 29 13:23:20 2020 +0200

    [ovn] Don't add conntrack entries for Geneve

    As Geneve UDP traffic is allowed, there's no reason to create
    conntrack entries as it may result in a performance hit.

    This patch is preventing Geneve traffic to be sent to conntrack.

    Conflicts:
           deployment/ovn/ovn-controller-container-puppet.yaml

    Fixed conflict caused by switch from puppet-firewall to tripleo-ansible
    for firewall rule management.

    Closes-Bug: #1885551
    Change-Id: I1eb6c77ea3cbdfaaa2b2a3fec0e6b8d2a71aae95
    Signed-off-by: Daniel Alvarez <email address hidden>
    (cherry picked from commit a9e95b26bc87c35f8efe54243fee8dcb4089f9b2)

tags: added: in-stable-train
Revision history for this message
frigo (rigault-francois) wrote (last edit ):

(train)
$ sudo iptables -t raw -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
CT udp -- anywhere anywhere multiport dports geneve state NEW /* 121 neutron geneve networks no conntrack ipv4 */ NOTRACK

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
CT udp -- anywhere anywhere multiport dports geneve state NEW /* 120 neutron geneve networks no conntrack ipv4 */ NOTRACK

$ sudo grep 6081 /proc/net/nf_conntrack | wc -l
161

$ sudo grep 6081 /proc/net/nf_conntrack | tail -3
ipv4 2 udp 17 19 src=10.64.245.158 dst=10.64.245.157 sport=29037 dport=6081 [UNREPLIED] src=10.64.245.157 dst=10.64.245.158 sport=6081 dport=29037 mark=0 secctx=system_u:object_r:unlabeled_t:s0 zone=0 use=2
ipv4 2 udp 17 20 src=10.64.245.158 dst=10.64.245.157 sport=29675 dport=6081 [UNREPLIED] src=10.64.245.157 dst=10.64.245.158 sport=6081 dport=29675 mark=0 secctx=system_u:object_r:unlabeled_t:s0 zone=0 use=2
ipv4 2 udp 17 26 src=10.64.245.158 dst=10.64.245.157 sport=11996 dport=6081 [UNREPLIED] src=10.64.245.157 dst=10.64.245.158 sport=6081 dport=11996 mark=0 secctx=system_u:object_r:unlabeled_t:s0 zone=0 use=2

is the change complete?

(looking at the conntracks I also like how the sport for geneve traffic seems selected at random outside the ephemeral port range, like "4" here:
ipv4 2 udp 17 9 src=172.16.232.217 dst=172.16.233.213 sport=4 dport=6081 [UNREPLIED] src=172.16.233.213 dst=172.16.232.217 sport=6081 dport=4 mark=0 secctx=system_u:object_r:unlabeled_t:s0 zone=0 use=2
)

EDIT: puppet firewall rule module automatically add a default "state new" https://opendev.org/openstack/puppet-tripleo/src/branch/stable/train/manifests/firewall/rule.pp#L57 and that prevents the rule to work properly.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/868743
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/4a27d7c628cfe1f719278303c327dcae7c27adaa
Submitter: "Zuul (22348)"
Branch: stable/train

commit 4a27d7c628cfe1f719278303c327dcae7c27adaa
Author: Luigi Dino Tamagnone <email address hidden>
Date: Wed Dec 28 09:51:15 2022 +0100

    [ovn] Don't add conntrack entries for Geneve

    As Geneve UDP traffic is allowed, there's no reason to create
    conntrack entries as it may result in a performance hit.

    This patch is preventing Geneve traffic to be sent to conntrack.

    Fix for Train only:
    firewall/rule.pp add NEW as default state in the rules,
    this fix add INVALID as state for the geneve UDP rules

    Closes-Bug: #1885551
    Change-Id: I1b582b86780881fe7228540edf281690733fb946

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates train-eol

This issue was fixed in the openstack/tripleo-heat-templates train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.