offloading disabled when using linuxbrige or bonding
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu Cloud Archive |
Won't Fix
|
Undecided
|
Unassigned | ||
openvswitch (Ubuntu) |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
Hi,
I'm running Ubuntu 18.04 kernel 4.15.0-47-generic, openvswitch 2.10.0-
I've found out that as soon as a linuxbrige or bond is involved the network path of openvswitch, the offloading features of the NIC (checksumming, segmentation, VXLAN encap) get disabled, which reduces the throughput a lot.
You can find the flows below. Examples of setup used :
"Normal" setup :
br-bond0 (192.168.11.11)
bond0
enp50s0f0
enp113s0f0 (handling traffic)
Only interface in the bond :
br-bond0 (192.168.11.11)
bond0
enp50s0f0 (handling traffic)
Interface directly in the bridge :
br-bond0 (192.168.11.11)
enp50s0f0
With the setups above, there is NO offloading happening (tcpdump reports packet sizes of the MTU (~9k)). If the IP address is directly on the enp50s0f0 interface, then offloading works.
For what it's worth, the NICs are Mellanox MT27800.
Thanks !
OVS flows :
$ sudo ovs-vsctl show
6f7e2040-
Manager "ptcp:6640:
Bridge "br-physnet1"
Controller "tcp:127.
fail_mode: secure
Port "phy-br-physnet1"
Port "br-physnet1"
Bridge br-int
Controller "tcp:127.
fail_mode: secure
Port "sg-eecf32f7-da"
tag: 2
Port "qvoc654e76e-de"
tag: 2
Port "qr-84debef4-b7"
tag: 2
Port "qr-3e9ad9a7-02"
tag: 3
Port "fg-997d43cb-27"
tag: 5
Port "qvoba76f8a7-50"
tag: 2
Port "qvod01a61c2-59"
tag: 2
Port "enp50s0f0_60"
tag: 2
Port "tap15879e40-d9"
tag: 2
Port "int-br-physnet1"
Port "qg-3f2f0aa4-fa"
tag: 5
Port "ha-a481b72f-6a"
tag: 4
Port "enp50s0f0_62"
tag: 2
Port "sg-5e3c2880-79"
tag: 3
Port "tap62452ada-90"
tag: 3
Port br-int
Port "qvo75f1a5f8-e6"
tag: 2
Port "tapbc9a8481-c2"
tag: 2
Port "tapd7155195-25"
tag: 1
Port patch-tun
Port "qvo1e23c34e-5f"
tag: 2
Port "enp50s0f0_61"
tag: 2
Bridge br-ex
Port br-ex
Bridge br-tun
Controller "tcp:127.
fail_mode: secure
Port patch-int
Port br-tun
Port "vxlan-c0a80a0b"
ovs_version: "2.10.0"
Features for all interfaces :
$ for i in br-bond0 bond0 enp50s0f0 enp113s0f0; do sudo ethtool -k $i; echo "======"; done
Features for br-bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
scatter-gather: on
tcp-segmentatio
udp-fragmentati
generic-
generic-
large-receive-
rx-vlan-offload: off [fixed]
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [requested on]
tx-fcoe-
tx-gre-
tx-gre-
tx-ipxip4-
tx-ipxip6-
tx-udp_
tx-udp_
tx-gso-partial: on
tx-sctp-
tx-esp-
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-
rx-vlan-
rx-vlan-
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-
rx-udp_
======
Features for bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
scatter-gather: on
tcp-segmentatio
udp-fragmentati
generic-
generic-
large-receive-
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-
tx-gre-
tx-gre-
tx-ipxip4-
tx-ipxip6-
tx-udp_
tx-udp_
tx-gso-partial: off [fixed]
tx-sctp-
tx-esp-
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-
rx-vlan-
rx-vlan-
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-
rx-udp_
======
Features for enp50s0f0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentatio
udp-fragmentati
generic-
generic-
large-receive-
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-
tx-gre-
tx-gre-
tx-ipxip4-
tx-ipxip6-
tx-udp_
tx-udp_
tx-gso-partial: on
tx-sctp-
tx-esp-
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-
rx-vlan-
rx-vlan-
l2-fwd-offload: off [fixed]
hw-tc-offload: on
esp-hw-offload: off [fixed]
esp-tx-
rx-udp_
======
Features for enp113s0f0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentatio
udp-fragmentati
generic-
generic-
large-receive-
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-
tx-gre-
tx-gre-
tx-ipxip4-
tx-ipxip6-
tx-udp_
tx-udp_
tx-gso-partial: on
tx-sctp-
tx-esp-
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-
rx-vlan-
rx-vlan-
l2-fwd-offload: off [fixed]
hw-tc-offload: on
esp-hw-offload: off [fixed]
esp-tx-
rx-udp_
======
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: openvswitch-switch 2.10.0-
ProcVersionSign
Uname: Linux 4.15.0-47-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.6
Architecture: amd64
CrashDB:
{
}
Date: Mon Apr 29 09:28:24 2019
ProcEnviron:
TERM=screen-
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
SourcePackage: openvswitch
UpgradeStatus: No upgrade log present (probably fresh install)
Changed in openvswitch (Ubuntu): | |
status: | New → Incomplete |
@axino
Just to confirm whether you tested - if you take OVS out of the picture and just test with bonded network interfaces, do you see offloading still being disabled? Slightly tricky for the VXLAN test but wanted to see if you had tested that combination.
I'm wondering whether this is an OVS issue or whether there is an issue with hardware offloading with linux bonds more generally.