| 2020-12-09 11:28:25 |
James Page |
description |
Ubuntu: Focal
OpenStack: Ussuri
OVN: 20.03.1
OVS: 2.12.1
Mellanox ConnectX 5 Ex cards connected to 100G TOR Cumulus switches in MLAG configuration.
Leaf and Spine network topology between racks.
8 core instances
Ubuntu: Focal
iperf -P4 -c XXXX -e
performance between to instances on different hypervisors is consistent by asymmetric - ~90Gbps in one direction, and ~30Gbps in the other.
some dmesg entries that might be pertinent (similar messages on both hypervisors):
[92583.318283] mlx5_core 0000:81:00.1 enp129s0f1: Matching on GENEVE options: can't match on 0 data field
[92583.318286] mlx5_core 0000:81:00.0 enp129s0f0: Matching on GENEVE options: can't match on 0 data field
[92693.121984] mlx5_core 0000:81:00.1: cmd_work_handler:887:(pid 1137976): failed to allocate command entry
[92693.131803] vport 31 error -11 reading stats
VF's for hardware offloaded ports are allocated on enp129s0f0
Monitoring the representator port on the hypervisor the initial TCP setup is observed but not the main iperf test (so offloading appears to be working). |
Ubuntu: Focal
OpenStack: Ussuri
OVN: 20.03.1
OVS: 2.12.1
Mellanox ConnectX 5 Ex cards connected to 100G TOR Cumulus switches in MLAG configuration.
Leaf and Spine network topology between racks.
8 core instances
Ubuntu: Focal
iperf -P4 -c XXXX -e
performance between to instances on different hypervisors is consistent by asymmetric - ~90Gbps in one direction, and ~30Gbps in the other.
some dmesg entries that might be pertinent (similar messages on both hypervisors):
[92583.318283] mlx5_core 0000:81:00.1 enp129s0f1: Matching on GENEVE options: can't match on 0 data field
[92583.318286] mlx5_core 0000:81:00.0 enp129s0f0: Matching on GENEVE options: can't match on 0 data field
[92693.121984] mlx5_core 0000:81:00.1: cmd_work_handler:887:(pid 1137976): failed to allocate command entry
[92693.131803] vport 31 error -11 reading stats
VF's for hardware offloaded ports are allocated on enp129s0f0
Monitoring the representator port on the hypervisor the initial TCP setup is observed but not the main iperf test (so offloading appears to be working). |
|
