poor network speed due to tso enabled

Bug #1551179 reported by Rossella Sblendido on 2016-02-29
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Medium
Unassigned

Bug Description

In some deployments we are experiencing low network speed. When disabling tso on all virtual interfaces the problem is fixed. See also [1]. I need to dig more into it, anyway I wonder if we should disable TSO automatically every time Neutron creates a vif...

[1] http://askubuntu.com/questions/503863/poor-upload-speed-in-kvm-guest-with-virtio-eth-driver-in-openstack-on-3-14

Changed in neutron:
status: New → Confirmed
importance: Undecided → Medium
Changed in neutron:
assignee: nobody → Mohammed Ashraf (mohammed-asharaf)
Changed in neutron:
assignee: Mohammed Ashraf (mohammed-asharaf) → nobody
Sreekumar S (sreesiv) wrote :

The perf issue with TSO, GSO enabled spans across hypervisors and Linux SKUs.
Details can be seen https://kris.io/2015/10/01/kvm-network-performance-tso-and-gso-turn-it-off/

Redhat has issued advisory for RHEL6 for "Network performance issues" https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/ch10s04.html

https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/Virtualization_Deployment_and_Administration_Guide/ch10s04.html

Related Bug: https://bugs.launchpad.net/qemu/+bug/1202289

However later RHEL docs https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/6.5_Technical_Notes/kernel.html mentions some fixes which 'may' address this.

Anyway this seems to be an unreliable feature to be enabled, and the folks at Qemu are not in favor of changing the defaults to "Off" for these flags, which seems fair...
https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01165.html

So looking at this holistically I think, we should maintain the default behavior as is. But we can add a config option to disable these features on the interface/bridge for troubleshooting purposes.
If someone hits nw perf issue, they can turn these in config and have TSO, GSO, GRO etc. disabled.

I will work on this. Welcoming suggestions...

Changed in neutron:
assignee: nobody → Sreekumar S (sreesiv)
Rossella Sblendido (rossella-o) wrote :

sreesiv before making any decision I think we need more investigation. we should identify the cause of the slow speed, how is tso involved, if the problem is in the kernel or somewhere else.

Sreekumar S (sreesiv) on 2017-03-13
Changed in neutron:
assignee: Sreekumar S (sreesiv) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers