According to the OpenStack Havana documentation http://docs.openstack.org/havana/install-guide/install/apt/content/install-neutron.install-plug-in.ovs.html it is extremely important that GRO be disabled on the underlying interface for br-ex device. We deployed using Fuel 4.0 with HA controllers, Centos, and Neutron GRE based networking. GRO was not turned off after deployment was complete and this resulted in painfully slow network communication between VMs and between VMs and the outside world. Ultimately after a few hours the network was unusable and VMs couldn't even ping their default gateway. After turning GRO off via "ethtool -K ethX gro off", we went from ~60KB/s downloads from CloudFront to ~11MB/s downloads from CloudFront and VM to VM communication was exceptionally fast.
"ethtool -K ethX gro off" will not persist after a reboot, so I believe in Centos ETHTOOL_OPTS="gro off" needs to be added to /etc/sysconfig/network-scripts/ifcfg-ethX. http://www.centos.org/docs/5/html/5.1/Deployment_Guide/s2-networkscripts-interfaces-eth0.html
Richard,
Thank you for reporting this bug!
Could you please provide network card model you run under br-ex?
Please also provide the driver version if you use not the one, shipped with Mirantis Openstack.
Right before the 4.0 release we did special investigation on bad Neutron+GRE performance and found, that setting GRO OFF helps in seldom cases only.
Actually, you were first, who confirmed this setting helped at all.
We also found Neutron+GRE performance greatly depends on underlying hardware and drivers.
Our approach to fix bad GRE performance was to use customized OVS module.
Anyway, I agree, setting GRO OFF for appropriate NICs on permanent basis may help and should not create any additional issues.
Kind regards,
Miroslav