IPv6 with LVS Performance issue in latest 3.13LTS kernels
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Trusty |
Fix Released
|
Medium
|
Unassigned |
Bug Description
We experienced a major performance regression between 12.04's 3.2 kernels
and 14.04's 3.13 kernels when using IPv6 with the LVS load-balancing
facility. Through analysis of perf events and a workaround we've
determined that an upstream fix is available which addresses the issue.
Ubuntu has picked up the "IPv6: remove rt6i_genid" fix that appears to
address our issue in their 3.16 and later kernels. This was checked
into the upstream 3.16 kernel back in late 2014. This fix addressed an
issue introduced by the (late 2012) "ipv6: use net->rt_genid to check dst
validity", which is the source of our issue due to the mismatch between
a dst/route instantiation's rt6i_genid and the IPv6 rt_genid field.
Since we have drivers and other software tied to the 3.13 kernels in
the field this report is requesting that the backport of that upstream
fix be done to the 3.13 LTS kernels as well since we were planning on
several more years for those deployed systems. It seems relatively
straight-forward.
In our 3.13 kernel we used the "systemtap" facility in a test system
to temporarily address the "obsolete" determination mistakenly made by
the ip6_dst_check() function to check dst validity on behalf of the LVS
(ip_vs) code. By updating rt6i_genid to the current global value we were
able to restore our test systems to the previous performance obtained
with the 3.2 kernels. But clearly we want the official upstream fix
incorporated, which pulls the troublesome rt6i_genid field out altogether
since its mishandling affected more than IPv6/LVS support based on the
upstream mail threads. Those were made in mid-2014 to the upstream
kernel folks, such as a per-socket route caching issue.
Note that the apport-bug collected info from a 3.13.0-87 system but we see the
issue on all 3.13.0-xxx kernels.
ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-
ProcVersionSign
Uname: Linux 3.13.0-87-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.21
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/
Date: Mon Aug 29 20:05:57 2016
HibernationDevice: RESUME=
InstallationDate: Installed on 2014-04-30 (852 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140417)
Lsusb:
Bus 001 Device 003: ID 0e0f:0002 VMware, Inc. Virtual USB Hub
Bus 001 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: VMware, Inc. VMware Virtual Platform
ProcEnviron:
LANGUAGE=en_US
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB: 0 svgadrmfb
ProcKernelCmdLine: BOOT_IMAGE=
PulseList:
Error: command ['pacmd', 'list'] failed with exit code 1: Home directory not accessible: Permission denied
No PulseAudio daemon running, or not running as session daemon.
RelatedPackageV
linux-
linux-
linux-firmware 1.127.22
RfKill:
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 05/20/2014
dmi.bios.vendor: Phoenix Technologies LTD
dmi.bios.version: 6.00
dmi.board.name: 440BX Desktop Reference Platform
dmi.board.vendor: Intel Corporation
dmi.board.version: None
dmi.chassis.
dmi.chassis.type: 1
dmi.chassis.vendor: No Enclosure
dmi.chassis.
dmi.modalias: dmi:bvnPhoenixT
dmi.product.name: VMware Virtual Platform
dmi.product.
dmi.sys.vendor: VMware, Inc.
Changed in linux (Ubuntu): | |
importance: | Undecided → Medium |
Changed in linux (Ubuntu Trusty): | |
importance: | Undecided → Medium |
status: | New → Confirmed |
Changed in linux (Ubuntu): | |
status: | Confirmed → In Progress |
Changed in linux (Ubuntu Trusty): | |
status: | Confirmed → In Progress |
Changed in linux (Ubuntu): | |
assignee: | nobody → Joseph Salisbury (jsalisbury) |
Changed in linux (Ubuntu Trusty): | |
assignee: | nobody → Joseph Salisbury (jsalisbury) |
tags: | added: verified-test-kernel-works |
Changed in linux (Ubuntu Trusty): | |
status: | In Progress → Fix Committed |
Changed in linux (Ubuntu): | |
status: | In Progress → Fix Released |
tags: | added: cscc |
This change was made by a bot.