RTL8111/8168/8411 throughput/power/CPU bug

[Brian Lee]

Hi,

I'm running into an issue where the RTL8111/8168/8411 (rev 15) gigabit ethernet adapter is stuck at 700mbps transfer speeds, but will be able to hit 940-950mbps speeds as soon as the CPU is stressed to 100%. It is the same as this bug posted several years ago: https://superuser.com/questions/1428360/poor-network-throughput-but-double-under-high-cpu-load/1749982

Exact instructions to reproduce this bug:

on mini PC run: iperf3 -s -B <interface IP>
on client run: iperf3 -R -c miniPC.lan

λ iperf3 -R -c 192.168.1.103 -t 600 # download
Connecting to host 192.168.1.103, port 5201
Reverse mode, remote host 192.168.1.103 is sending
[ 5] local 192.168.1.127 port 34124 connected to 192.168.1.103 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 82.6 MBytes 693 Mbits/sec
[ 5] 1.00-2.00 sec 83.8 MBytes 703 Mbits/sec
[ 5] 2.00-3.00 sec 83.3 MBytes 699 Mbits/sec
[ 5] 3.00-4.00 sec 82.8 MBytes 694 Mbits/sec
[ 5] 4.00-5.00 sec 82.8 MBytes 694 Mbits/sec
[ 5] 5.00-6.00 sec 83.2 MBytes 698 Mbits/sec
[ 5] 6.00-7.00 sec 83.3 MBytes 699 Mbits/sec
[ 5] 7.00-8.00 sec 83.1 MBytes 697 Mbits/sec
[ 5] 8.00-9.00 sec 82.5 MBytes 692 Mbits/sec
[ 5] 9.00-10.00 sec 82.6 MBytes 693 Mbits/sec
[ 5] 10.00-11.00 sec 82.7 MBytes 693 Mbits/sec
[ 5] 11.00-12.00 sec 83.4 MBytes 699 Mbits/sec
[ 5] 12.00-13.00 sec 83.1 MBytes 697 Mbits/sec
[ 5] 13.00-14.00 sec 82.8 MBytes 695 Mbits/sec
[ 5] 14.00-15.00 sec 82.7 MBytes 694 Mbits/sec
[ 5] 15.00-16.00 sec 83.1 MBytes 697 Mbits/sec
[ 5] 16.00-17.00 sec 83.8 MBytes 703 Mbits/sec
[ 5] 17.00-18.00 sec 83.2 MBytes 698 Mbits/sec
[ 5] 18.00-19.00 sec 82.5 MBytes 692 Mbits/sec
[ 5] 19.00-19.09 sec 7.42 MBytes 694 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-19.09 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-19.09 sec 1.55 GBytes 696 Mbits/sec receiver
iperf3: interrupt - the client has terminated

under stress test

I ran s-tui on the mini PC, enabled the stress test option, and saw all cores boosted to 2.90 GHz (max turbo on an Intel N5095). The ethernet speeds immediately jumped to gigabit wire speeds:

λ iperf3 -R -c 192.168.1.103 -t 600 # download
Connecting to host 192.168.1.103, port 5201
Reverse mode, remote host 192.168.1.103 is sending
[ 5] local 192.168.1.127 port 34144 connected to 192.168.1.103 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 113 MBytes 946 Mbits/sec
[ 5] 1.00-2.00 sec 113 MBytes 949 Mbits/sec
[ 5] 2.00-3.00 sec 113 MBytes 949 Mbits/sec
[ 5] 3.00-4.00 sec 113 MBytes 949 Mbits/sec
[ 5] 4.00-5.00 sec 113 MBytes 949 Mbits/sec
[ 5] 5.00-6.00 sec 113 MBytes 949 Mbits/sec
[ 5] 6.00-6.27 sec 31.0 MBytes 949 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-6.27 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-6.27 sec 709 MBytes 949 Mbits/sec receiver
iperf3: interrupt - the client has terminated

I have tried both the r8169 and r8168 drivers on both 22.04 (kernel 5.15 and 5.19) and also on 22.10 (only kernel 5.19 tested). This issue remains on all combinations tested. When I tested using r8168 same issue showed up but speeds were constantly ~800mbps instead of 700mbps, but same thing - stress testing the cpu to 100% made it go to 940-950mbps instead.

I verified the driver being used by running the following:

root@snuminipc:~# ethtool -i enp2s0
driver: r8169
version: 5.19.0-23-generic
firmware-version: rtl8168h-2_0.0.2 02/26/15
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

ProblemType: Bug
DistroRelease: Ubuntu 22.10
Package: linux-image-5.19.0-23-generic 5.19.0-23.24
ProcVersionSignature: Ubuntu 5.19.0-23.24-generic 5.19.7
Uname: Linux 5.19.0-23-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version k5.19.0-23-generic.
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.23.1-0ubuntu3
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/hwC0D2', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D12p', '/dev/snd/pcmC0D11p', '/dev/snd/pcmC0D10p', '/dev/snd/pcmC0D9p', '/dev/snd/pcmC0D8p', '/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer'
Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer'
CasperMD5CheckResult: pass
Date: Fri Oct 28 14:38:41 2022
InstallationDate: Installed on 2022-10-16 (11 days ago)
InstallationMedia: Ubuntu-Server 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
 Bus 002 Device 002: ID 174c:1153 ASMedia Technology Inc. ASM1153 SATA 3Gb/s bridge
 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
 Bus 001 Device 003: ID 8087:0a2a Intel Corp. Bluetooth wireless interface
 Bus 001 Device 002: ID 0bda:0129 Realtek Semiconductor Corp. RTS5129 Card Reader Controller
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: GEEKOM MiniAir 11
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.19.0-23-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
 linux-restricted-modules-5.19.0-23-generic N/A
 linux-backports-modules-5.19.0-23-generic N/A
 linux-firmware 20220923.gitf09bebf3-0ubuntu1
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: Upgraded to kinetic on 2022-10-28 (0 days ago)
dmi.bios.date: 06/17/2022
dmi.bios.release: 5.19
dmi.bios.vendor: American Megatrends International, LLC.
dmi.bios.version: 2.06
dmi.board.asset.tag: Default string
dmi.board.name: MiniAir 11
dmi.board.vendor: GEEKOM
dmi.board.version: Default string
dmi.chassis.asset.tag: Default string
dmi.chassis.type: 23
dmi.chassis.vendor: Default string
dmi.chassis.version: Default string
dmi.ec.firmware.release: 2.10
dmi.modalias: dmi:bvnAmericanMegatrendsInternational,LLC.:bvr2.06:bd06/17/2022:br5.19:efr2.10:svnGEEKOM:pnMiniAir11:pvrDefaultstring:rvnGEEKOM:rnMiniAir11:rvrDefaultstring:cvnDefaultstring:ct23:cvrDefaultstring:skuDefaultstring:
dmi.product.family: Default string
dmi.product.name: MiniAir 11
dmi.product.sku: Default string
dmi.product.version: Default string
dmi.sys.vendor: GEEKOM

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This change was made by a bot.

[Brian Lee (snuupy)]

I wanted to try v6.1-rc2 kernel but no builds for amd64 were available, so I tried 2022-10-28 Mainline instead.

root@snuminipc:~# uname -r
6.1.0-060100rc2daily20221028-generic
root@snuminipc:~# ethtool -i enp2s0
driver: r8169
version: 6.1.0-060100rc2daily20221028-ge
firmware-version: rtl8168h-2_0.0.2 02/26/15
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

Same issue:

λ iperf3 -R -c 192.168.1.103 -t 600 # download
Connecting to host 192.168.1.103, port 5201
Reverse mode, remote host 192.168.1.103 is sending
[ 5] local 192.168.1.127 port 63859 connected to 192.168.1.103 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 82.7 MBytes 693 Mbits/sec
[ 5] 1.00-2.00 sec 77.7 MBytes 652 Mbits/sec
[ 5] 2.00-3.00 sec 83.2 MBytes 698 Mbits/sec
[ 5] 3.00-4.00 sec 82.9 MBytes 696 Mbits/sec
[ 5] 4.00-5.00 sec 82.9 MBytes 696 Mbits/sec
[ 5] 5.00-6.00 sec 83.7 MBytes 702 Mbits/sec
[ 5] 6.00-7.00 sec 83.3 MBytes 699 Mbits/sec
[ 5] 7.00-8.00 sec 83.0 MBytes 697 Mbits/sec
[ 5] 8.00-8.99 sec 81.5 MBytes 694 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-8.99 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-8.99 sec 741 MBytes 692 Mbits/sec receiver
iperf3: interrupt - the client has terminated

s-tui stress test:

λ iperf3 -R -c 192.168.1.103 -t 600 # download
Connecting to host 192.168.1.103, port 5201
Reverse mode, remote host 192.168.1.103 is sending
[ 5] local 192.168.1.127 port 63872 connected to 192.168.1.103 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 113 MBytes 945 Mbits/sec
[ 5] 1.00-2.00 sec 113 MBytes 949 Mbits/sec
[ 5] 2.00-3.00 sec 113 MBytes 949 Mbits/sec
[ 5] 3.00-4.00 sec 113 MBytes 949 Mbits/sec
[ 5] 4.00-5.00 sec 113 MBytes 949 Mbits/sec
[ 5] 5.00-6.00 sec 113 MBytes 949 Mbits/sec
[ 5] 6.00-7.00 sec 113 MBytes 949 Mbits/sec
[ 5] 7.00-8.00 sec 113 MBytes 949 Mbits/sec
[ 5] 8.00-8.82 sec 92.9 MBytes 949 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-8.82 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-8.82 sec 998 MBytes 949 Mbits/sec receiver
iperf3: interrupt - the client has terminated