WARNING: CPU: 28 PID: 34085 at /build/linux-90Gc2C/linux-3.13.0/net/core/dev.c:1433 dev_disable_lro+0x87/0x90()
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Trusty |
Fix Released
|
Medium
|
Unassigned |
Bug Description
[impact]
with a specific network configuration, a kernel WARNING is issued.
[test case]
configure a trusty system with e/n/i like this:
auto bond0.2004
iface bond0.2004 inet manual
mtu 9100
vlan-raw-device bond0
auto bond0
iface bond0 inet manual
mtu 9100
bond-slaves eth1 eth2
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate fast
bond-updelay 3000
bond-downdelay 1000
bond-ad-select bandwidth
bond-xmit-
post-up sleep 45
auto br-mesh
iface br-mesh inet static
bridge_ports bond0.2004
address 192.168.210.3/24
auto eth1
iface eth1 inet manual
mtu 9100
bond-master bond0
auto eth2
iface eth2 inet manual
mtu 9100
bond-master bond0
adjusting the interface names as needed, of course. then reboot, and the kernel WARNING will appear in the dmesg.
[regression potential]
the kernel patch has the potential to incorrectly set, or incorrectly not set, features on devices.
[other info]
previous bug description:
I have multiple instances of this dev_disable_lro error in kern.log. Also seeing this:
systemd-
<4>May 1 22:56:42 xxx kernel: [ 404.520990] bonding: bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond
<4>May 1 22:56:44 xxx kernel: [ 406.926429] bonding: bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond
<4>May 1 22:56:45 xxx kernel: [ 407.569020] ------------[ cut here ]------------
<4>May 1 22:56:45 xxx kernel: [ 407.569029] WARNING: CPU: 28 PID: 34085 at /build/
<4>May 1 22:56:45 xxx kernel: [ 407.569032] netdevice: bond0.2004
<4>May 1 22:56:45 xxx kernel: [ 407.569032] failed to disable LRO!
<4>May 1 22:56:45 xxx kernel: [ 407.569035] Modules linked in: 8021q garp mrp bridge stp llc bonding iptable_filter ip_tables x_tables nf_conntrack_
<4>May 1 22:56:45 xxx kernel: [ 407.569112] CPU: 28 PID: 34085 Comm: brctl Not tainted 3.13.0-142-generic #191-Ubuntu
<4>May 1 22:56:45 xxx kernel: [ 407.569115] Hardware name: Dell Inc. PowerEdge R730xd/072T6D, BIOS 2.7.1 001/22/2018
<4>May 1 22:56:45 xxx kernel: [ 407.569118] 0000000000000000 ffff881fcc753c70 ffffffff8172e7fc ffff881fcc753cb8
<4>May 1 22:56:45 xxx kernel: [ 407.569129] 0000000000000009 ffff881fcc753ca8 ffffffff8106afad ffff883fcc6f8000
<4>May 1 22:56:45 xxx kernel: [ 407.569139] ffff883fcc696880 ffff883fcc6f8000 0000000000000000 ffff881fce82dd40
<4>May 1 22:56:45 xxx kernel: [ 407.569150] Call Trace:
<4>May 1 22:56:45 xxx kernel: [ 407.569160] [<ffffffff8172e
<4>May 1 22:56:45 xxx kernel: [ 407.569168] [<ffffffff8106a
<4>May 1 22:56:45 xxx kernel: [ 407.569175] [<ffffffff8106b
<4>May 1 22:56:45 xxx kernel: [ 407.569183] [<ffffffff81634
<4>May 1 22:56:45 xxx kernel: [ 407.569195] [<ffffffffa03f2
<4>May 1 22:56:45 xxx kernel: [ 407.569205] [<ffffffffa03f2
<4>May 1 22:56:45 xxx kernel: [ 407.569215] [<ffffffffa03f3
<4>May 1 22:56:45 xxx kernel: [ 407.569223] [<ffffffff81646
<4>May 1 22:56:45 xxx kernel: [ 407.569230] [<ffffffff8162d
<4>May 1 22:56:45 xxx kernel: [ 407.569237] [<ffffffff81646
<4>May 1 22:56:45 xxx kernel: [ 407.569245] [<ffffffff81614
<4>May 1 22:56:45 xxx kernel: [ 407.569252] [<ffffffff81615
<4>May 1 22:56:45 xxx kernel: [ 407.569260] [<ffffffff811d6
<4>May 1 22:56:45 xxx kernel: [ 407.569267] [<ffffffff811c4
<4>May 1 22:56:45 xxx kernel: [ 407.569273] [<ffffffff811d6
<4>May 1 22:56:45 xxx kernel: [ 407.569283] [<ffffffff8173f
<4>May 1 22:56:45 xxx kernel: [ 407.569287] ---[ end trace df5aa31d75a7e2b1 ]---
<4>May 1 22:56:54 xxx kernel: [ 416.320138] bonding: bond1: Warning: the permanent HWaddr of enp131s0f0 - a0:36:9f:c1:25:d0 - is still in use by bond1. Set the HWaddr of enp131s0f0 to a different address to avoid conflicts.
<4>May 1 22:57:00 xxx kernel: [ 422.362677] bonding: bond1: Warning: No 802.3ad response from the link partner for any adapters in the bond
<4>May 1 22:57:07 xxx kernel: [ 429.522843] ------------[ cut here ]------------
And here:
<4>May 1 22:57:07 xxx kernel: [ 429.522843] ------------[ cut here ]------------
<4>May 1 22:57:07 xxx kernel: [ 429.522853] WARNING: CPU: 0 PID: 34191 at /build/
<4>May 1 22:57:07 xxx kernel: [ 429.522857] netdevice: bond1.2001
<4>May 1 22:57:07 xxx kernel: [ 429.522857] failed to disable LRO!
<4>May 1 22:57:07 xxx kernel: [ 429.522861] Modules linked in: 8021q garp mrp bridge stp llc bonding iptable_filter ip_tables x_tables nf_conntrack_
<4>May 1 22:57:07 xxx kernel: [ 429.522933] CPU: 0 PID: 34191 Comm: brctl Tainted: G W 3.13.0-142-generic #191-Ubuntu
<4>May 1 22:57:07 xxx kernel: [ 429.522937] Hardware name: Dell Inc. PowerEdge R730xd/072T6D, BIOS 2.7.1 001/22/2018
<4>May 1 22:57:07 xxx kernel: [ 429.522940] 0000000000000000 ffff88056ca57c70 ffffffff8172e7fc ffff88056ca57cb8
<4>May 1 22:57:07 xxx kernel: [ 429.522951] 0000000000000009 ffff88056ca57ca8 ffffffff8106afad ffff883fcc9a3000
<4>May 1 22:57:07 xxx kernel: [ 429.522961] ffff881fce800880 ffff883fcc9a3000 0000000000000000 ffff883fcdb93d40
<4>May 1 22:57:07 xxx kernel: [ 429.522972] Call Trace:
<4>May 1 22:57:07 xxx kernel: [ 429.522981] [<ffffffff8172e
<4>May 1 22:57:07 xxx kernel: [ 429.522989] [<ffffffff8106a
<4>May 1 22:57:07 xxx kernel: [ 429.522996] [<ffffffff8106b
<4>May 1 22:57:07 xxx kernel: [ 429.523004] [<ffffffff81634
<4>May 1 22:57:07 xxx kernel: [ 429.523016] [<ffffffffa03f2
<4>May 1 22:57:07 xxx kernel: [ 429.523027] [<ffffffffa03f2
<4>May 1 22:57:07 xxx kernel: [ 429.523038] [<ffffffffa03f3
<4>May 1 22:57:07 xxx kernel: [ 429.523047] [<ffffffff81646
<4>May 1 22:57:07 xxx kernel: [ 429.523054] [<ffffffff8162d
<4>May 1 22:57:07 xxx kernel: [ 429.523061] [<ffffffff81646
<4>May 1 22:57:07 xxx kernel: [ 429.523070] [<ffffffff81614
<4>May 1 22:57:07 xxx kernel: [ 429.523076] [<ffffffff81615
<4>May 1 22:57:07 xxx kernel: [ 429.523084] [<ffffffff811d6
<4>May 1 22:57:07 xxx kernel: [ 429.523091] [<ffffffff811c4
<4>May 1 22:57:07 xxx kernel: [ 429.523098] [<ffffffff811d6
<4>May 1 22:57:07 xxx kernel: [ 429.523107] [<ffffffff8173f
<4>May 1 22:57:07 xxx kernel: [ 429.523111] ---[ end trace df5aa31d75a7e2b2 ]---
root@xxx:/var/log# ip link|grep enp
6: enp3s0f0: <BROADCAST,
7: enp3s0f1: <BROADCAST,
8: enp131s0f0: <BROADCAST,
9: enp131s0f1: <BROADCAST,
root@xxx:/ar/log# ethtool -i enp3s0f0
driver: ixgbe
version: 3.15.1-k
firmware-version: 0x800008bd
bus-info: 0000:03:00.0
supports-
supports-test: yes
supports-
supports-
supports-
root@xxx:/var/log# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
root@xxx:/var/log# cat /proc/version
Linux version 3.13.0-142-generic (buildd@
CVE References
Changed in linux (Ubuntu): | |
importance: | Undecided → Medium |
status: | Incomplete → Triaged |
description: | updated |
Changed in linux (Ubuntu Trusty): | |
importance: | Undecided → Medium |
status: | New → Triaged |
Changed in linux (Ubuntu Trusty): | |
status: | Triaged → Fix Committed |
Changed in linux (Ubuntu): | |
status: | Triaged → Fix Released |
This occurred during the deployment of the DPDK compute. The DPDK was not deployed. The bond0 and bond1 configuration appears to have failed.
------- ----8<- ------- ---
Preliminary analysis: the LRO appears to be off after the kernel traces, despite the message. I am inquiring as to the nature of the failure.
This bug appears to be relevant: /bugs.launchpad .net/ubuntu/ +source/ linux/+ bug/1660146
https:/
All NICs involved: statistics: yes eeprom- access: yes register- dump: yes priv-flags: no
driver: ixgbe
version: 3.15.1-k
firmware-version: 0x800008bd
bus-info: 0000:03:00.0
supports-
supports-test: yes
supports-
supports-
supports-
Bonds:
bond0: Adding slave enp131s0f1.
bond0: Adding slave enp3s0f0.
bond1: Adding slave enp131s0f0.
bond1: Adding slave enp3s0f1.
Overview of dmesg:
[ 27.423698] bonding: bond0: link status definitely up for interface enp131s0f1, 10000 Mbps full duplex.
[ 27.423704] bonding: bond0: link status definitely up for interface enp3s0f0, 10000 Mbps full duplex.
[ 27.423711] bonding: bond1: link status definitely up for interface enp3s0f1, 10000 Mbps full duplex.
[ 27.423716] bonding: bond1: link status definitely up for interface enp131s0f0, 10000 Mbps full duplex.
[ 27.434165] device bond0.2004 entered promiscuous mode linux-90Gc2C/ linux-3. 13.0/net/ core/dev. c:1433 dev_disable_ lro+0x87/ 0x90()
[ 27.434692] ------------[ cut here ]------------
[ 27.434704] WARNING: CPU: 1 PID: 3942 at /build/
[ 27.434708] netdevice: bond0.2004
[ 27.434708] failed to disable LRO!
[ 27.523777] bonding: bond0: link status down for interface enp131s0f1, disabling it in 1000 ms.
[ 28.916892] ixgbe 0000:83:00.1 enp131s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 28.920854] bonding: bond0: link status up again after 1000 ms for interface enp131s0f1.
[ 29.020935] bonding: bond0: link status down for interface enp3s0f0, disabling it in 1000 ms.
[ 29.401337] ixgbe 0000:03:00.0 enp3s0f0: detected SFP+: 3
[ 29.541388] ixgbe 0000:03:00.0 enp3s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 29.621400] bonding: bond0: link status up again after 600 ms for interface enp3s0f0.
[ 29.338269] device bond1.2001 entered promiscuous mode linux-90Gc2C/ linux-3. 13.0/net/ core/dev. c:1433 dev_disable_ lro+0x87/ 0x90()
[ 29.338689] ------------[ cut here ]------------
[ 29.338699] WARNING: CPU: 0 PID: 3944 at /build/
[ 29.338702] netdevice: bond1.2001
[ 29.338702] failed to disable LRO!
[ 29.429252] bonding: bond1: link status down for interface enp3s0f1, disabling it in 1000 ms. NETDEV_ UP): br-mesh: link is not ready
[ 29.824836] ixgbe 0000:03:00.1 enp3s0f1: detected SFP+: 4
[ 29.829564] bonding: bond1: link status down for interface enp131s0f0, disabling it in 1000 ms.
[ 30.069790] ixgbe 0000:03:00.1 enp3s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 30.129796] bonding: bond1: link status up again after 700 ms for interface enp3s0f1.
[ 31.244068] IPv6: ADDRCONF(
[ 31.246724] ixgbe 0000:83:00.0 enp131s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 31.250679] bonding: bond1: link status up again after 1000 ms for interface enp131s0f0.
[ 33.324580] device bond0 entered promiscuous mode
[ 33.324585] device enp131s0f1 entered promiscuous mode
[ 33.324995] de...