I am running ubuntu Focal version with following kernel and NIC and noticed where strange error in kernel logs and my nic went down, does any one know about this?
# uname -a
Linux ostack-phx-comp-gen-1-27.v1v0x.net 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64x
I am running ubuntu Focal version with following kernel and NIC and noticed where strange error in kernel logs and my nic went down, does any one know about this?
# uname -a phx-comp- gen-1-27. v1v0x.net 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64x
Linux ostack-
# lspci | grep -i eth
06:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)
06:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)
[Thu Sep 16 01:43:51 2021] ------------[ cut here ]------------ sch_generic. c:447 dev_watchdog+ 0x258/0x260 netlink ebt_arp nft_compat nf_tables_set nft_meta_bridgel watchdog+ 0x258/0x260 6b4e30 EFLAGS: 00010286 0(0000) GS:ffff937bdf8c 0000(0000) knlGS:000000000 0000000 enqueue+ 0x150/0x150 fn+0x32/ 0x130 part.0+ 0x180/0x280 handle+ 0x33/0x60 timer+0x3d/ 0x80 softirq+ 0x2a/0x50 0xe1/0x2d6 interrupt+ 0x13b/0x220 timer_interrupt +0x7b/0x140 interrupt+ 0xf/0x20 enter_state+ 0xc5/0x450 34fe38 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 enter_state+ 0xa1/0x450 enter+0x2e/ 0x40 0x23/0x40 entry+0x20/ 0x30 +0x167/ 0x1c0 startup_ 64+0xa4/ 0xb0
[Thu Sep 16 01:43:51 2021] NETDEV WATCHDOG: eno49 (ixgbe): transmit queue 34 timed out
[Thu Sep 16 01:43:51 2021] WARNING: CPU: 11 PID: 0 at net/sched/
[Thu Sep 16 01:43:51 2021] Modules linked in: nf_conntrack_
[Thu Sep 16 01:43:51 2021] ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_mi
[Thu Sep 16 01:43:51 2021] CPU: 11 PID: 0 Comm: swapper/11 Not tainted 5.4.0-42-generic #46-Ubuntu
[Thu Sep 16 01:43:51 2021] Hardware name: HP ProLiant BL460c Gen9, BIOS I36 02/17/2017
[Thu Sep 16 01:43:51 2021] RIP: 0010:dev_
[Thu Sep 16 01:43:51 2021] Code: 85 c0 75 e5 eb 9f 4c 89 ff c6 05 1f f5 e7 00 01 e8 8d bb fa ff 44 89 e9 4c 89 fe 47
[Thu Sep 16 01:43:51 2021] RSP: 0018:ffffb92d46
[Thu Sep 16 01:43:51 2021] RAX: 0000000000000000 RBX: ffff9373ef57cec0 RCX: 000000000000083f
[Thu Sep 16 01:43:51 2021] RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 000000000000083f
[Thu Sep 16 01:43:51 2021] RBP: ffffb92d466b4e60 R08: ffff937bdf8d78c8 R09: 0000000000000004
[Thu Sep 16 01:43:51 2021] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000040
[Thu Sep 16 01:43:51 2021] R13: 0000000000000022 R14: ffff9373ef580480 R15: ffff9373ef580000
[Thu Sep 16 01:43:51 2021] FS: 000000000000000
[Thu Sep 16 01:43:51 2021] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Thu Sep 16 01:43:51 2021] CR2: 00007f3697ec19dc CR3: 000000103ce0a001 CR4: 00000000001626e0
[Thu Sep 16 01:43:51 2021] Call Trace:
[Thu Sep 16 01:43:51 2021] <IRQ>
[Thu Sep 16 01:43:51 2021] ? pfifo_fast_
[Thu Sep 16 01:43:51 2021] call_timer_
[Thu Sep 16 01:43:51 2021] __run_timers.
[Thu Sep 16 01:43:51 2021] ? tick_sched_
[Thu Sep 16 01:43:51 2021] ? tick_sched_
[Thu Sep 16 01:43:51 2021] ? ktime_get+0x3e/0xa0
[Thu Sep 16 01:43:51 2021] run_timer_
[Thu Sep 16 01:43:51 2021] __do_softirq+
[Thu Sep 16 01:43:51 2021] ? hrtimer_
[Thu Sep 16 01:43:51 2021] irq_exit+0xae/0xb0
[Thu Sep 16 01:43:51 2021] smp_apic_
[Thu Sep 16 01:43:51 2021] apic_timer_
[Thu Sep 16 01:43:51 2021] </IRQ>
[Thu Sep 16 01:43:51 2021] RIP: 0010:cpuidle_
[Thu Sep 16 01:43:51 2021] Code: ff e8 bf 08 81 ff 80 7d c7 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 65 03 00 0d
[Thu Sep 16 01:43:51 2021] RSP: 0018:ffffb92d46
[Thu Sep 16 01:43:51 2021] RAX: ffff937bdf8ead00 RBX: ffffffff8d159c00 RCX: 000000000000001f
[Thu Sep 16 01:43:51 2021] RDX: 0000000000000000 RSI: 000000003342629e RDI: 0000000000000000
[Thu Sep 16 01:43:51 2021] RBP: ffffb92d4634fe78 R08: 0011dc129ea2a1f1 R09: 0011dc17357faf00
[Thu Sep 16 01:43:51 2021] R10: ffff937bdf8e9a00 R11: ffff937bdf8e99e0 R12: ffffd9253fac20a8
[Thu Sep 16 01:43:51 2021] R13: 0000000000000004 R14: 0000000000000004 R15: ffffd9253fac20a8
[Thu Sep 16 01:43:51 2021] ? cpuidle_
[Thu Sep 16 01:43:51 2021] cpuidle_
[Thu Sep 16 01:43:51 2021] call_cpuidle+
[Thu Sep 16 01:43:51 2021] do_idle+0x1dd/0x270
[Thu Sep 16 01:43:51 2021] cpu_startup_
[Thu Sep 16 01:43:51 2021] start_secondary
[Thu Sep 16 01:43:51 2021] secondary_
[Thu Sep 16 01:43:51 2021] ---[ end trace cb80e9f61341ace0 ]---
[Thu Sep 16 01:43:51 2021] ixgbe 0000:06:00.0 eno49: initiating reset due to tx timeout
[Thu Sep 16 01:43:51 2021] ixgbe 0000:06:00.0 eno49: Reset adapter
[Thu Sep 16 01:43:51 2021] ixgbe 0000:06:00.0 eno49: TXDCTL.ENABLE for one or more queues not cleared within the pod
[Thu Sep 16 01:43:51 2021] ixgbe 0000:06:00.0 eno49: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[Thu Sep 16 01:43:52 2021] ixgbe 0000:06:00.0 eno49: NIC Link is Down
[Thu Sep 16 01:43:52 2021] bond0: (slave eno49): link status definitely down, disabling slave
[Thu Sep 16 01:43:52 2021] bond0: (slave eno50): making interface the new active one
[Thu Sep 16 01:43:52 2021] device eno49 left promiscuous mode
[Thu Sep 16 01:43:52 2021] device eno50 entered promiscuous mode
[Thu Sep 16 01:43:53 2021] ixgbe 0000:06:00.0 eno49: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[Thu Sep 16 01:43:53 2021] bond0: (slave eno49): link status definitely up, 10000 Mbps full duplex
[Thu Sep 16 01:43:53 2021] bond0: (slave eno49): making interface the new active one
[Thu Sep 16 01:43:53 2021] device eno50 left promiscuous mode
[Thu Sep 16 01:43:53 2021] device eno49 entered promiscuous mode
[Thu Sep 16 01:51:07 2021] ixgbe 0000:06:00.0 eno49: initiating reset due to tx timeout
[Thu Sep 16 01:51:07 2021] ixgbe 0000:06:00.0 eno49: Reset adapter