Activity log for bug #1981658

Date Who What changed Old value New value Message
2022-07-14 03:22:40 Haw Loeung bug added bug
2022-07-14 03:23:14 Haw Loeung bug task added linux (Ubuntu)
2022-07-14 03:23:23 Haw Loeung nominated for series Ubuntu Bionic
2022-07-14 03:23:23 Haw Loeung bug task added linux (Ubuntu Bionic)
2022-07-14 03:23:23 Haw Loeung bug task added linux-hwe-5.4 (Ubuntu Bionic)
2022-07-14 03:23:34 Haw Loeung bug added subscriber The Canonical Sysadmins
2022-07-14 03:23:46 Haw Loeung description Hi, On one of the main US Ubuntu Archive servers, we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122 (maybe LP:1978719?). Hi, On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122 (maybe LP:1978719?).
2022-07-14 03:28:05 Haw Loeung description Hi, On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122 (maybe LP:1978719?). Hi, On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122. | [hloeung@banjo ~]$ lsb_release -a | No LSB modules are available. | Distributor ID: Ubuntu | Description: Ubuntu 18.04.6 LTS | Release: 18.04 | Codename: bionic
2022-07-14 03:30:05 Ubuntu Kernel Bot linux (Ubuntu): status New Incomplete
2022-07-14 03:30:07 Ubuntu Kernel Bot linux (Ubuntu Bionic): status New Incomplete
2022-07-14 03:48:39 Haw Loeung tags apport-collected bionic
2022-07-14 03:48:40 Haw Loeung description Hi, On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122. | [hloeung@banjo ~]$ lsb_release -a | No LSB modules are available. | Distributor ID: Ubuntu | Description: Ubuntu 18.04.6 LTS | Release: 18.04 | Codename: bionic Hi, On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122. | [hloeung@banjo ~]$ lsb_release -a | No LSB modules are available. | Distributor ID: Ubuntu | Description: Ubuntu 18.04.6 LTS | Release: 18.04 | Codename: bionic --- ProblemType: Bug AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Jul 14 03:13 seq crw-rw---- 1 root audio 116, 33 Jul 14 03:13 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu7.28 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: DistroRelease: Ubuntu 18.04 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' MachineType: HPE ProLiant DL385 Gen10 Package: linux-hwe-5.4 PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.utf8 SHELL=/bin/bash ProcFB: 0 mgag200drmfb ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.4.0-121-generic root=UUID=a5a2675d-dc52-48f0-9273-2c6dadac446f ro console=ttyS0,115200 nosplash ProcVersionSignature: Ubuntu 5.4.0-121.137~18.04.1-generic 5.4.189 RelatedPackageVersions: linux-restricted-modules-5.4.0-121-generic N/A linux-backports-modules-5.4.0-121-generic N/A linux-firmware 1.173.21 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' Tags: bionic Uname: Linux 5.4.0-121-generic x86_64 UpgradeStatus: Upgraded to bionic on 2019-12-16 (940 days ago) UserGroups: adm _MarkForUpload: True dmi.bios.date: 02/10/2022 dmi.bios.vendor: HPE dmi.bios.version: A40 dmi.board.name: ProLiant DL385 Gen10 dmi.board.vendor: HPE dmi.chassis.type: 23 dmi.chassis.vendor: HPE dmi.modalias: dmi:bvnHPE:bvrA40:bd02/10/2022:svnHPE:pnProLiantDL385Gen10:pvr:rvnHPE:rnProLiantDL385Gen10:rvr:cvnHPE:ct23:cvr: dmi.product.family: ProLiant dmi.product.name: ProLiant DL385 Gen10 dmi.product.sku: 878615-B21 dmi.sys.vendor: HPE
2022-07-14 03:48:41 Haw Loeung attachment added CRDA.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603024/+files/CRDA.txt
2022-07-14 03:48:42 Haw Loeung attachment added CurrentDmesg.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603025/+files/CurrentDmesg.txt
2022-07-14 03:48:44 Haw Loeung attachment added Lspci.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603026/+files/Lspci.txt
2022-07-14 03:48:45 Haw Loeung attachment added Lsusb.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603027/+files/Lsusb.txt
2022-07-14 03:48:47 Haw Loeung attachment added ProcCpuinfo.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603028/+files/ProcCpuinfo.txt
2022-07-14 03:48:48 Haw Loeung attachment added ProcCpuinfoMinimal.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603029/+files/ProcCpuinfoMinimal.txt
2022-07-14 03:48:49 Haw Loeung attachment added ProcInterrupts.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603030/+files/ProcInterrupts.txt
2022-07-14 03:48:51 Haw Loeung attachment added ProcModules.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603031/+files/ProcModules.txt
2022-07-14 03:48:52 Haw Loeung attachment added UdevDb.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603032/+files/UdevDb.txt
2022-07-14 03:48:54 Haw Loeung attachment added WifiSyslog.txt https://bugs.launchpad.net/bugs/1981658/+attachment/5603033/+files/WifiSyslog.txt
2022-07-14 03:50:22 Haw Loeung linux (Ubuntu): status Incomplete Confirmed
2022-07-14 03:50:26 Haw Loeung linux (Ubuntu Bionic): status Incomplete Confirmed
2022-07-14 03:50:32 Haw Loeung linux-hwe-5.4 (Ubuntu): status New Confirmed
2022-07-14 03:50:35 Haw Loeung linux-hwe-5.4 (Ubuntu Bionic): status New Confirmed
2022-07-25 02:19:15 susew bug added subscriber susew
2022-07-25 06:55:15 Tom Reynolds bug added subscriber Tom Reynolds
2022-07-25 17:20:17 Dexuan Cui bug added subscriber Dexuan Cui
2022-07-28 09:24:19 Druggo Yang bug added subscriber Druggo Yang
2022-07-29 18:41:09 Vladimir Grevtsev bug added subscriber Vladimir Grevtsev
2022-08-01 11:42:13 Stefan Andres bug added subscriber Stefan Andres
2022-08-02 13:24:44 Tim Gardner linux (Ubuntu Bionic): status Confirmed In Progress
2022-08-05 10:35:31 Stéphane Bidoul (Acsone) bug added subscriber Stéphane Bidoul (Acsone)
2022-08-05 14:44:59 Samuel Matildes bug added subscriber Samuel Matildes
2022-08-09 20:56:24 Launchpad Janitor linux (Ubuntu Bionic): status In Progress Fix Released
2022-08-09 20:56:24 Launchpad Janitor cve linked 2022-1652
2022-08-09 20:56:24 Launchpad Janitor cve linked 2022-1679
2022-08-09 20:56:24 Launchpad Janitor cve linked 2022-1734
2022-08-09 20:56:24 Launchpad Janitor cve linked 2022-2586
2022-08-09 20:56:24 Launchpad Janitor cve linked 2022-2588
2022-08-09 20:56:24 Launchpad Janitor cve linked 2022-34918
2022-08-10 13:51:45 Junien F bug added subscriber Junien Fridrick
2022-10-27 09:24:57 Jorge Rodríguez bug added subscriber Jorge Rodríguez
2023-09-11 10:53:01 Hassan El Jacifi bug task added linux-hwe-6.2 (Ubuntu)
2023-09-14 12:25:34 Alex Ball bug added subscriber Alex Ball
2023-09-27 09:24:22 Marin Gilles attachment added journalctl -k https://bugs.launchpad.net/ubuntu/+source/linux-hwe-6.2/+bug/1981658/+attachment/5704875/+files/dmesg