There is not much delta between -181 and -182, and I successfully triggered this on 5.4.0-181 + 181 source code. So the bisect result in #1 is invalid.
As this test fail rate is not 100% reproducible, I am going to run it for multiple times.
I can reproduce this all the way back to 5.4.0-170-generic.
There is not much delta between -181 and -182, and I successfully triggered this on 5.4.0-181 + 181 source code. So the bisect result in #1 is invalid.
As this test fail rate is not 100% reproducible, I am going to run it for multiple times.
I can reproduce this all the way back to 5.4.0-170-generic.
And even 5.4.0-150 is failing too, the trace is a bit different though: 2530-000/ DGXA100, BIOS 1.25 08/31/2023 event_raw_ event_wbt_ timer+0x6f/ 0x100 ad8da0 EFLAGS: 00010286 0(0000) GS:ffff8eaf7fbc 0000(0000) knlGS:000000000 0000000 fn+0x1d6/ 0x3c0 free_callback_ rcu+0x30/ 0x30 timer_fn+ 0x134/0x140 fn+0x32/ 0x130 part.0+ 0x180/0x280 raw_event_ softirq+ 0x5d/0xa0 softirq+ 0x2a/0x50 0xd1/0x2c1 timer_interrupt +0x7b/0x140 interrupt+ 0xf/0x20 safe_halt+ 0xe/0x10 6ebe70 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff13 idle+0x20/ 0x140 idle+0x15/ 0x20 idle_call+ 0x23/0x30 entry+0x20/ 0x30 +0x173/ 0x1d0 startup_ 64+0xa4/ 0xb0 event_raw_ event_wbt_ timer+0x6f/ 0x100 ad8da0 EFLAGS: 00010286 0(0000) GS:ffff8eaf7fbc 0000(0000) knlGS:000000000 0000000 000-0xffffffffb fffffff)
[ 3706.368639] BUG: kernel NULL pointer dereference, address: 0000000000000050
[ 3706.376413] #PF: supervisor read access in kernel mode
[ 3706.382146] #PF: error_code(0x0000) - not-present page
[ 3706.387878] PGD 0 P4D 0
[ 3706.390704] Oops: 0000 [#1] SMP NOPTI
[ 3706.394789] CPU: 255 PID: 0 Comm: swapper/255 Tainted: G OE 5.4.0-150-generic #167-Ubuntu
[ 3706.405274] Hardware name: NVIDIA DGXA100 920-23687-
[ 3706.414504] RIP: 0010:trace_
[ 3706.421206] Code: 59 80 e5 02 0f 85 8f 00 00 00 4c 89 e6 ba 34 00 00 00 48 8d 7d a0 e8 50 cb c9 ff 49 89 c4 48 85 c0 74 37 49 8b 87 b8 03 00 00 <48> 8b 70 50 48 85 f6 74 45 49 8d 7c 24 08 ba 20 00 00 00 e8 99 4b
[ 3706.442162] RSP: 0018:ffffa29e5c
[ 3706.447993] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000080000100
[ 3706.455956] RDX: ffff8eaf36ab00ac RSI: 0000000000000100 RDI: ffff8eaf36ab00a8
[ 3706.463918] RBP: ffffa29e5cad8e08 R08: ffff8eaf36ab00a8 R09: 0000000000000100
[ 3706.471878] R10: 0000000000000004 R11: 0000000000000004 R12: ffff8eaf36ab00ac
[ 3706.479840] R13: 0000000000000000 R14: 0000000000000003 R15: ffff8e0fc0048800
[ 3706.487804] FS: 000000000000000
[ 3706.496834] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3706.503243] CR2: 0000000000000050 CR3: 000000bd7e40a000 CR4: 0000000000340ee0
[ 3706.511204] Call Trace:
[ 3706.513932] <IRQ>
[ 3706.516177] wb_timer_
[ 3706.520169] ? blk_stat_
[ 3706.525610] blk_stat_
[ 3706.530183] call_timer_
[ 3706.534261] __run_timers.
[ 3706.539026] ? trace_event_
[ 3706.544759] run_timer_
[ 3706.549137] __do_softirq+
[ 3706.553125] irq_exit+0xae/0xb0
[ 3706.556629] smp_apic_
[ 3706.561780] apic_timer_
[ 3706.566348] </IRQ>
[ 3706.568689] RIP: 0010:native_
[ 3706.573741] Code: 7b ff ff ff eb bd 90 90 90 90 90 90 e9 07 00 00 00 0f 00 2d f6 14 51 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d e6 14 51 00 fb f4 <c3> 90 0f 1f 44 00 00 55 48 89 e5 41 55 41 54 53 e8 1d 3b 62 ff 65
[ 3706.594697] RSP: 0018:ffffa29e59
[ 3706.603145] RAX: 0000000000023800 RBX: ffff8e2f5d0a3768 RCX: 00000000000416f2
[ 3706.611108] RDX: 00000000000416f2 RSI: 0000000000000000 RDI: ffffffffb1ec5f60
[ 3706.619070] RBP: ffffa29e596ebe90 R08: 0000000000000000 R09: 0000000000000000
[ 3706.627031] R10: 0000000000000000 R11: 0000000000000004 R12: 00000000000000ff
[ 3706.634993] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 3706.642957] ? default_
[ 3706.647140] arch_cpu_
[ 3706.651127] default_
[ 3706.655506] do_idle+0x1fb/0x270
[ 3706.659108] ? complete+0x49/0x50
[ 3706.662805] cpu_startup_
[ 3706.667181] start_secondary
[ 3706.671560] secondary_
[ 3706.676217] Modules linked in: nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua kvm_amd kvm ipmi_ssif input_leds binfmt_misc mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) ccp ipmi_si ipmi_devintf ipmi_msghandler sch_fq_codel msr efi_pstore ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic crct10dif_pclmul ast crc32_pclmul drm_vram_helper ghash_clmulni_intel ttm mlx5_core(OE) aesni_intel pci_hyperv_intf crypto_simd drm_kms_helper mlxdevm(OE) cryptd glue_helper syscopyarea auxiliary(OE) sysfillrect mpt3sas igb sysimgblt mlx_compat(OE) tls raid_class fb_sys_fops dca uas usbhid i2c_algo_bit scsi_transport_sas usb_storage hid drm mlxfw(OE) nvme psample nvme_core
[ 3706.755113] CR2: 0000000000000050
[ 3706.758930] ---[ end trace d6371d7677eb0b6d ]---
[ 3706.863060] RIP: 0010:trace_
[ 3706.869763] Code: 59 80 e5 02 0f 85 8f 00 00 00 4c 89 e6 ba 34 00 00 00 48 8d 7d a0 e8 50 cb c9 ff 49 89 c4 48 85 c0 74 37 49 8b 87 b8 03 00 00 <48> 8b 70 50 48 85 f6 74 45 49 8d 7c 24 08 ba 20 00 00 00 e8 99 4b
[ 3706.890717] RSP: 0018:ffffa29e5c
[ 3706.896545] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000080000100
[ 3706.904505] RDX: ffff8eaf36ab00ac RSI: 0000000000000100 RDI: ffff8eaf36ab00a8
[ 3706.912466] RBP: ffffa29e5cad8e08 R08: ffff8eaf36ab00a8 R09: 0000000000000100
[ 3706.920428] R10: 0000000000000004 R11: 0000000000000004 R12: ffff8eaf36ab00ac
[ 3706.928388] R13: 0000000000000000 R14: 0000000000000003 R15: ffff8e0fc0048800
[ 3706.936352] FS: 000000000000000
[ 3706.945380] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3706.951789] CR2: 0000000000000050 CR3: 000000bd7e40a000 CR4: 0000000000340ee0
[ 3706.959751] Kernel panic - not syncing: Fatal exception in interrupt
[ 3706.967412] Kernel Offset: 0x2f400000 from 0xffffffff81000000 (relocation range: 0xffffffff80000
[ 3707.430969] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---