net:udpgro_frglist.sh in ubuntu_kernel_selftests cause soft lockup on K-5.19 P9

[Po-Hsu Lin]

Issue found while debugging bug 2015745

This net:udpgro_frglist.sh test will cause soft lockup on K-5.19 Power9

The test itself can pass without any issue:
$ sudo ./udpgro_frglist.sh
ipv6
tcp - over veth touching data
-l 4 -6 -D 2001:db8::1 -t rx -4 -t
-4 -t
tcp tx: 525 MB/s 8908 calls/s 8908 msg/s
tcp rx: 525 MB/s 13895 calls/s
tcp tx: 543 MB/s 9223 calls/s 9223 msg/s
tcp rx: 544 MB/s 19994 calls/s
tcp tx: 544 MB/s 9230 calls/s 9230 msg/s
tcp rx: 544 MB/s 26196 calls/s
tcp tx: 566 MB/s 9601 calls/s 9601 msg/s
udp gso - over veth touching data
-l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
-4 -v
udp rx: 0 MB/s 7 calls/s
udp tx: 0 MB/s 1 calls/s 1 msg/s
udp gso and gro - over veth touching data
-l 4 -6 -D 2001:db8::1 -S 0 rx -4 -G
-4 -G
udp rx: 0 MB/s 8 calls/s
udp tx: 0 MB/s 1 calls/s 1 msg/s
$ echo $?
0

But it will cause soft lockup and thus tainting the kernel.

dmesg output:
[ 444.086079] watchdog: BUG: soft lockup - CPU#49 stuck for 26s! [udpgso_bench_tx:7963]
[ 444.086148] Modules linked in: cls_bpf sch_ingress algif_hash af_alg veth cfg80211 joydev input_leds mac_hid binfmt_misc ofpart cmdlinepart powernv_flash opal_prd ipmi_powernv ipmi_devintf ipmi_msghandler ibmpowernv mtd vmx_crypto at24 uio_pdrv_genirq uio dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua pstore_blk ramoops reed_solomon pstore_zone ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid ses enclosure scsi_transport_sas ast i2c_algo_bit drm_vram_helper drm_ttm_helper ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm crct10dif_vpmsum crc32c_vpmsum i40e aacraid xhci_pci drm_panel_orientation_quirks xhci_pci_renesas
[ 444.086281] CPU: 49 PID: 7963 Comm: udpgso_bench_tx Not tainted 5.19.0-38-generic #39-Ubuntu
[ 444.086287] NIP: c0000000012e5d74 LR: c00000000135c628 CTR: 000000001355bc49
[ 444.086291] REGS: c000000fffdfb6d0 TRAP: 0900 Not tainted (5.19.0-38-generic)
[ 444.086294] MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 4822288e XER: 20040153
[ 444.086313] CFAR: 0000000000000000 IRQMASK: 0
               GPR00: c00000000135c628 c000000fffdfb970 c000000002afde00 c00000061443010e
               GPR04: 0000000000000000 0000000000000000 0000000000000000 0016ce21ccbbf672
               GPR08: c0000008dee73c8a 0039766f02a34334 0000000000000000 0000000000000000
               GPR12: 0000000000000000 c000000ffffaa680 0000000000000000 0000000000000000
               GPR16: 0000000000000001 0000000000000000 ffffffffffffffff 0000000000000000
               GPR20: c000000fffdfbdd8 c000000002b33a80 0000000000000000 0000000000000000
               GPR24: 0000000020000000 000000000000000a c0000000059d4c00 0000000000000000
               GPR28: 000000000000000e 0000000000000014 c00000061443010e c0000000059db400
[ 444.086376] NIP [c0000000012e5d74] ip_send_check+0x94/0xf0
[ 444.086385] LR [c00000000135c628] inet_gso_segment+0x278/0x5b0
[ 444.086390] Call Trace:
[ 444.086392] [c000000fffdfb970] [c00000000135c570] inet_gso_segment+0x1c0/0x5b0 (unreliable)
[ 444.086399] [c000000fffdfb9e0] [c00000000121dde8] skb_mac_gso_segment+0xe8/0x180
[ 444.086406] [c000000fffdfba20] [c0000000011bd650] __skb_gso_segment+0xc0/0x1d0
[ 444.086413] [c000000fffdfba60] [c0000000013454bc] udp_queue_rcv_skb.part.0+0x9c/0x280
[ 444.086419] [c000000fffdfbaa0] [c000000001345c68] udp_unicast_rcv_skb+0x38/0x100
[ 444.086425] [c000000fffdfbac0] [c000000001347838] __udp4_lib_rcv+0x388/0x7b0
[ 444.086431] [c000000fffdfbb90] [c0000000012e10a0] ip_protocol_deliver_rcu+0x70/0x410
[ 444.086438] [c000000fffdfbbe0] [c0000000012e14d4] ip_local_deliver_finish+0x94/0x110
[ 444.086445] [c000000fffdfbc20] [c0000000012e0df0] ip_rcv_finish+0x110/0x160
[ 444.086451] [c000000fffdfbc60] [c0000000011cb914] __netif_receive_skb_one_core+0x74/0xd0
[ 444.086458] [c000000fffdfbcb0] [c0000000011cbd1c] process_backlog+0x11c/0x260
[ 444.086464] [c000000fffdfbd00] [c0000000011ccda8] __napi_poll+0x68/0x3a0
[ 444.086471] [c000000fffdfbd90] [c0000000011cd5ac] net_rx_action+0x23c/0x490
[ 444.086477] [c000000fffdfbe80] [c0000000014e922c] __do_softirq+0x16c/0x47c
[ 444.086484] [c000000fffdfbf90] [c0000000000180ec] do_softirq_own_stack+0x4c/0xb0
[ 444.086490] [c00000002499b5f0] [c0000000011dcb54] __neigh_event_send+0x344/0x530
[ 444.086496] [c00000002499b630] [c000000000196624] do_softirq+0xd4/0xe0
[ 444.086503] [c00000002499b660] [c0000000001966f8] __local_bh_enable_ip+0xc8/0x100
[ 444.086509] [c00000002499b680] [c0000000013deddc] ip6_finish_output2+0x24c/0x780
[ 444.086516] [c00000002499b730] [c0000000013e2db8] ip6_finish_output+0x1f8/0x4e0
[ 444.086522] [c00000002499b790] [c000000001469c4c] ip6_local_out+0x6c/0xb0
[ 444.086528] [c00000002499b7d0] [c0000000013e3af8] ip6_send_skb+0x48/0x110
[ 444.086534] [c00000002499b810] [c0000000014218dc] udp_v6_send_skb+0x1dc/0x620
[ 444.086541] [c00000002499b870] [c000000001422a44] udpv6_sendmsg+0xbc4/0x1030
[ 444.086546] [c00000002499ba80] [c0000000013d8df8] inet6_sendmsg+0x68/0xe0
[ 444.086553] [c00000002499bac0] [c00000000117ed10] sock_sendmsg+0x90/0xe0
[ 444.086559] [c00000002499baf0] [c00000000117f21c] ____sys_sendmsg+0x2ec/0x390
[ 444.086566] [c00000002499bb90] [c0000000011840e0] ___sys_sendmsg+0xd0/0x150
[ 444.086571] [c00000002499bd00] [c0000000011841f4] sys_sendmsg+0x94/0x130
[ 444.086577] [c00000002499bdb0] [c000000000031b3c] system_call_exception+0x11c/0x2f0
[ 444.086583] [c00000002499be10] [c00000000000c0e8] system_call_vectored_common+0xe8/0x278
[ 444.086590] --- interrupt: 3000 at 0x717d0396cf98
[ 444.086595] NIP: 0000717d0396cf98 LR: 0000000000000000 CTR: 0000000000000000
[ 444.086598] REGS: c00000002499be80 TRAP: 3000 Not tainted (5.19.0-38-generic)
[ 444.086601] MSR: 900000000000f033 <SF,HV,EE,PR,FP,ME,IR,DR,RI,LE> CR: 44002478 XER: 00000000
[ 444.086619] IRQMASK: 0
               GPR00: 0000000000000155 00007fffd548b480 0000717d03a76e00 0000000000000003
               GPR04: 00007fffd548b590 0000000000000000 0000000000000000 0000006700000011
               GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
               GPR12: 0000000000000000 0000717d03bba7a0 0000000000000000 000000000000f180
               GPR16: 0000000000000003 000000000000000a 00000c39af050040 000001876f2e3cbf
               GPR20: 00007fffd548b648 00000c39aea10060 0000000000000000 0000000000000000
               GPR24: 0000000000000000 000001876f2e38d7 0000000000000003 00000c39aea10010
               GPR28: 00000c39aea10100 00000c39aea10100 00000c39aea10040 00007fffd548b4c0
[ 444.086680] NIP [0000717d0396cf98] 0x717d0396cf98
[ 444.086683] LR [0000000000000000] 0x0
[ 444.086686] --- interrupt: 3000
[ 444.086688] Instruction dump:
[ 444.086691] 394affff 7d292a14 2c2a0001 85480004 7d295214 41820030 78caf082 7d4903a6
[ 444.086703] 80880004 80a80008 80c8000c 85480010 <7d292214> 7ce72a14 7d293214 7ce75214