network protocol appear call trace (rawv6_getsockopt->ipv6_getsockopt...))
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
kunpeng920 |
Won't Fix
|
Undecided
|
Unassigned | ||
Ubuntu-18.04 |
Won't Fix
|
Undecided
|
Unassigned | ||
Ubuntu-18.04-hwe |
Won't Fix
|
Undecided
|
Unassigned | ||
Ubuntu-19.04 |
Won't Fix
|
Undecided
|
Unassigned | ||
Ubuntu-19.10 |
Won't Fix
|
Undecided
|
Unassigned | ||
Ubuntu-20.04 |
Won't Fix
|
Undecided
|
Unassigned | ||
Upstream-kernel |
Invalid
|
Undecided
|
Unassigned |
Bug Description
[Bug Description]
ubuntu 18.04.3 appear network protocal call trace and result in one cpu stuck (watchdog: BUG: soft lockup - CPU#4 stuck for 22s ).
[Steps to Reproduce]
1) taishan 2280V2 client <==> taishan 2280V2 server
2)modprobe ib_umad
modprobe hns-roce-hw-v2
3) server : ib_atomic_bw -d hns_0 -A CMP_AND_SWAP -e
client:
4)server operation:
mount -t debugfs none /mnt ;echo 7 > /proc/sys/
echo '1' > /sys/class/
echo help > /mnt/hns3/
ethtool -s ethx msglvl 0x0
ethtool -s ethX msglvl probe on
ping client
5) seRver : ib_send_bw -d hnsX -x 0 -c RC --use-srq -R
client: ib_send_bw -d hnsX -x 0 -c RC --use-srq -R --run_infinitely
6) server: apt install netperf
ethtool -N ethx rx-flow-hash udp4 sdfn
watch -d -n 1 "ethtool -S ethx | grep rxq|grep pktnum"
ethtool -N ethx rx-flow-hash udp4 d
ethtool -n ethx rx-flow-hash udp4
client (3 repeat): netperf -H IPxxx -t UDP_STREAM -l 10 -- -m 10240
7) server: ib_send_bw -d hns_0 -n 5 -s1024 -S 3 -x 3
client :ib_send_bw -d hns_0 -n 5 -s1024 -S 3 -x 3 serverip
8) ufw disable
[Actual Results]
Oct 15 11:46:08 ubuntu kernel: [108358.278824] watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [ip6tables:41601]
Oct 15 11:46:08 ubuntu kernel: [108358.286208] Modules linked in: binfmt_misc sctp libcrc32c iptable_filter bpfilter ip6table_filter ip6_tables 8021q garp mrp stp llc ipmi_ssif aes_ce_blk crypto_simd nls_iso8859_1 cryptd aes_ce_cipher rpcrdma sunrpc rdma_ucm crct10dif_ce ib_iser ghash_ce rdma_cm iw_cm aes_arm64 ib_umad libiscsi ib_ipoib sha2_ce scsi_transport_
Oct 15 11:46:08 ubuntu kernel: [108358.286253] CPU: 4 PID: 41601 Comm: ip6tables Not tainted 5.0.0-23-generic #24~18.04.1-Ubuntu
Oct 15 11:46:08 ubuntu kernel: [108358.286254] Hardware name: Huawei TaiShan 2280 V2/BC82AMDA, BIOS TA BIOS 2280-A CS V2.B070.01 09/04/2019
Oct 15 11:46:08 ubuntu kernel: [108358.286255] pstate: 20400009 (nzCv daif +PAN -UAO)
Oct 15 11:46:08 ubuntu kernel: [108358.286262] pc : alloc_counters.
Oct 15 11:46:08 ubuntu kernel: [108358.286264] lr : alloc_counters.
Oct 15 11:46:08 ubuntu kernel: [108358.286265] sp : ffff000023ff3be0
Oct 15 11:46:08 ubuntu kernel: [108358.286265] x29: ffff000023ff3be0 x28: ffffdd595f213040
Oct 15 11:46:08 ubuntu kernel: [108358.286267] x27: ffff000020b49000 x26: 0000000000000000
Oct 15 11:46:08 ubuntu kernel: [108358.286268] x25: ffffdd593f8eff58 x24: 0000000000000000
Oct 15 11:46:08 ubuntu kernel: [108358.286270] x23: ffff3de097bcc738 x22: 0000000000000000
Oct 15 11:46:08 ubuntu kernel: [108358.286271] x21: ffff3de097bcd3b4 x20: ffffdd595f213000
Oct 15 11:46:08 ubuntu kernel: [108358.286272] x19: ffffdd595f213040 x18: 0000000000000000
Oct 15 11:46:08 ubuntu kernel: [108358.286273] x17: 0000000000000000 x16: ffff3de09711af48
Oct 15 11:46:08 ubuntu kernel: [108358.286274] x15: 0000000000000000 x14: ffffa2e540000000
Oct 15 11:46:08 ubuntu kernel: [108358.286275] x13: 0140000000000000 x12: 0040000000000041
Oct 15 11:46:08 ubuntu kernel: [108358.286277] x11: ffff000060b49000 x10: ffff3de097c1e1c8
Oct 15 11:46:08 ubuntu kernel: [108358.286278] x9 : 0000000000000000 x8 : 0000000000000001
Oct 15 11:46:08 ubuntu kernel: [108358.286279] x7 : 0040000000000001 x6 : 0000000000001000
Oct 15 11:46:08 ubuntu kernel: [108358.286280] x5 : ffff000020b49000 x4 : ffff3ddf977f5f58
Oct 15 11:46:08 ubuntu kernel: [108358.286281] x3 : 000000005f656369 x2 : 00009f79a80fa000
Oct 15 11:46:08 ubuntu kernel: [108358.286282] x1 : 00009f79a80fa000 x0 : ffff5ddf7f740000
Oct 15 11:46:08 ubuntu kernel: [108358.286284] Call trace:
Oct 15 11:46:08 ubuntu kernel: [108358.286286] alloc_counters.
Oct 15 11:46:08 ubuntu kernel: [108358.286288] do_ip6t_
Oct 15 11:46:08 ubuntu kernel: [108358.286292] nf_getsockopt+
Oct 15 11:46:08 ubuntu kernel: [108358.286294] ipv6_getsockopt
Oct 15 11:46:08 ubuntu kernel: [108358.286296] rawv6_getsockop
Oct 15 11:46:08 ubuntu kernel: [108358.286298] sock_common_
Oct 15 11:46:08 ubuntu kernel: [108358.286299] __arm64_
Oct 15 11:46:08 ubuntu kernel: [108358.286302] el0_svc_
Oct 15 11:46:08 ubuntu kernel: [108358.286303] el0_svc_
Oct 15 11:46:08 ubuntu kernel: [108358.286305] el0_svc+0x8/0xc
Oct 15 11:46:41 ubuntu kernel: [108390.854822] rcu: INFO: rcu_sched self-detected stall on CPU
Oct 15 11:46:41 ubuntu kernel: [108390.860471] rcu: 4-....: (14998 ticks this GP) idle=672/
Oct 15 11:46:41 ubuntu kernel: [108390.870454] rcu: (t=15003 jiffies g=3365597 q=977)
Oct 15 11:46:41 ubuntu kernel: [108390.875499] Task dump for CPU 4:
Oct 15 11:46:41 ubuntu kernel: [108390.875500] ip6tables R running task 0 41601 41600 0x00000006
Oct 15 11:46:41 ubuntu kernel: [108390.875503] Call trace:
Oct 15 11:46:41 ubuntu kernel: [108390.875506] dump_backtrace+
Oct 15 11:46:41 ubuntu kernel: [108390.875507] show_stack+
Oct 15 11:46:41 ubuntu kernel: [108390.875509] sched_show_
Oct 15 11:46:41 ubuntu kernel: [108390.875511] dump_cpu_
Oct 15 11:46:41 ubuntu kernel: [108390.875513] rcu_dump_
Oct 15 11:46:41 ubuntu kernel: [108390.875514] rcu_check_
Oct 15 11:46:41 ubuntu kernel: [108390.875515] update_
Oct 15 11:46:41 ubuntu kernel: [108390.875517] tick_sched_
Oct 15 11:46:41 ubuntu kernel: [108390.875518] tick_sched_
Oct 15 11:46:41 ubuntu kernel: [108390.875520] __hrtimer_
Oct 15 11:46:41 ubuntu kernel: [108390.875522] hrtimer_
Oct 15 11:46:41 ubuntu kernel: [108390.875524] arch_timer_
Oct 15 11:46:41 ubuntu kernel: [108390.875525] handle_
Oct 15 11:46:41 ubuntu kernel: [108390.875527] generic_
Oct 15 11:46:41 ubuntu kernel: [108390.875529] __handle_
Oct 15 11:46:41 ubuntu kernel: [108390.875530] gic_handle_
Oct 15 11:46:41 ubuntu kernel: [108390.875531] el1_irq+0xe8/0x1c0
Oct 15 11:46:41 ubuntu kernel: [108390.875533] alloc_counters.
Oct 15 11:46:41 ubuntu kernel: [108390.875535] do_ip6t_
Oct 15 11:46:41 ubuntu kernel: [108390.875537] nf_getsockopt+
Oct 15 11:46:41 ubuntu kernel: [108390.875538] ipv6_getsockopt
Oct 15 11:46:41 ubuntu kernel: [108390.875539] rawv6_getsockop
Oct 15 11:46:41 ubuntu kernel: [108390.875542] sock_common_
Oct 15 11:46:41 ubuntu kernel: [108390.875543] __arm64_
Oct 15 11:46:41 ubuntu kernel: [108390.875545] el0_svc_
Oct 15 11:46:41 ubuntu kernel: [108390.875547] el0_svc_
Oct 15 11:46:41 ubuntu kernel: [108390.875548] el0_svc+0x8/0xc
Oct 15 11:47:08 ubuntu kernel: [108418.278822] watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [ip6tables:41601]
Oct 15 11:47:08 ubuntu kernel: [108418.286202] Modules linked in: binfmt_misc sctp libcrc32c iptable_filter bpfilter ip6table_filter ip6_tables 8021q garp mrp stp llc ipmi_ssif aes_ce_blk crypto_simd nls_iso8859_1 cryptd aes_ce_cipher rpcrdma sunrpc rdma_ucm crct10dif_ce ib_iser ghash_ce rdma_cm iw_cm aes_arm64 ib_umad libiscsi ib_ipoib sha2_ce scsi_transport_
Oct 15 11:47:08 ubuntu kernel: [108418.286230] CPU: 4 PID: 41601 Comm: ip6tables Tainted: G L 5.0.0-23-generic #24~18.04.1-Ubuntu
Oct 15 11:47:08 ubuntu kernel: [108418.286231] Hardware name: Huawei TaiShan 2280 V2/BC82AMDA, BIOS TA BIOS 2280-A CS V2.B070.01 09/04/2019
Oct 15 11:47:08 ubuntu kernel: [108418.286232] pstate: 20400009 (nzCv daif +PAN -UAO)
Oct 15 11:47:08 ubuntu kernel: [108418.286234] pc : alloc_counters.
Oct 15 11:47:08 ubuntu kernel: [108418.286236] lr : alloc_counters.
Oct 15 11:47:08 ubuntu kernel: [108418.286237] sp : ffff000023ff3be0
Oct 15 11:47:08 ubuntu kernel: [108418.286237] x29: ffff000023ff3be0 x28: ffffdd595f213040
Oct 15 11:47:08 ubuntu kernel: [108418.286239] x27: ffff000020b49000 x26: 0000000000000000
Oct 15 11:47:08 ubuntu kernel: [108418.286241] x25: ffffdd593f8eff58 x24: 0000000000000000
Oct 15 11:47:08 ubuntu kernel: [108418.286242] x23: ffff3de097bcc738 x22: 0000000000000000
Oct 15 11:47:08 ubuntu kernel: [108418.286243] x21: ffff3de097bcd3b4 x20: ffffdd595f213000
Oct 15 11:47:08 ubuntu kernel: [108418.286244] x19: ffffdd595f213040 x18: 0000000000000000
Oct 15 11:47:08 ubuntu kernel: [108418.286246] x17: 0000000000000000 x16: ffff3de09711af48
Oct 15 11:47:08 ubuntu kernel: [108418.286247] x15: 0000000000000000 x14: ffffa2e540000000
Oct 15 11:47:08 ubuntu kernel: [108418.286248] x13: 0140000000000000 x12: 0040000000000041
Oct 15 11:47:08 ubuntu kernel: [108418.286249] x11: ffff000060b49000 x10: ffff3de097c1e1c8
Oct 15 11:47:08 ubuntu kernel: [108418.286250] x9 : 0000000000000000 x8 : 0000000000000001
Oct 15 11:47:08 ubuntu kernel: [108418.286252] x7 : 0040000000000001 x6 : 0000000000001000
Oct 15 11:47:08 ubuntu kernel: [108418.286253] x5 : ffff000020b49000 x4 : ffff3ddf977f5f58
Oct 15 11:47:08 ubuntu kernel: [108418.286254] x3 : 000000005f656369 x2 : 00009f79a80fa000
Oct 15 11:47:08 ubuntu kernel: [108418.286255] x1 : 00009f79a80fa000 x0 : ffff5ddf7f740000
Oct 15 11:47:08 ubuntu kernel: [108418.286256] Call trace:
Oct 15 11:47:08 ubuntu kernel: [108418.286258] alloc_counters.
Oct 15 11:47:08 ubuntu kernel: [108418.286260] do_ip6t_
Oct 15 11:47:08 ubuntu kernel: [108418.286261] nf_getsockopt+
Oct 15 11:47:08 ubuntu kernel: [108418.286263] ipv6_getsockopt
Oct 15 11:47:08 ubuntu kernel: [108418.286264] rawv6_getsockop
Oct 15 11:47:08 ubuntu kernel: [108418.286265] sock_common_
Oct 15 11:47:08 ubuntu kernel: [108418.286266] __arm64_
Oct 15 11:47:08 ubuntu kernel: [108418.286268] el0_svc_
Oct 15 11:47:08 ubuntu kernel: [108418.286269] el0_svc_
Oct 15 11:47:08 ubuntu kernel: [108418.286270] el0_svc+0x8/0xc
[Expected Results]
no calltrace
[Reproducibility]
low probability
[Additional information]
(Firmware version, kernel version, affected hardware, etc. if required):
[Resolution]
summary: |
- network protocal appear call trace + network protocol appear call trace (rawv6_getsockopt->ipv6_getsockopt...)) |
tags: | added: ikeradar |
tags: | removed: ikeradar |
I am a little confused by the reproduction steps. Are all of these commands necessary? Should it 100% reproduce the problem? I attempted to run these steps as recorded in the attached file, but I was not able to reproduce. Please check the attached file. I also confirmed I am running the 5.0.0-23 kernel on both sides.