tls in ubuntu_kselftests_net cause oops on J-xilinx ZCU boards

[Po-Hsu Lin]

Issue found on xilinx-ZCU104 and xilinx-ZCU106 with 5.15.0-1029-xilinx-zynqmp

$ sudo ./tls
TAP version 13
1..179
# Starting 179 tests from 6 test cases.
# RUN global.non_established ...
# OK global.non_established
ok 1 global.non_established
# RUN global.keysizes ...
# OK global.keysizes
ok 2 global.keysizes
# RUN tls_basic.base_base ...
# OK tls_basic.base_base
ok 3 tls_basic.base_base
# RUN tls.12_gcm.sendfile ...
# sendfile: Test terminated unexpectedly by signal 11
# FAIL tls.12_gcm.sendfile
not ok 4 tls.12_gcm.sendfile
# RUN tls.12_gcm.send_then_sendfile ...
(I hit Ctrl-c here)

[ 355.715691] ------------[ cut here ]------------
[ 355.715691] Unable to handle kernel read from unreadable memory at virtual address 0000000000000000
[ 355.715703] Mem abort info:
[ 355.715705] ESR = 0x0000000096000004
[ 355.715708] EC = 0x25: DABT (current EL), IL = 32 bits
[ 355.724739] WARNING: CPU: 3 PID: 2391 at include/linux/skmsg.h:138 __sk_msg_free+0xa8/0x100
[ 355.724758] Modules linked in: tls binfmt_misc ina2xx_adc
[ 355.727553] SET = 0, FnV = 0
[ 355.731276] xilinx_can can_dev mali(C) uio_pdrv_genirq sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua dmaproxy(C) pstore_blk ramoops reed_solomon
[ 355.736613] EA = 0, S1PTW = 0
[ 355.739644] pstore_zone efi_pstore ip_tables x_tables autofs4 raid10 raid456 async_raid6_recov async_memcpy
[ 355.742791] FSC = 0x04: level 0 translation fault
[ 355.742796] async_pq async_xor async_tx raid1 raid0 multipath
[ 355.747666] Data abort info:
[ 355.747668] ISV = 0, ISS = 0x00000004
[ 355.750530] linear i2c_mux_pca954x crct10dif_ce rtc_zynqmp spi_zynqmp_gqspi i2c_cadence
[ 355.754358] CM = 0, WnR = 0
[ 355.754361] zynqmp_dpsub ahci_ceva aes_neon_blk crypto_simd cryptd
[ 355.757320] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000802b55000
[ 355.757325] CPU: 3 PID: 2391 Comm: tls Tainted: G C 5.15.0-1029-xilinx-zynqmp #33-Ubuntu
[ 355.763751] Hardware name: ZynqMP ZCU106 RevA (DT)
[ 355.763756] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 355.763760] [0000000000000000] pgd=0000000000000000, p4d=0000000000000000
[ 355.763764] pc : __sk_msg_free+0xa8/0x100
[ 355.770540] lr : __sk_msg_free+0x78/0x100
[ 355.770546] sp : ffff80000cec3c30

[ 355.770549] x29: ffff80000cec3c30 x28: ffff000809de8000 x27: 0000000000000000
[ 355.770556] Internal error: Oops: 0000000096000004 [#1] SMP

[ 355.777676] x26: 0000000000000000
[ 355.783232] Modules linked in:
[ 355.784706] x25: dead000000000100
[ 355.788005] tls
[ 355.791043] x24: dead000000000122
[ 355.794429] binfmt_misc

[ 355.799637] ina2xx_adc
[ 355.802154] x23: 0000000000004010
[ 355.803630] xilinx_can
[ 355.806061] x22: ffff0008038622e0
[ 355.809360] can_dev
[ 355.811790] x21: 0000000000000001
[ 355.815176] mali(C)

[ 355.820731] uio_pdrv_genirq
[ 355.822902] x20: ffff00080bb1bd40
[ 355.824377] sch_fq_codel
[ 355.827242] x19: 0000000000000001
[ 355.830541] dm_multipath
[ 355.833145] x18: 0000000000000000
[ 355.836531] scsi_dh_rdac

[ 355.842520] scsi_dh_emc
[ 355.845124] x17: 0000000000000000
[ 355.846601] scsi_dh_alua
[ 355.849118] x16: 0000000000000000
[ 355.852417] dmaproxy(C)
[ 355.855021] x15: 0000000000000000
[ 355.858406] pstore_blk

[ 355.864310] ramoops
[ 355.866740] x14: 0000000000000000
[ 355.868216] reed_solomon
[ 355.870386] x13: 0000000000000000
[ 355.873685] pstore_zone
[ 355.876289] x12: ffff8000099d4e78
[ 355.879675] efi_pstore

[ 355.885577] ip_tables
[ 355.888008] x11: ffff80000aa8dad0
[ 355.889485] x_tables
[ 355.891828] x10: ffff80000cec3b80
[ 355.895127] autofs4
[ 355.897383] x9 : ffff00087f79cfa0
[ 355.900769] raid10

[ 355.906325] raid456
[ 355.908408] x8 : fffffc00200e1a08
[ 355.909884] async_raid6_recov
[ 355.912054] x7 : fffffc00200e1c08
[ 355.915353] async_memcpy
[ 355.918391] x6 : fffffc00200e1c00
[ 355.921777] async_pq

[ 355.927767] async_xor
[ 355.930023] x5 : ffff800875238000
[ 355.931499] async_tx
[ 355.933843] x4 : 000000000000000d
[ 355.937142] raid1
[ 355.939399] x3 : 0000000000000040
[ 355.942785] raid0

[ 355.948167] multipath
[ 355.950163] x2 : ffff00087f79cf00
[ 355.951639] linear
[ 355.953983] x1 : 0000000000000704
[ 355.957282] i2c_mux_pca954x
[ 355.959365] x0 : 0000000000000001
[ 355.962750] crct10dif_ce

[ 355.969001] rtc_zynqmp
[ 355.971606] Call trace:
[ 355.973081] spi_zynqmp_gqspi
[ 355.975513] __sk_msg_free+0xa8/0x100
[ 355.977942] i2c_cadence
[ 355.980894] sk_msg_free+0x20/0x30
[ 355.984539] zynqmp_dpsub
[ 355.987057] tls_sw_release_resources_tx+0x14c/0x230 [tls]
[ 355.990442] ahci_ceva
[ 355.993047] tls_sk_proto_cleanup+0x80/0x184 [tls]
[ 355.998516] aes_neon_blk
[ 356.000859] tls_sk_proto_close+0x98/0x260 [tls]
[ 356.005634] crypto_simd
[ 356.008238] inet_release+0x58/0xac
[ 356.012839] cryptd
[ 356.015357] __sock_release+0x4c/0xcc

[ 356.020912] sock_close+0x20/0x30
[ 356.024560] CPU: 2 PID: 142 Comm: firmware:zynqmp Tainted: G C 5.15.0-1029-xilinx-zynqmp #33-Ubuntu
[ 356.026034] __fput+0x88/0x280
[ 356.029333] Hardware name: ZynqMP ZCU106 RevA (DT)
[ 356.039663] ____fput+0x18/0x24
[ 356.042702] pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 356.047476] task_work_run+0xc8/0x170
[ 356.050601] pc : flush_dcache_page+0x8/0x60
[ 356.057545] do_notify_resume+0x1e8/0x210
[ 356.061192] lr : skcipher_walk_done+0xcc/0x2c0
[ 356.065358] el0_svc+0x4c/0xb0
[ 356.069351] sp : ffff80000b7dbba0
[ 356.073779] el0t_64_sync_handler+0xa4/0x12c
[ 356.076817] x29: ffff80000b7dbba0
[ 356.080116] el0t_64_sync+0x1a4/0x1a8
[ 356.084369] x28: ffff000800c45cc0
[ 356.087669] ---[ end trace 73e93e8d345dd260 ]---
[ 356.091314] x27: 0000000000000010
[ 356.094838] Unable to handle kernel access to user memory outside uaccess routines at virtual address 0000000000000008

[ 356.099303] x26: ffff80000885df50 x25: ffff000802c0b400 x24: 0000000000000010
[ 356.099313] x23: ffff000803862780 x22: 0000000000000ed0 x21: 0000000000003130
[ 356.099323] x20: 0000000000000001 x19: ffff80000b7dbc30 x18: 00000000000c0000
[ 356.102740] Mem abort info:
[ 356.113398] x17: 0000000000560000 x16: 00000000000003fb x15: 0000000011e2923e
[ 356.113408] x14: 0000000000000171 x13: 0000000050000000 x12: 00000000000003f2
[ 356.113418] x11: 00000000000002e0 x10: 0000000089000000
[ 356.114910] ESR = 0x0000000096000004
[ 356.122018] x9 : ffff80000884d49c
[ 356.122023] x8 : 00000000cb5befb4 x7 : 0000000000000010 x6 : 0000000000000010
[ 356.122032] x5 : 07705642c007a4b9
[ 356.129162] EC = 0x25: DABT (current EL), IL = 32 bits
[ 356.136272] x4 : 0000000000001000 x3 : 0000000000000000
[ 356.136279] x2 : 0000000000000000 x1 : ffff000803862300 x0 : 0000000000000000
[ 356.139066] SET = 0, FnV = 0

[ 356.146180] Call trace:
[ 356.146184] flush_dcache_page+0x8/0x60
[ 356.153312] EA = 0, S1PTW = 0
[ 356.158513] crypto_ctr_crypt+0xe0/0x200
[ 356.158520] crypto_skcipher_encrypt+0x4c/0x80
[ 356.162254] FSC = 0x04: level 0 translation fault
[ 356.165640] crypto_gcm_encrypt+0xa4/0xdc
[ 356.165648] crypto_aead_encrypt+0x4c/0x80
[ 356.172777] Data abort info:
[ 356.176065] zynqmp_handle_aes_req+0x104/0x130
[ 356.176075] crypto_pump_requests.constprop.0+0x1ac/0x360
[ 356.176083] crypto_pump_work+0x1c/0x30
[ 356.181386] ISV = 0, ISS = 0x00000004
[ 356.186673] kthread_worker_fn+0x11c/0x340
[ 356.186681] kthread+0x110/0x114
[ 356.186687] ret_from_fork+0x10/0x20
[ 356.193807] CM = 0, WnR = 0
[ 356.196844] Code: d503201f d503201f aa1e03e9 d503201f (f9400001)
[ 356.196850] ---[ end trace 73e93e8d345dd261 ]---
[ 356.198324] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000807d63000
[ 356.280357] [0000000000000008] pgd=0000000000000000, p4d=0000000000000000
[ 356.287167] Internal error: Oops: 0000000096000004 [#2] SMP
[ 356.292731] Modules linked in: tls binfmt_misc ina2xx_adc xilinx_can can_dev mali(C) uio_pdrv_genirq sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua dmaproxy(C) pstore_blk ramoops reed_solomon pstore_zone efi_pstore ip_tables x_tables autofs4 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx raid1 raid0 multipath linear i2c_mux_pca954x crct10dif_ce rtc_zynqmp spi_zynqmp_gqspi i2c_cadence zynqmp_dpsub ahci_ceva aes_neon_blk crypto_simd cryptd
[ 356.335388] CPU: 3 PID: 2391 Comm: tls Tainted: G D WC 5.15.0-1029-xilinx-zynqmp #33-Ubuntu
[ 356.344770] Hardware name: ZynqMP ZCU106 RevA (DT)
[ 356.349545] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 356.356498] pc : sk_msg_free_elem+0x60/0x140
[ 356.360769] lr : __sk_msg_free+0x78/0x100
[ 356.364770] sp : ffff80000cec3c00
[ 356.368069] x29: ffff80000cec3c00 x28: ffff000809de8000 x27: 0000000000000000
[ 356.375196] x26: 0000000000000000 x25: dead000000000100 x24: dead000000000122
[ 356.382323] x23: 0000000000004010 x22: ffff0008038622e0 x21: 0000000000000001
[ 356.389450] x20: 0000000000000000 x19: ffff000803862320 x18: 0000000000000000
[ 356.396577] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[ 356.403704] x14: 0000000000000000 x13: 0000000000000000 x12: ffff8000099d4e78
[ 356.410831] x11: ffff80000aa8dad0 x10: ffff80000cec3b80 x9 : ffff800009219e68
[ 356.417958] x8 : fffffc00200e1a08 x7 : fffffc00200e1c08 x6 : fffffc00200e1c00
[ 356.425085] x5 : 0000000000000000 x4 : 000000000000000d x3 : 00000000001fffff
[ 356.432212] x2 : ffff000803862300 x1 : 000000000005bff3 x0 : 0000000000000000
[ 356.439340] Call trace:
[ 356.441772] sk_msg_free_elem+0x60/0x140
[ 356.445685] __sk_msg_free+0x78/0x100
[ 356.449339] sk_msg_free+0x20/0x30
[ 356.452734] tls_sw_release_resources_tx+0x14c/0x230 [tls]
[ 356.458229] tls_sk_proto_cleanup+0x80/0x184 [tls]
[ 356.463021] tls_sk_proto_close+0x98/0x260 [tls]
[ 356.467639] inet_release+0x58/0xac
[ 356.471120] __sock_release+0x4c/0xcc
[ 356.474774] sock_close+0x20/0x30
[ 356.478082] __fput+0x88/0x280
[ 356.481129] ____fput+0x18/0x24
[ 356.484263] task_work_run+0xc8/0x170
[ 356.487918] do_notify_resume+0x1e8/0x210
[ 356.491919] el0_svc+0x4c/0xb0
[ 356.494966] el0t_64_sync_handler+0xa4/0x12c
[ 356.499228] el0t_64_sync+0x1a4/0x1a8
[ 356.502888] Code: 12001c63 35000463 f9401040 927ef400 (f9400401)
[ 356.508972] ---[ end trace 73e93e8d345dd262 ]---

Behaviour on ZCU102 is a bit different:
$ sudo ./tls
TAP version 13
1..179
# Starting 179 tests from 6 test cases.
# RUN global.non_established ...
# OK global.non_established
ok 1 global.non_established
# RUN global.keysizes ...
# OK global.keysizes
ok 2 global.keysizes
# RUN tls_basic.base_base ...
# OK tls_basic.base_base
ok 3 tls_basic.base_base
# RUN tls.12_gcm.sendfile ...
# OK tls.12_gcm.sendfile
ok 4 tls.12_gcm.sendfile
# RUN tls.12_gcm.send_then_sendfile ...
# OK tls.12_gcm.send_then_sendfile
ok 5 tls.12_gcm.send_then_sendfile
# RUN tls.12_gcm.recv_max ...
# OK tls.12_gcm.recv_max
ok 6 tls.12_gcm.recv_max
# RUN tls.12_gcm.recv_small ...
# OK tls.12_gcm.recv_small
ok 7 tls.12_gcm.recv_small
# RUN tls.12_gcm.msg_more ...
# OK tls.12_gcm.msg_more
ok 8 tls.12_gcm.msg_more
# RUN tls.12_gcm.msg_more_unsent ...
# OK tls.12_gcm.msg_more_unsent
ok 9 tls.12_gcm.msg_more_unsent
# RUN tls.12_gcm.sendmsg_single ...
# OK tls.12_gcm.sendmsg_single
ok 10 tls.12_gcm.sendmsg_single
# RUN tls.12_gcm.sendmsg_fragmented ...
# OK tls.12_gcm.sendmsg_fragmented
ok 11 tls.12_gcm.sendmsg_fragmented
# RUN tls.12_gcm.sendmsg_large ...
# tls.c:394:sendmsg_large:Expected sendmsg(self->fd, &msg, 0) (-1) == send_len (16384)
# tls.c:394:sendmsg_large:Expected sendmsg(self->fd, &msg, 0) (-1) == send_len (16384)
(repeats)
# tls.c:394:sendmsg_large:Expected sendmsg(self->fd, &msg, 0) (-1) == send_len (16384)
# sendmsg_large: Test terminated by timeout
# FAIL tls.12_gcm.sendmsg_large
not ok 12 tls.12_gcm.sendmsg_large
# RUN tls.12_gcm.sendmsg_multiple ...
# sendmsg_multiple: Test terminated by timeout
# FAIL tls.12_gcm.sendmsg_multiple
not ok 13 tls.12_gcm.sendmsg_multiple
# RUN tls.12_gcm.sendmsg_multiple_stress ...
# sendmsg_multiple_stress: Test terminated by timeout
# FAIL tls.12_gcm.sendmsg_multiple_stress
not ok 14 tls.12_gcm.sendmsg_multiple_stress
# RUN tls.12_gcm.splice_from_pipe ...
^C

[ 504.379099] Internal error: Oops: 0000000096000004 [#1] SMP
[ 504.384662] Modules linked in: tls binfmt_misc xilinx_can can_dev ina2xx_adc mali(C) uio_pdrv_genirq sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua dmaproxy(C) ramoops reed_solomon pstore_blk pstore_zone efi_pstore ip_tables x_tables autofs4 raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx raid1 raid0 multipath linear i2c_mux_pca954x crct10dif_ce rtc_zynqmp spi_zynqmp_gqspi i2c_cadence zynqmp_dpsub ahci_ceva aes_neon_blk crypto_simd cryptd
[ 504.427320] CPU: 3 PID: 140 Comm: firmware:zynqmp Tainted: G C 5.15.0-1029-xilinx-zynqmp #33-Ubuntu
[ 504.437657] Hardware name: ZynqMP ZCU102 Rev1.0 (DT)
[ 504.442606] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 504.449558] pc : scatterwalk_copychunks+0x140/0x194
[ 504.454437] lr : scatterwalk_copychunks+0x13c/0x194
[ 504.459307] sp : ffff80000b7cbae0
[ 504.462605] x29: ffff80000b7cbae0 x28: 0000000000000000 x27: ffff0008008bb0c0
[ 504.469732] x26: ffff0008083ae700 x25: ffff80000b7cbc50 x24: 0000000000000010
[ 504.476859] x23: 0000000000000000 x22: 0000000000000000 x21: 0000040000000000
[ 504.483986] x20: ffff000000000000 x19: 0000000000000000 x18: 0000000000b70000
[ 504.491113] x17: 00000000008c0000 x16: 0000000000000311 x15: 0000000000000000
[ 504.498240] x14: 0000000000000001 x13: 0000000000004000 x12: 0000000000000010
[ 504.505367] x11: 0000000000004000 x10: 00000000ffff8000 x9 : ffff80000884d748
[ 504.512494] x8 : ffff0008083ae780 x7 : 0000000000000000 x6 : 000000000000003f
[ 504.519621] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000001000
[ 504.526749] x2 : 0000000000000000 x1 : 0000000000000002 x0 : 0000000000000000
[ 504.533876] Call trace:
[ 504.536308] scatterwalk_copychunks+0x140/0x194
[ 504.540829] skcipher_next_slow+0xb8/0x140
[ 504.544918] skcipher_walk_next+0x1a4/0x32c
[ 504.549093] skcipher_walk_done+0xf4/0x2c0
[ 504.553182] crypto_ctr_crypt+0x184/0x200
[ 504.557183] crypto_skcipher_encrypt+0x4c/0x80
[ 504.561619] crypto_gcm_encrypt+0xa4/0xdc
[ 504.565621] crypto_aead_encrypt+0x4c/0x80
[ 504.569710] zynqmp_handle_aes_req+0x104/0x130
[ 504.574146] crypto_pump_requests.constprop.0+0x1ac/0x360
[ 504.579537] crypto_pump_work+0x1c/0x30
[ 504.583365] kthread_worker_fn+0x11c/0x340
[ 504.587454] kthread+0x110/0x114
[ 504.590674] ret_from_fork+0x10/0x20
[ 504.594247] Code: 6b0400bf 54fff928 9402892a f9000320 (b9400801)
[ 504.600331] ---[ end trace c58a48f38f7f791e ]---

This will cause ubuntu_kselftests_net unable to finish properly. And thus generating the sut-test failure.