Unable to handle kernel paging request at virtual address ffffff8a012abe1a

[Heinrich Schuchardt]

I have been running the Mantic preinstalled image on the VisionFive 2 1.3B board.
http://cdimage.ubuntu.com/ubuntu-server/daily-preinstalled/20231010/mantic-preinstalled-server-riscv64+visionfive2.img.xz

Kernel Ubuntu 6.5.0-9.9.1-generic 6.5.3

[ 4050.418337] Unable to handle kernel paging request at virtual address ffffffb4fffffffc
[ 4050.426326] Oops [#1]
[ 4050.428601] Modules linked in: cfg80211 binfmt_misc nls_iso8859_1 cdns3 cdns_usb_common udc_core jh7110_tdm ofpart cdns3_starfive cmdlinepart snd_soc_core spi_nor snd_compress ac97_bus snd_pcm_dmaengine axp20x_pek mtd snd_pcm snd_timer dw_axi_dmac_platform pwm_starfive sfctemp snd soundcore uio_pdrv_genirq uio dm_multipath drm efi_pstore backlight ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear motorcomm dwmac_starfive nvme stmmac_platform axp20x_regulator stmmac nvme_core dw_mmc_starfive dw_mmc_pltfm xhci_pci pcs_xpcs axp20x_i2c clk_starfive_jh7110_vout phylink xhci_pci_renesas axp20x dw_mmc pinctrl_starfive_jh7110_aon clk_starfive_jh7110_isp nvme_common clk_starfive_jh7110_aon jh7110_trng phy_jh7110_usb spi_cadence_quadspi
[ 4050.503975] CPU: 1 PID: 1866 Comm: check-new-relea Not tainted 6.5.0-9-generic #9.1-Ubuntu
[ 4050.512235] Hardware name: StarFive VisionFive 2 v1.3B (DT)
[ 4050.517804] epc : __d_lookup_rcu+0x4a/0x100
[ 4050.521997] ra : lookup_fast+0x36/0xfc
[ 4050.525837] epc : ffffffff80315146 ra : ffffffff803042bc sp : ffffffc800a6bab0
[ 4050.533056] gp : ffffffff822602d8 tp : ffffffd8c56ea880 t0 : ffffffc800a6bce8
[ 4050.540279] t1 : 0000000000000000 t2 : ffffffff812002e8 s0 : ffffffc800a6bae0
[ 4050.547502] s1 : ffffffc800a6bc00 a0 : ffffffd8c9fd8a80 a1 : ffffffc800a6bc10
m[ 4050.554810] a2 : ffffffc800a6bc44 a3 : 0000000000000001 a4 : 0001563500000000
[ 4050.562033] a5 : 0001563500000000 a6 : 0003523800000000 a7 : 0000000000000000
[ 4050.569254] s2 : 0000000000000002 s3 : ffffffd8c9fd8a80 s4 : 000000000000002e
[ 4050.576476] s5 : 0000000000200000 s6 : 0000000000700000 s7 : fffffffffffffff6
[ 4050.583697] s8 : 0000000000000002 s9 : ffffffd8ca244025 s10: ffffffc800a6bc00
[ 4050.590918] s11: 000000000000002f t3 : ffffffd8c9fd8a80 t4 : 0000000335238e2d
[ 4050.598141] t5 : ffffffc800a6bc44 t6 : ffffffd8ca244021
[ 4050.603452] status: 0000000200000120 badaddr: ffffffb4fffffffc cause: 000000000000000d
[ 4050.611367] [<ffffffff80315146>] __d_lookup_rcu+0x4a/0x100
[ 4050.616858] [<ffffffff803084cc>] walk_component+0x26/0x194
[ 4050.622348] [<ffffffff803087fc>] link_path_walk.part.0.constprop.0+0x1c2/0x2c2
[ 4050.629575] [<ffffffff80308944>] path_parentat+0x48/0x96
[ 4050.634890] [<ffffffff80308a30>] __filename_parentat+0x9e/0x188
[ 4050.640813] [<ffffffff8030b10e>] do_renameat2+0xc8/0x4e2
[ 4050.646128] [<ffffffff8030b582>] sys_renameat2+0x5a/0x76
[ 4050.651441] [<ffffffff80ce48f0>] do_trap_ecall_u+0xd6/0x154
[ 4050.657018] [<ffffffff80003e30>] ret_from_exception+0x0/0x64
[ 4050.662691] Code: 4685 8e2a 8f32 7793 ffe7 e563 00e6 a885 639c c7b5 (a703) ffc7
[ 4050.670148] ---[ end trace 0000000000000000 ]---
[ 4050.670271] Unable to handle kernel paging request at virtual address 0000005dfffffffc
[ 4050.670360] Unable to handle kernel paging request at virtual address ffffffb4fffffffc
[ 4050.670368] Oops [#2]
[ 4050.670371] Modules linked in: cfg80211 binfmt_misc nls_iso8859_1 cdns3 cdns_usb_common udc_core jh7110_tdm ofpart cdns3_starfive cmdlinepart snd_soc_core spi_nor snd_compress ac97_bus snd_pcm_dmaengine axp20x_pek mtd snd_pcm snd_timer dw_axi_dmac_platform pwm_starfive sfctemp snd soundcore uio_pdrv_genirq uio dm_multipath drm efi_pstore backlight ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear motorcomm dwmac_starfive nvme stmmac_platform axp20x_regulator stmmac nvme_core dw_mmc_starfive dw_mmc_pltfm xhci_pci pcs_xpcs axp20x_i2c clk_starfive_jh7110_vout phylink xhci_pci_renesas axp20x dw_mmc pinctrl_starfive_jh7110_aon clk_starfive_jh7110_isp nvme_common clk_starfive_jh7110_aon jh7110_trng phy_jh7110_usb spi_cadence_quadspi
[ 4050.670556] CPU: 3 PID: 788 Comm: rs:main Q:Reg Tainted: G D 6.5.0-9-generic #9.1-Ubuntu
[ 4050.670562] Hardware name: StarFive VisionFive 2 v1.3B (DT)
[ 4050.670565] epc : __d_lookup_rcu+0x4a/0x100
[ 4050.670574] ra : lookup_fast+0x36/0xfc
[ 4050.670582] epc : ffffffff80315146 ra : ffffffff803042bc sp : ffffffc80212bb70
[ 4050.670587] gp : ffffffff822602d8 tp : ffffffd8cb66d100 t0 : ffffffc80212bdb8
[ 4050.670591] t1 : 0000000000000000 t2 : ffffffff812002e8 s0 : ffffffc80212bba0
[ 4050.670595] s1 : ffffffc80212bcd0 a0 : ffffffd8c9fd8a80 a1 : ffffffc80212bce0
[ 4050.670600] a2 : ffffffc80212bd14 a3 : 0000000000000001 a4 : 0001563500000000
[ 4050.670604] a5 : 0001563500000000 a6 : 0003523800000000 a7 : 0000000000000000
[ 4050.670608] s2 : 0000000000000002 s3 : ffffffd8c9fd8a80 s4 : 000000000000002e
[ 4050.670612] s5 : 0000000000200000 s6 : 0000000000700000 s7 : fffffffffffffff6
[ 4050.670616] s8 : 0000000000000002 s9 : ffffffd8c9619025 s10: ffffffc80212bcd0
[ 4050.670620] s11: 000000000000002f t3 : ffffffd8c9fd8a80 t4 : 0000000335238e2d
[ 4050.670624] t5 : ffffffc80212bd14 t6 : ffffffd8c9619021
[ 4050.670628] status: 0000000200000120 badaddr: ffffffb4fffffffc cause: 000000000000000d
[ 4050.670632] [<ffffffff80315146>] __d_lookup_rcu+0x4a/0x100
[ 4050.670640] [<ffffffff803084cc>] walk_component+0x26/0x194
[ 4050.670648] [<ffffffff803087fc>] link_path_walk.part.0.constprop.0+0x1c2/0x2c2
[ 4050.670657] [<ffffffff803090ec>] path_lookupat+0x44/0x196
[ 4050.670665] [<ffffffff80309ba4>] filename_lookup+0x96/0x166
[ 4050.670673] [<ffffffff80309da4>] user_path_at_empty+0x42/0x6c
[ 4050.670682] [<ffffffff802f0fce>] do_faccessat+0xd4/0x284
[ 4050.670689] [<ffffffff802f119c>] sys_faccessat+0x1e/0x2c
[ 4050.670695] [<ffffffff80ce48f0>] do_trap_ecall_u+0xd6/0x154
[ 4050.670703] [<ffffffff80003e30>] ret_from_exception+0x0/0x64
[ 4050.670717] Code: 4685 8e2a 8f32 7793 ffe7 e563 00e6 a885 639c c7b5 (a703) ffc7
[ 4050.670721] ---[ end trace 0000000000000000 ]---
[ 4050.693300] Unable to handle kernel paging request at virtual address ffffff8a012abe1a
[ 4050.693392] Unable to handle kernel paging request at virtual address ffffff92fffffffc
[ 4050.693401] Oops [#3]
[ 4050.693405] Modules linked in: cfg80211 binfmt_misc nls_iso8859_1 cdns3 cdns_usb_common udc_core jh7110_tdm ofpart cdns3_starfive cmdlinepart snd_soc_core spi_nor snd_compress ac97_bus snd_pcm_dmaengine axp20x_pek mtd snd_pcm snd_timer dw_axi_dmac_platform pwm_starfive sfctemp snd soundcore uio_pdrv_genirq uio dm_multipath drm efi_pstore backlight ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear motorcomm dwmac_starfive nvme stmmac_platform axp20x_regulator stmmac nvme_core dw_mmc_starfive dw_mmc_pltfm xhci_pci pcs_xpcs axp20x_i2c clk_starfive_jh7110_vout phylink xhci_pci_renesas axp20x dw_mmc pinctrl_starfive_jh7110_aon clk_starfive_jh7110_isp nvme_common clk_starfive_jh7110_aon jh7110_trng phy_jh7110_usb spi_cadence_quadspi
[ 4050.693605] CPU: 0 PID: 1 Comm: systemd Tainted: G D 6.5.0-9-generic #9.1-Ubuntu
[ 4050.693611] Hardware name: StarFive VisionFive 2 v1.3B (DT)
[ 4050.693614] epc : __d_lookup_rcu+0x4a/0x100
[ 4050.693629] ra : lookup_fast+0x36/0xfc
[ 4050.693638] epc : ffffffff80315146 ra : ffffffff803042bc sp : ffffffc800023b80
[ 4050.693643] gp : ffffffff822602d8 tp : ffffffd8c027ec00 t0 : ffffffd8c0bc2380
[ 4050.693648] t1 : 0000000000000000 t2 : ffffffff812002e8 s0 : ffffffc800023bb0
[ 4050.693652] s1 : ffffffc800023d10 a0 : ffffffd8c9fd8a80 a1 : ffffffc800023d20
[ 4050.693656] a2 : ffffffc800023d54 a3 : 0000000000000001 a4 : 0011579300000000
[ 4050.693660] a5 : 0011579300000000 a6 : 000874d600000000 a7 : 0000000000000000
[ 4050.693664] s2 : 0000000000000002 s3 : ffffffd8c9fd8a80 s4 : 000000000000002e
[ 4050.693668] s5 : 0000000000200000 s6 : 0000000000700000 s7 : fffffffffffffff6
[ 4050.693672] s8 : 0000000000000002 s9 : ffffffd8c9610026 s10: ffffffc800023d10
[ 4050.693677] s11: 000000000000002f t3 : ffffffd8c9fd8a80 t4 : 00000004874d6bef
[ 4050.693681] t5 : ffffffc800023d54 t6 : ffffffd8c9610021
[ 4050.693684] status: 0000000200000120 badaddr: ffffff92fffffffc cause: 000000000000000d
[ 4050.693689] [<ffffffff80315146>] __d_lookup_rcu+0x4a/0x100
[ 4050.693697] [<ffffffff803084cc>] walk_component+0x26/0x194
[ 4050.693706] [<ffffffff803087fc>] link_path_walk.part.0.constprop.0+0x1c2/0x2c2
[ 4050.693716] [<ffffffff80309602>] path_openat+0x74/0x26e
[ 4050.693724] [<ffffffff8030a05a>] do_filp_open+0x88/0x118
[ 4050.693732] [<ffffffff802f32e8>] do_sys_openat2+0xac/0xe4
[ 4050.693740] [<ffffffff802f34e2>] sys_openat+0x48/0x94
[ 4050.693747] [<ffffffff80ce48f0>] do_trap_ecall_u+0xd6/0x154
[ 4050.693756] [<ffffffff80003e30>] ret_from_exception+0x0/0x64
[ 4050.693773] Code: 4685 8e2a 8f32 7793 ffe7 e563 00e6 a885 639c c7b5 (a703) ffc7
[ 4050.693779] ---[ end trace 0000000000000000 ]---
[ 4050.693787] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 4050.693790] SMP: stopping secondary CPUs
[ 4051.220077] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---
[ 4051.228989] ------------[ cut here ]------------
[ 4051.228993] WARNING: CPU: 0 PID: 1 at kernel/sched/core.c:3357 set_task_cpu+0x19c/0x1aa
[ 4051.229007] Modules linked in: cfg80211 binfmt_misc nls_iso8859_1 cdns3 cdns_usb_common udc_core jh7110_tdm ofpart cdns3_starfive cmdlinepart snd_soc_core spi_nor snd_compress ac97_bus snd_pcm_dmaengine axp20x_pek mtd snd_pcm snd_timer dw_axi_dmac_platform pwm_starfive sfctemp snd soundcore uio_pdrv_genirq uio dm_multipath drm efi_pstore backlight ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear motorcomm dwmac_starfive nvme stmmac_platform axp20x_regulator stmmac nvme_core dw_mmc_starfive dw_mmc_pltfm xhci_pci pcs_xpcs axp20x_i2c clk_starfive_jh7110_vout phylink xhci_pci_renesas axp20x dw_mmc pinctrl_starfive_jh7110_aon clk_starfive_jh7110_isp nvme_common clk_starfive_jh7110_aon jh7110_trng phy_jh7110_usb spi_cadence_quadspi
[ 4051.229152] CPU: 0 PID: 1 Comm: systemd Tainted: G D 6.5.0-9-generic #9.1-Ubuntu
[ 4051.229158] Hardware name: StarFive VisionFive 2 v1.3B (DT)
[ 4051.229161] epc : set_task_cpu+0x19c/0x1aa
[ 4051.229168] ra : try_to_wake_up+0x1a8/0x6f4
[ 4051.229174] epc : ffffffff80057900 ra : ffffffff8005cae6 sp : ffffffc800003bc0
[ 4051.229178] gp : ffffffff822602d8 tp : ffffffd8c027ec00 t0 : 0000000000046600
[ 4051.229182] t1 : 0000000000000000 t2 : 20646e65205b2d2d s0 : ffffffc800003c00
[ 4051.229186] s1 : ffffffd8c5bc2880 a0 : ffffffd8c5bc2880 a1 : 0000000000000003
[ 4051.229189] a2 : 0000000000000000 a3 : 0000000000000000 a4 : ffffffff82264240
[ 4051.229193] a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000
[ 4051.229196] s2 : ffffffff810a21c0 s3 : 0000000000000003 s4 : 0000000000000003
[ 4051.229200] s5 : 0000000200000120 s6 : 0000000000000003 s7 : ffffffff822d19e8
[ 4051.229204] s8 : ffffffc800003c08 s9 : 0000000000000000 s10: 0000000000000000
[ 4051.229207] s11: 000000000000002f t3 : ffffffd8c0124f00 t4 : ffffffd8c0124f00
[ 4051.229211] t5 : ffffffd8c0124000 t6 : ffffffc800023608
[ 4051.229214] status: 0000000200000100 badaddr: 0000000000000000 cause: 0000000000000003
[ 4051.229218] [<ffffffff80057900>] set_task_cpu+0x19c/0x1aa
[ 4051.229226] [<ffffffff8005cae6>] try_to_wake_up+0x1a8/0x6f4
[ 4051.229232] [<ffffffff8005d142>] default_wake_function+0x20/0x48
[ 4051.229238] [<ffffffff8007cd56>] autoremove_wake_function+0x1c/0x46
[ 4051.229248] [<ffffffff8007ac7c>] __wake_up_common+0x70/0x128
[ 4051.229255] [<ffffffff8007ada2>] __wake_up_common_lock+0x6e/0xbe
[ 4051.229261] [<ffffffff8007ae82>] __wake_up+0x1c/0x2c
[ 4051.229268] [<ffffffff80093698>] wake_up_klogd_work_func+0x8a/0xa0
[ 4051.229275] [<ffffffff8018864c>] irq_work_run_list+0x4a/0x8a
[ 4051.229283] [<ffffffff80188994>] irq_work_tick+0x38/0x46
[ 4051.229290] [<ffffffff800cb1a0>] update_process_times+0xc2/0xc4
[ 4051.229300] [<ffffffff800dd526>] tick_sched_handle+0x30/0x6c
[ 4051.229308] [<ffffffff800ddd32>] tick_sched_timer+0x66/0xc2
[ 4051.229315] [<ffffffff800cba50>] __hrtimer_run_queues+0x102/0x1e0
[ 4051.229323] [<ffffffff800ccbca>] hrtimer_interrupt+0xe0/0x204
[ 4051.229330] [<ffffffff80a63108>] riscv_timer_interrupt+0x38/0x46
[ 4051.229340] [<ffffffff8009df7e>] handle_percpu_devid_irq+0x90/0x18c
[ 4051.229347] [<ffffffff80096d2e>] generic_handle_domain_irq+0x28/0x3e
[ 4051.229354] [<ffffffff8067643c>] riscv_intc_irq+0x34/0x54
[ 4051.229363] [<ffffffff80ce3d9c>] handle_riscv_irq+0x4a/0x78
[ 4051.229370] [<ffffffff80ce4a0e>] do_irq+0x5e/0xac
[ 4051.229377] ---[ end trace 0000000000000000 ]---

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
https://iso.qa.ubuntu.com/qatracker/reports/bugs/2038928

[Heinrich Schuchardt (xypron)]

echo blacklist jh7110_tdm > /etc/modprobe.d/blacklist-jh7110_tdm.conf
did not solve the issue. I still get oopses like:

[ 201.478403] Oops [#1]
[ 201.480681] Modules linked in: tls cfg80211 binfmt_misc nls_iso8859_1 cdns3 cdns_usb_common udc_core cdns3_starfive ofpart cmdlinepart spi_nor mtd axp20x_pek pwm_starfive dw_axi_dmac_platform sfctemp uio_pdrv_genirq uio drm dm_multipath efi_pstore backlight ip_tables x_tables autofs4 btrfs blake2b_generic raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear motorcomm dwmac_starfive stmmac_platform stmmac nvme xhci_pci axp20x_regulator xhci_pci_renesas nvme_core dw_mmc_starfive pcs_xpcs phylink nvme_common dw_mmc_pltfm clk_starfive_jh7110_aon clk_starfive_jh7110_isp clk_starfive_jh7110_vout pinctrl_starfive_jh7110_aon dw_mmc axp20x_i2c axp20x spi_cadence_quadspi jh7110_trng phy_jh7110_usb
[ 201.548071] CPU: 0 PID: 612 Comm: systemd-network Not tainted 6.5.0-9-generic #9.1-Ubuntu
[ 201.556251] Hardware name: StarFive VisionFive 2 v1.3B (DT)
[ 201.561824] epc : __d_lookup_rcu+0x4a/0x100
[ 201.566024] ra : lookup_fast+0x36/0xfc
[ 201.569870] epc : ffffffff80315146 ra : ffffffff803042bc sp : ffffffc800ecb8b0
[ 201.577091] gp : ffffffff822602d8 tp : ffffffd8c6f6d100 t0 : ffffffc800ecba58
[ 201.584320] t1 : 0000000000000000 t2 : 0000000000000008 s0 : ffffffc800ecb8e0
[ 201.591547] s1 : ffffffc800ecb970 a0 : ffffffd8c49b8000 a1 : ffffffc800ecb980
[ 201.598772] a2 : ffffffc800ecb9b4 a3 : 0000000000000001 a4 : 3b91a71338836970
[ 201.605996] a5 : 3b91a71338836970 a6 : 0000361d00000000 a7 : 0000000000000000
[ 201.613222] s2 : 0000000000000001 s3 : ffffffd8c49b8000 s4 : fffffffffffff000
[ 201.620444] s5 : 0000000000000001 s6 : ffffffc800ecbca8 s7 : 0000000000000000
[ 201.627666] s8 : ffffffd8c705ca00 s9 : 000000000000000a s10: 7fffffffffffffff
[ 201.634887] s11: 0000003fc689f768 t3 : ffffffd8c49b8000 t4 : 000000060361d211
[ 201.642109] t5 : ffffffc800ecb9b4 t6 : ffffffd8c24b102d
[ 201.647420] status: 0000000200000120 badaddr: ffffff933883696c cause: 000000000000000d
[ 201.655337] [<ffffffff80315146>] __d_lookup_rcu+0x4a/0x100
[ 201.660834] [<ffffffff803084cc>] walk_component+0x26/0x194
[ 201.666325] [<ffffffff8030911e>] path_lookupat+0x76/0x196
[ 201.671730] [<ffffffff80309ba4>] filename_lookup+0x96/0x166
[ 201.677306] [<ffffffff80309cac>] kern_path+0x38/0x60
[ 201.682274] [<ffffffff80be4e4c>] unix_find_other+0x50/0x36a
[ 201.687850] [<ffffffff80be7306>] unix_dgram_sendmsg+0x43c/0x81a
[ 201.693771] [<ffffffff80a8cc6e>] sock_sendmsg+0x4e/0x98
[ 201.699004] [<ffffffff80a8cfce>] ____sys_sendmsg+0x1d8/0x20a
[ 201.704664] [<ffffffff80a8f3c0>] ___sys_sendmsg+0x90/0xd6
[ 201.710066] [<ffffffff80a8f9cc>] __sys_sendmsg+0x84/0xd2
[ 201.715380] [<ffffffff80a8fa38>] sys_sendmsg+0x1e/0x2c
[ 201.720521] [<ffffffff80ce48f0>] do_trap_ecall_u+0xd6/0x154
[ 201.726099] [<ffffffff80003e30>] ret_from_exception+0x0/0x64
[ 201.731773] Code: 4685 8e2a 8f32 7793 ffe7 e563 00e6 a885 639c c7b5 (a703) ffc7
[ 201.739422] Unable to handle kernel paging request at virtual address ffffffbf0099039c
[ 201.739559] Unable to han...

[Heinrich Schuchardt (xypron)]

The problem seems to be related to a U-Boot reporting the wrong memory size on a VisionFive 2 with 4GiB.

[Heinrich Schuchardt (xypron)]

This patch for upstream U-Boot seems to resolve the issue: https://lore<email address hidden>/T/#u. I will retest the image.

[Brian Murray (brian-murray)]

Setting to Incomplete pending the results of the testing in comment #6.

[Heinrich Schuchardt (xypron)]

With u-boot-starfive - 2023.09.22-next-5d2fae79c7d6-0ubuntu1~ppa4 from ppa:ubuntu-risc-v-team/release the problem is resolved.