Ubuntu
linux-azure package

panic due to unhandled page fault via BPF_PROG_RUN syscall

Bug #2045778 reported by Lorenz Bauer on 2023-12-06

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	linux-azure (Ubuntu)	Triaged	High	Marcelo Cerri

Bug Description

Here is a kernel oops triggered from user space by invoking a BPF program:

[ 1191.051531] BUG: unable to handle page fault for address: ffffffffea053c70
[ 1191.053848] #PF: supervisor read access in kernel mode
[ 1191.055183] #PF: error_code(0x0000) - not-present page
[ 1191.056513] PGD 334e15067 P4D 334e15067 PUD 334e17067 PMD 0
[ 1191.058016] Oops: 0000 [#1] SMP NOPTI
[ 1191.058984] CPU: 1 PID: 2557 Comm: ebpf.test Not tainted 6.2.0-1016-azure #16~22.04.1-Ubuntu
[ 1191.061167] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018
[ 1191.063804] RIP: 0010:bpf_test_run+0x104/0x2e0
[ 1191.065064] Code: 00 00 48 89 90 50 14 00 00 48 89 b5 60 ff ff ff eb 3e 0f 1f 44 00 00 48 8b 53 30 4c 89 ee 4c 89 e7 e8 50 8c f8 ff 89 c2 66 90 <48> 8b 45 80 4d 89 f0 48 8d 4d 8c be 01 00 00 00 48 8d 7d a0 89 10
[ 1191.069766] RSP: 0018:ffffa64e03053c50 EFLAGS: 00010246
[ 1191.071117] RAX: 0000000000000001 RBX: ffffa64e0005a000 RCX: ffffa64e03053c3f
[ 1191.073415] RDX: 0000000000000001 RSI: ffffa64e03053c3f RDI: ffffffff8a468580
[ 1191.075351] RBP: ffffffffea053cf0 R08: 0000000000000000 R09: 0000000000000000
[ 1191.077722] R10: 0000000000000000 R11: 0000000000000000 R12: ffff97dc75673c00
[ 1191.079681] R13: ffffa64e0005a048 R14: ffffa64e03053d34 R15: 0000000000000001
[ 1191.081636] FS: 00007fd4a2ffd640(0000) GS:ffff97df6fc80000(0000) knlGS:0000000000000000
[ 1191.083866] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1191.085455] CR2: ffffffffea053c70 CR3: 000000019ff80001 CR4: 0000000000370ee0
[ 1191.087405] Call Trace:
[ 1191.088121] <TASK>
[ 1191.088745] ? show_regs+0x6a/0x80
[ 1191.089710] ? __die+0x25/0x70
[ 1191.090591] ? page_fault_oops+0x79/0x180
[ 1191.091708] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.093027] ? search_exception_tables+0x61/0x70
[ 1191.094421] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.095686] ? kernelmode_fixup_or_oops+0xa2/0x120
[ 1191.097014] ? __bad_area_nosemaphore+0x16f/0x280
[ 1191.098323] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.099584] ? apparmor_file_alloc_security+0x1f/0xd0
[ 1191.100989] ? bad_area_nosemaphore+0x16/0x20
[ 1191.102235] ? do_kern_addr_fault+0x62/0x80
[ 1191.103393] ? exc_page_fault+0xd8/0x160
[ 1191.104505] ? asm_exc_page_fault+0x27/0x30
[ 1191.105669] ? bpf_test_run+0x104/0x2e0
[ 1191.106745] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.108010] ? bpf_prog_test_run_skb+0x2e4/0x4f0
[ 1191.109350] ? __fdget+0x13/0x20
[ 1191.110304] ? __sys_bpf+0x706/0xea0
[ 1191.111299] ? __x64_sys_bpf+0x1a/0x30
[ 1191.112307] ? do_syscall_64+0x5c/0x90
[ 1191.113366] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.114634] ? exit_to_user_mode_loop+0xec/0x160
[ 1191.115929] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.117466] ? __set_task_blocked+0x29/0x70
[ 1191.118904] ? exit_to_user_mode_prepare+0x49/0x100
[ 1191.120482] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.122073] ? sigprocmask+0xb8/0xe0
[ 1191.123360] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.124868] ? exit_to_user_mode_prepare+0x49/0x100
[ 1191.126523] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.128028] ? syscall_exit_to_user_mode+0x27/0x40
[ 1191.129599] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.131033] ? do_syscall_64+0x69/0x90
[ 1191.132242] ? srso_alias_return_thunk+0x5/0x7f
[ 1191.134199] ? do_syscall_64+0x69/0x90
[ 1191.135504] ? entry_SYSCALL_64_after_hwframe+0x73/0xdd
[ 1191.137137] </TASK>
[ 1191.137942] Modules linked in: nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack_netlink xfrm_user xfrm_algo xt_addrtype br_netfilter bridge stp llc xt_tcpudp tls xt_owner xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables libcrc32c nfnetlink overlay nvme_fabrics udf crc_itu_t binfmt_misc nls_iso8859_1 kvm_amd ccp joydev kvm hid_generic irqbypass crct10dif_pclmul crc32_pclmul polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 aesni_intel crypto_simd cryptd hyperv_drm drm_kms_helper syscopyarea sysfillrect serio_raw sysimgblt drm_shmem_helper hid_hyperv hv_netvsc hid hyperv_keyboard pata_acpi dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua sch_fq_codel drm efi_pstore i2c_core ip_tables x_tables autofs4
[ 1191.156484] CR2: ffffffffea053c70
[ 1191.158026] ---[ end trace 0000000000000000 ]---
[ 1191.159518] RIP: 0010:bpf_test_run+0x104/0x2e0
[ 1191.160912] Code: 00 00 48 89 90 50 14 00 00 48 89 b5 60 ff ff ff eb 3e 0f 1f 44 00 00 48 8b 53 30 4c 89 ee 4c 89 e7 e8 50 8c f8 ff 89 c2 66 90 <48> 8b 45 80 4d 89 f0 48 8d 4d 8c be 01 00 00 00 48 8d 7d a0 89 10
[ 1191.166336] RSP: 0018:ffffa64e03053c50 EFLAGS: 00010246
[ 1191.168046] RAX: 0000000000000001 RBX: ffffa64e0005a000 RCX: ffffa64e03053c3f
[ 1191.170129] RDX: 0000000000000001 RSI: ffffa64e03053c3f RDI: ffffffff8a468580
[ 1191.172210] RBP: ffffffffea053cf0 R08: 0000000000000000 R09: 0000000000000000
[ 1191.174546] R10: 0000000000000000 R11: 0000000000000000 R12: ffff97dc75673c00
[ 1191.176719] R13: ffffa64e0005a048 R14: ffffa64e03053d34 R15: 0000000000000001
[ 1191.178807] FS: 00007fd4a2ffd640(0000) GS:ffff97df6fc80000(0000) knlGS:0000000000000000
[ 1191.181128] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1191.182936] CR2: ffffffffea053c70 CR3: 000000019ff80001 CR4: 0000000000370ee0
[ 1191.185355] note: ebpf.test[2557] exited with irqs disabled

Release info:

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

You can reproduce this by checking out https://github.com/cilium/ebpf and running the following in the root of the project:

go test -exec sudo -run 'TestKfunc$' -timeout 30s -v .

The same test executes fine on upstream 6.1 and 6.6. I also tested against 6.2.9 from kernel.org and didn't get the same splat.

Joseph Salisbury (jsalisbury) on 2023-12-07

Changed in linux-azure (Ubuntu):
importance:	Undecided → High
status:	New → Triaged