Kernel Oops - kernel NULL pointer dereference (apparmor_bprm_creds_for_exec, profile_transition) with linux-image-6.5.0-18-generic (linux-image-generic-hwe-22.04)

[Pavel Malyshev]

Today one of my NUCs got new kernel 6.5.0-14 and the host started to hang few minutes after the boot.
I can't find anything in the journal for the previous boot it just ends without any error. Remote syslog server does not receive any error log from this host. I've tried running journalctl on the local console and wait for the hang got nothing.
On the screen (this is Ubuntu Server) I see 18 hex numbers at the top of the screen and the rest is what was on the console before the hang:
 ff ff 73 01 c3 48 8b 0d 74 ed 12 00 f7 d8 64 89 01 48

Booting the previous HWE linux-image-6.2.0-39-generic fixes the issue, so it is definitely the new kernel.

I don't have enough time to generate report with ubuntu-bug on the new kernel, so attaching dmidecode, lspci, lsusb and dmesg from the older one.

[Pavel Malyshev (afunix)]

I was able to find one single line that syslog server was able to catch:
[ 353.589347] BUG: kernel NULL pointer dereference, address: 0000000000000008

I guess it doesn't give a lot of information...

[Pavel Malyshev (afunix)]

Still reproducible with linux-image-6.5.0-15-generic

[Pavel Malyshev (afunix)]

Still reproducible with 6.5.0-18-generic

[Pavel Malyshev (afunix)]

I was able to disable mode switch and record a video with the kernel attempting to output the oops.
It seems to be an issue with apparmor.

The text I manually recovered from the video:

---
BUG: kernel NULL pointer dereference, address: 0000000000000008
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
P?D 0 P4D 0
Ooops: 0000 [#1] PREEMPT SMP WDPTI
...
Call Trace:
 <TASK>
 ? show_regs+0x6d/0x80
 ? __die+0x24/0x80
 ? page_fault_oops+0x99/0x1b0
 ? do_user_addr_fault+0x31d/0x6b0
 ? exc_page_fault+0x83/0x1b0
 ? asm_exc_page_fault+0x27/0x30
 ? find_attach+0x11a/0x5e0
 ? d_namespace_path_constprop.0+0xa3/0x310
 profile_transition+0x5e0/0x910
 apparmor_bprm_creds_for_exec+0x3b9/0xc00
 security_bprm_creds_for_exec+0x2e/0x50
 bprm_execve.part.0+0x12f/0x250
 bprm_execve+0x5d/0xa0
 do_execveat_common.isra.0+0x1a9/0x260
 __x64_sys_execve+0x87/0x60
 do_syscall_64+0x58/0x?0
 ? syscall_exit_to_user_mode+0x37/0x60
 ? do_syscall_64+0x67/0x90
 entry_SYSCALL_64_after_hwframe+0x6r/0xd8
---

I will also attach screenshots from the video

[Pavel Malyshev (afunix)]

I can confirm that booting with "apparmor=0" added to kernel command line works around the issue.
This host runs lxd containers, so from the security standpoint running it without apparmor is not acceptable. However this proves there is an apparmor bug.

[Pavel Malyshev (afunix)]

Still reproducible with 6.5.0-25-generic

[Pavel Malyshev (afunix)]

Still reproducible with 6.5.0-26-generic
kdump won't generate any dump or send anything over ssh.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[lousuan (lousuan)]

Same Problem here. Tried with 6.5.0-25-generic and 6.5.0-27-generic but no luck.
Here's what I got by `dmesg -T`.

Update: linux-image-6.2.0-39-generic works fine.

Update: Recently I updated docker-ce to 26.0.1. Not sure whether is this releated.