EXT4-fs errors on black screen after resume from sleep

[Anton Keks]

Some time after updating to Ubuntu 22.04 and Kernel 5.15, Intel Tiger Lake laptop started getting black screen and EXT4-fs errors 10-20 seconds after resume from sleep. This is reproducible approximately every other time, i.e. sometimes the resume works normally.

Usually, after resume I can enter GDM password, unlock the desktop and see running apps, but soon when they start writing to the disk (NVME), the screen goes blank with a bunch of errors regarding read-only filesystem, unable to unmount partitions, etc.

Upgrading to Ubuntu 22.10 / Kernel 5.19 does not fix the issue. The same laptop worked flawlessly before Ubuntu 22.04.

After hard reset and boot, the system works normally.

The logs are not written anywhere besides the screen.

OCR from an photo using Google Lens:

[19449.602615) systemd[9792) home.mount: Failed at step EXEC spawning /bin/umount: Input/output error
[19449.609995) systemd[1]: Failed unmounting /home.
[19449.932421] systemd[9794): docker.service: Failed at step EXEC spawning /usr/bin/dockerd: Input/output error
[19449.959791) systemd[1]: Failed to start Docker Application Container Engine.
[19452.164163) systemd[9795): docker.service: Failed at step EXEC spawning /usr/bin/dockerd: Input/output error
[19452.165445] systemd[1]: Failed to start Docker Application Container Engine.
[19454.397310] EXT4-fs error (device nvmeon1p5): ext4_find_entry: 1658: inode #655370: comm systemd: reading directory Iblock o
[19454.397999) EXT4-fs error (device nvmeon1p5): ext4_find_entry: 1658: inode #2: comm systemd: reading directory 1block o
[19454.997891] EXT4-fs error (device nvmeon1p5): ext4_find_entry: 1658: inode #655370: comm systemd: reading directory 1block 0
[19454.397949] EXT4-fs error (device nvmeon1p5): ext4_find_entry: 1658: inode #2: comm systemd: reading directory Iblock o
[19454.420294] systemd[9797]: docker.service: Failed at step EXEC spawning /usr/bin/dockerd: Input/output error
119454.421508] systemd[1]: Failed to start Docker Application Container Engine.
[19456.207832 EXT4-fs error (device nvmeon1p5): __ext4_find_entry: 1658: inode #922214: comm gmain: reading directory Iblock 0
[19456.264664 EXT4-fs error (device nvme0n1p5): ext4_find_entry: 1658: inode #1187760: comm gmain: reading directory Iblock o
[13456.264765] EXT4-fs error (device Avme0n1p5): ext4_find_entry: 1658: inode #1187760: comm gmain: reading directory Iblock o
[19456.264816] EXT4-fs error (device nvme0n1p5): ext4_find_entry: 1658: inode #1187760: comm gmain: reading directory Iblock 0
[13456.467816] EXT4-fs error (device nvme0n1p5): ext4_find_entry: 1658: inode #1179649: comm thermald: reading directory Iblock 0
[13456.468150] EXT4-fs error (device nvmeon1p5): __ext4_find_entry: 1658: inode #1179649: comm thermald: reading directory lblock 0
113456.648475] systemd[1]: Failed to start Docker Application Container Engine.
113459.330449] EXT4-fs error (device nvme0n1p6): ext4_find_entry: 1658: inode #48896417: comm Daemon periodic: reading directory Iblock
[13459.330594] EXT4-fs error (device nvme0n1p6): ext4_find_entry: 1658: inode #48896417: comm Daemon periodic: reading directory Iblock
[13459.330680] EXT4-fs error (device nvmeon1p6): ext4_find_entry: 1658: inode #48896417: comm Daemon periodic: reading directory lblock
113459.420371] systemd [9799]: gpu-manager.service: Failed at step EXEC spawning /usr/bin/gpu-manager: Input/output error
113459.421974] systemd[1]: Failed to start Detect the available GPUS and deal with any system changes
113459.424728] systemd [9800]: gdm.service: Failed at step EXEC spawning /usr/share/gdm/generate-config: Input/output error
[13459.426019] systemd[1]: Failed to start GNOME Display Manager.
113459.647301] EXT4-fs error (device nvme0n1p5): ext4_find_entry: 1658: inode #1315384: comm systemd-logind: reading directory Iblock 0
113459.649171] EXT4-fs error (device nvme0n1p6): ext4_find_entry: 1658: inode #45744136: comm systemd: reading directory lblock 0
113459.649295] EXT4-fs error (device nvme0n1p6): ext4_find_entry: 1658: inode #45744134: comm systemd: reading directory lblock 0
113459.649318] EXT4-fs error (device nvmeon1p5): ext4_find_entry: 1658: inode #1315384: comm systemd-logind: reading directory lblock 0
113459.649359] EXT4-fs error (device nvmeon1p5): ext4_find_entry: 1658: inode #922214: comm systemd: reading directory lblock 0
113459.649472] EXT4-fs error (device nymeon1p5): ext4_find_entry:1658: inode #655370: comm systemd: reading directory Iblock 0
(13459.649594] EXT4-fs error (device nvmeon1p6): ext4_find_entry: 1658: inode #45744136: comm systemd: reading directory Iblack 0
[13459.649673] EXT4-fs error (device nymeon1p6): ext4_find_entry: 1658: inode #45744134: comm systemd: reading directory 1block 0
(13459.6497291 EXT4-fs error (device nymeon1p5); ext4_find_entry: 1658: inode #922214: comm systemd: reading directory lblock o
[18459.649787] EXT4-fs error (device nymeon1p5); ext4_find_entry:1658: inode #655370: comm systemd: reading directory lblock 0
[18459.649924] EXT4-fs error (device nymeon1p5):ext4_find_entry: 1658: Inode #655370: comm NetworkManager: reading directory lblock 0
(13459,650080) EXT4-fs error (device nymeonips); ext4 find_entry:1658: inode #655365: comm NetworkManager: reading directory lblock 0
(13459.650194] EXT4-fs error (device nymeonips): ext4_find_entry:1658: inode #2: comm NetworkManager: reading directory lblock 0
[18459.650626] EXT4-fs error (device nymeon1p5); ext4_find_entry:1658: inode #1315384: comm systemd-logind: reading directory lblock 0
[13460.664307] systemd [9801]: gpu-manager, service: Failed at step EXEC spawning /usr/bin/gpu-manager: Input/output error
[13460.665714] systemd[1]: Failed to start Detect the available GPUs and deal with any system changes.
[13460.668564] systemd [9802] gdm, service: Failed at step EXEC spawning /usr/share/gdm/generate-config: Input/output error
[13460.669827] systemd[1]: Failed to start GNOME Display Manager,
[13461.9285141 systemd [9803) gpu-manager, service: Failed at step EXEC spawning /usr/bin/gpu-manager: Input/output error
[13461.9300611 systemd[1]: Failed to start Detect the available GPUS and deal with any system changes.
[13461.932897] systemd [9804]: gdm.service: Failed at step EXEC spawning /usr/share/gdm/generate-config: Input/output error
ailed to start GNOME Display Manager.
r/bin/gpu-manager: Input/output error

ProblemType: Bug
DistroRelease: Ubuntu 22.10
Package: linux-image-5.19.0-21-generic 5.19.0-21.21
ProcVersionSignature: Ubuntu 5.19.0-21.21-generic 5.19.7
Uname: Linux 5.19.0-21-generic x86_64
ApportVersion: 2.23.1-0ubuntu3
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: anton 2702 F.... pipewire
                      anton 2705 F.... wireplumber
 /dev/snd/seq: anton 2702 F.... pipewire
CRDA: N/A
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Mon Oct 17 22:21:29 2022
InstallationDate: Installed on 2021-04-03 (562 days ago)
InstallationMedia: Ubuntu 21.04 "Hirsute Hippo" - Beta amd64 (20210331.1)
MachineType: ASUSTeK COMPUTER INC. ZenBook UX393EA_UX393EA
ProcFB: 0 i915drmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.19.0-21-generic root=UUID=026d3f6d-d78e-4225-b283-76118fb789a8 ro quiet splash vt.handoff=7
PulseList: Error: command ['pacmd', 'list'] failed with exit code 1: No PulseAudio daemon running, or not running as session daemon.
RelatedPackageVersions:
 linux-restricted-modules-5.19.0-21-generic N/A
 linux-backports-modules-5.19.0-21-generic N/A
 linux-firmware 20220923.gitf09bebf3-0ubuntu1
SourcePackage: linux
UpgradeStatus: Upgraded to kinetic on 2022-10-16 (1 days ago)
dmi.bios.date: 04/22/2022
dmi.bios.release: 5.19
dmi.bios.vendor: American Megatrends International, LLC.
dmi.bios.version: UX393EA.312
dmi.board.asset.tag: ATN12345678901234567
dmi.board.name: UX393EA
dmi.board.vendor: ASUSTeK COMPUTER INC.
dmi.board.version: 1.0
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: ASUSTeK COMPUTER INC.
dmi.chassis.version: 1.0
dmi.modalias: dmi:bvnAmericanMegatrendsInternational,LLC.:bvrUX393EA.312:bd04/22/2022:br5.19:svnASUSTeKCOMPUTERINC.:pnZenBookUX393EA_UX393EA:pvr1.0:rvnASUSTeKCOMPUTERINC.:rnUX393EA:rvr1.0:cvnASUSTeKCOMPUTERINC.:ct10:cvr1.0:sku:
dmi.product.family: ZenBook
dmi.product.name: ZenBook UX393EA_UX393EA
dmi.product.version: 1.0
dmi.sys.vendor: ASUSTeK COMPUTER INC.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This change was made by a bot.

[Anton Keks (anton-azib)]

It seems that some logs are written on resume before the freeze: /var/log/kern.log

Oct 23 20:42:29 aziber kernel: [ 1478.088305] Filesystems sync: 0.010 seconds
Oct 24 11:55:24 aziber kernel: [ 1478.355281] Freezing user space processes ... (elapsed 0.002 seconds) done.
Oct 24 11:55:24 aziber kernel: [ 1478.358121] OOM killer disabled.
Oct 24 11:55:24 aziber kernel: [ 1478.358124] Freezing remaining freezable tasks ... (elapsed 6.366 seconds) done.
Oct 24 11:55:24 aziber kernel: [ 1484.724770] printk: Suspending console(s) (use no_console_suspend to debug)
Oct 24 11:55:24 aziber kernel: [ 1485.095679] ACPI: EC: interrupt blocked
Oct 24 11:55:24 aziber kernel: [56249.613366] typec port0-partner: PM: parent port0 should not be sleeping
Oct 24 11:55:24 aziber kernel: [56249.799810] asus_wmi: Unknown key code 0xcf
Oct 24 11:55:24 aziber kernel: [56253.854934] ACPI: EC: interrupt unblocked
Oct 24 11:55:24 aziber kernel: [56254.819343] i915 0000:00:02.0: [drm] GuC firmware i915/tgl_guc_70.1.1.bin version 70.1
Oct 24 11:55:24 aziber kernel: [56254.819348] i915 0000:00:02.0: [drm] HuC firmware i915/tgl_huc_7.9.3.bin version 7.9
Oct 24 11:55:24 aziber kernel: [56254.822876] i915 0000:00:02.0: [drm] HuC authenticated
Oct 24 11:55:24 aziber kernel: [56254.822890] i915 0000:00:02.0: [drm] GuC submission disabled
Oct 24 11:55:24 aziber kernel: [56254.822891] i915 0000:00:02.0: [drm] GuC SLPC disabled
Oct 24 11:55:24 aziber kernel: [56255.040555] OOM killer enabled.
Oct 24 11:55:24 aziber kernel: [56255.040559] Restarting tasks ...
Oct 24 11:55:24 aziber kernel: [56255.044577] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
Oct 24 11:55:24 aziber kernel: [56255.047165] mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915])
Oct 24 11:55:24 aziber kernel: [56255.051527] done.
Oct 24 11:55:24 aziber kernel: [56255.059513] thermal thermal_zone7: failed to read out thermal zone (-61)
Oct 24 11:55:24 aziber kernel: [56255.070581] PM: suspend exit
Oct 24 11:55:24 aziber kernel: [56255.168618] audit: type=1400 audit(1666601724.319:107): apparmor="DENIED" operation="capable" class="cap" profile="/usr/sbin/cupsd" pid=25817 comm="cupsd" capability=12 capname="net_admin"

[Anton Keks (anton-azib)]

I tried mainline kernels in Ubuntu 22.10:

https://kernel.ubuntu.com/~kernel-ppa/mainline/v6.0.3/ - resume now works, but both resume and suspend are really slow (20+ seconds)

https://kernel.ubuntu.com/~kernel-ppa/mainline/v6.1-rc1/ - works as well, without the long delay

Probably something was finally fixed in the kernel.

Note: mainline 6.x kernels also log the following error:

[ 49.449840] Freezing user space processes ... (elapsed 0.002 seconds) done.
[ 49.452576] OOM killer disabled.
[ 49.452578] Freezing remaining freezable tasks ... (elapsed 2.760 seconds) done.
[ 52.216297] printk: Suspending console(s) (use no_console_suspend to debug)
[ 52.594630] ACPI: EC: interrupt blocked
[ 52.637203] ================================================================================
[ 52.637205] UBSAN: array-index-out-of-bounds in /home/kernel/COD/linux/drivers/acpi/acpica/dswexec.c:401:12
[ 52.637208] index -1 is out of range for type 'acpi_operand_object *[9]'
[ 52.637210] CPU: 0 PID: 240 Comm: kworker/u16:4 Tainted: G U OE 6.1.0-060100rc1-generic #202210162332
[ 52.637214] Hardware name: ASUSTeK COMPUTER INC. ZenBook UX393EA_UX393EA/UX393EA, BIOS UX393EA.312 04/22/2022
[ 52.637216] Workqueue: events_unbound async_run_entry_fn
[ 52.637224] Call Trace:
[ 52.637226] <TASK>
[ 52.637229] show_stack+0x4e/0x61
[ 52.637234] dump_stack_lvl+0x4a/0x6f
[ 52.637241] dump_stack+0x10/0x18
[ 52.637245] ubsan_epilogue+0x9/0x43
[ 52.637249] __ubsan_handle_out_of_bounds.cold+0x42/0x47
[ 52.637253] acpi_ds_exec_end_op+0x1ca/0x828
[ 52.637259] acpi_ps_parse_loop+0x58d/0x678
[ 52.637263] acpi_ps_parse_aml+0x2e6/0x59e
[ 52.637265] acpi_ps_execute_method+0x241/0x315
[ 52.637269] acpi_ns_evaluate+0x34c/0x515
[ 52.637273] acpi_evaluate_object+0x212/0x3a5
[ 52.637275] __acpi_power_off+0x26/0x70
[ 52.637279] acpi_power_off_unlocked+0x54/0x90
[ 52.637283] acpi_power_off_list.isra.0+0x43/0xa0
[ 52.637287] acpi_power_transition+0xab/0x100
[ 52.637291] acpi_device_set_power+0x319/0x470
[ 52.637295] ? _raw_spin_lock_irqsave+0xe/0x20
[ 52.637298] acpi_pci_set_power_state+0x98/0x150
[ 52.637303] pci_set_power_state+0x9f/0x220
[ 52.637306] ? acpi_pci_choose_state+0x26/0x50
[ 52.637309] pci_prepare_to_sleep+0x7e/0xd0
[ 52.637312] pci_pm_suspend_noirq+0x1f7/0x2c0
[ 52.637316] ? pci_pm_suspend_late+0x50/0x50
[ 52.637319] dpm_run_callback+0x63/0x160
[ 52.637322] __device_suspend_noirq+0x8f/0x290
[ 52.637325] async_suspend_noirq+0x23/0x70
[ 52.637327] async_run_entry_fn+0x30/0x130
[ 52.637331] process_one_work+0x222/0x400
[ 52.637334] worker_thread+0x50/0x3e0
[ 52.637337] ? process_one_work+0x400/0x400
[ 52.637339] kthread+0xe6/0x110
[ 52.637342] ? kthread_complete_and_exit+0x20/0x20
[ 52.637344] ret_from_fork+0x1f/0x30
[ 52.637350] </TASK>
[ 52.637351] ================================================================================
[ 60.734902] ACPI: EC: interrupt unblocked
[ 61.575520] pcieport 10000:e0:1d.0: can't derive routing for PCI INT A
[ 61.575528] nvme 10000:e1:00.0: PCI INT A: no GSI
[ 61.576151] i915 0000:00:02.0: [drm] ...

[Anton Keks (anton-azib)]

The failing stack already has a patch:
https://bugzilla.kernel.org/show_bug.cgi?id=216266