Comment 0 for bug 2070261

I noticed performance issues on the server running some backend applications after updating OS packages with Ubuntu Jammy. After further investigation, the slowness was caused by disk IO.

After the update, running on kernel 5.15.0-112, the disk writes take about twice as long. I can reproduce this by changing the GRUB configuration to boot into kernel 5.15.0-107, where the writes are consistently slower. See below.

# uname -a
Linux dev03 5.15.0-112-generic #122-Ubuntu SMP Thu May 23 07:48:21 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
# time dd if=/dev/urandom of=deleteme bs=1024 count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB, 977 MiB) copied, 6.54711 s, 156 MB/s

real 0m6.782s
user 0m0.192s
sys 0m5.968s

# uname -a
Linux dev03 5.15.0-107-generic #117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
# time dd if=/dev/urandom of=deleteme bs=1024 count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB, 977 MiB) copied, 3.77936 s, 271 MB/s

real 0m3.781s
user 0m0.132s
sys 0m3.648s

These times are with NVMe drives on ext4 filesystem. I observe the same performance difference for ZFS filesystem, and also with HDDs (spinning drives). The disk performance is also worse for 6.5.0-41 kernel. I also tried 6.5.0-35 and linux-image-generic-hwe-22.04 - same problem - writes are slow.

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: linux-image-5.15.0-112-generic 5.15.0-112.122
ProcVersionSignature: Ubuntu 5.15.0-112.122-generic 5.15.152
Uname: Linux 5.15.0-112-generic x86_64
NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
AlsaDevices:
 total 0
 drwxr-xr-x 2 root root 60 Jun 24 08:23 by-path
 crw-rw---- 1 root audio 116, 2 Jun 24 08:23 controlC0
 crw-rw---- 1 root audio 116, 1 Jun 24 08:23 seq
 crw-rw---- 1 root audio 116, 33 Jun 24 08:23 timer
AlsaVersion: Advanced Linux Sound Architecture Driver Version k5.15.0-112-generic.
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer'
Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer'
CasperMD5CheckResult: unknown
Date: Mon Jun 24 10:24:44 2024
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
 Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
 Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Lsusb-t:
 /: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
 /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
 /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
 /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.15.0-112-generic root=UUID=3bff164a-d728-43bd-a926-ad3be6ff4715 ro consoleblank=0 systemd.show_status=true nomodeset consoleblank=0
RelatedPackageVersions:
 linux-restricted-modules-5.15.0-112-generic N/A
 linux-backports-modules-5.15.0-112-generic N/A
 linux-firmware 20220329.git681281e4-0ubuntu3.31
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/09/2020
dmi.bios.release: 5.14
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: L1.02W
dmi.board.name: B450D4U-V1L
dmi.board.vendor: ASRockRack
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: To Be Filled By O.E.M.
dmi.chassis.version: To Be Filled By O.E.M.
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrL1.02W:bd07/09/2020:br5.14:svnHetzner:pn:pvr1.0:rvnASRockRack:rnB450D4U-V1L:rvr:cvnToBeFilledByO.E.M.:ct3:cvrToBeFilledByO.E.M.:sku:
dmi.product.version: 1.0
dmi.sys.vendor: Hetzner