Ubuntu Server 22.04.3 randomly freezes

[Tushar]

Hi,
I have a physical machine on which I have installed Ubuntu Server 22.04.3 . It randomly freezes.
I read few forums and gathered previous boot report as well.

Other information as requested:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: linux-image-5.15.0-87-generic 5.15.0-87.97
ProcVersionSignature: Ubuntu 5.15.0-87.97-generic 5.15.122
Uname: Linux 5.15.0-87-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version k5.15.0-87-generic.
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/hwC0D2', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D10p', '/dev/snd/pcmC0D9p', '/dev/snd/pcmC0D8p', '/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/pcmC0D2c', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer'
Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer'
CasperMD5CheckResult: pass
CloudArchitecture: x86_64
CloudID: none
CloudName: none
CloudPlatform: none
CloudSubPlatform: config
Date: Sat Oct 21 02:41:33 2023
InstallationDate: Installed on 2023-09-19 (31 days ago)
InstallationMedia: Ubuntu-Server 22.04.3 LTS "Jammy Jellyfish" - Release amd64 (20230810)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Lsusb-t:
 /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 20000M/x2
 /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
MachineType: Gigabyte Technology Co., Ltd. H510M H
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.15.0-87-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
 linux-restricted-modules-5.15.0-87-generic N/A
 linux-backports-modules-5.15.0-87-generic N/A
 linux-firmware 20220329.git681281e4-0ubuntu3.21
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 03/10/2023
dmi.bios.release: 5.19
dmi.bios.vendor: American Megatrends International, LLC.
dmi.bios.version: F17
dmi.board.asset.tag: Default string
dmi.board.name: H510M H
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.asset.tag: Default string
dmi.chassis.type: 3
dmi.chassis.vendor: Default string
dmi.chassis.version: Default string
dmi.modalias: dmi:bvnAmericanMegatrendsInternational,LLC.:bvrF17:bd03/10/2023:br5.19:svnGigabyteTechnologyCo.,Ltd.:pnH510MH:pvr-CF:rvnGigabyteTechnologyCo.,Ltd.:rnH510MH:rvrx.x:cvnDefaultstring:ct3:cvrDefaultstring:skuDefaultstring:
dmi.product.family: H510 MB
dmi.product.name: H510M H
dmi.product.sku: Default string
dmi.product.version: -CF
dmi.sys.vendor: Gigabyte Technology Co., Ltd.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This change was made by a bot.

[gco (gcostes)]

Same thing here, but installed on a VPS at OVH.
Strangely enough, it freezes for no obvious reason (no special activities/events in syslog) at around 14:30 CET every 2 days.
The CPU load of the VPS increases linearly, while RAM usage decreases linearly from the time of the freeze.
From that time, even the console (TTY) is freezed.
Only hard reboot possible.

[gco (gcostes)]

Changing VM settings in /etc/sysctl.conf seems to have resolved freezes on my Ubuntu 22.04.3 kernel 5.15.0-88 server

I added the following at the end of /etc/sysctl.conf :

vm.min_free_kbytes = 2097152
vm.swappiness = 5
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
vm.dirty_writeback_centisecs = 500
vm.dirty_expire_centisecs = 3000

Then I ran sysctl -p /etc/sysctl.conf
... to perm save settings

vm.min_free_kbytes and vm.swappiness seems to be the main ones doing the trick.

These settings depend on your server config and main use so be sure to adapt accordingly.
Documention can be found here : https://sysctl-explorer.net/vm

I used <stress> tool to test new settings and <htop> to monitor cpu and mem alloc.
i.e. stress --vm 8 --vm-bytes 8G --timeout 300
to stress test mem up to 64GB using 8 vm workers spinning during 5 minutes.
For info, I have a 8GB swap

Before changing these settings (=> default values), my server freezed when stress testing it to the RAM limit not using the swap. After, it gently switched to swap when approaching the RAM limit without freezing.

Hope this helps someone else...

[Tushar (er-tushar-yahoo)]

Hi @gco,

Thanks for posting this. Although, your case is for VM and looks like the settings you have mentioned here are for the VM. I have a physical machine. Let me go through these docs and give it a try.

Thanks.