kernel cannot find lz4hc and z3fold modules on boot

[Ken Sharp]

I've been using zswap without any issues for a while, but fairly recently (can't be certain when) the kernel became unable to find the modules for lz4hc compression and the z3fold zpool.

$ egrep 'lz4hc|z3fold' /var/log/syslog
Jan 25 06:35:11 vps-5d014c2f systemd-modules-load[127]: Inserted module 'lz4hc'
Jan 25 06:35:11 vps-5d014c2f systemd-modules-load[127]: Inserted module 'z3fold'
Jan 25 06:35:11 vps-5d014c2f kernel: [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.5.0-15-generic root=PARTUUID=948a082e-cfd1-43af-bc7b-3e0b9b20370e ro console=tty1 console=ttyS0 zswap.enabled=1 zswap.compressor=lz4hc zswap.max_pool_percent=60 zswap.zpool=z3fold transparent_hugepage=always panic=-1
Jan 25 06:35:11 vps-5d014c2f kernel: [ 0.020859] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.5.0-15-generic root=PARTUUID=948a082e-cfd1-43af-bc7b-3e0b9b20370e ro console=tty1 console=ttyS0 zswap.enabled=1 zswap.compressor=lz4hc zswap.max_pool_percent=60 zswap.zpool=z3fold transparent_hugepage=always panic=-1
Jan 25 06:35:11 vps-5d014c2f kernel: [ 1.128588] zswap: compressor lz4hc not available, using default lzo
Jan 25 06:35:11 vps-5d014c2f kernel: [ 1.130368] zswap: zpool z3fold not available, using default zbud

But the modules are in the initrd (as they always have been):

$ lsinitramfs /boot/initrd.img | grep -E 'lz4hc|z3fold'
usr/lib/modules/6.2.0-37-generic/kernel/crypto/lz4hc.ko
usr/lib/modules/6.2.0-37-generic/kernel/lib/lz4/lz4hc_compress.ko
usr/lib/modules/6.2.0-37-generic/kernel/mm/z3fold.ko

I'm at a complete loss. If the modules are there, why can't the kernel find them?

Both modules can be loaded (obviously from a different location – not the initrd) and the relevant parameters changed with startup scripts but this is a workaround not a solution.

From: https://askubuntu.com/q/1234721/170177

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: linux-image-6.2.0-37-generic 6.2.0-37.38~22.04.1
ProcVersionSignature: Ubuntu 6.2.0-37.38~22.04.1-generic 6.2.16
Uname: Linux 6.2.0-37-generic x86_64
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
CasperMD5CheckResult: unknown
CloudArchitecture: x86_64
CloudID: openstack
CloudName: openstack
CloudPlatform: openstack
CloudSubPlatform: metadata (http://169.254.169.254)
Date: Sun Dec 10 11:22:41 2023
Ec2AMI: ami-0008be8e
Ec2AMIManifest: FIXME
Ec2AvailabilityZone: nova
Ec2InstanceType: vps2020-starter-1-2-20
Ec2Kernel: unavailable
Ec2Ramdisk: unavailable
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: linux-signed-hwe-6.2
UpgradeStatus: Upgraded to jammy on 2022-08-15 (482 days ago)

[Ken Sharp (kennybobs)]

New kernel, no change.

[Ken Sharp (kennybobs)]

I tried again in Qemu with 16.04, 20.04, 22.04 and 24.04 and none of them see this issue.

So I tried with Jammy on LXD and the issue does present itself. So the issue appears to be with the UEC images.

I also tried Mantic on LXD and it works fine. So something has been fixed between Jammy and Mantic. I can test others if it will help.

If the documentation is up to date --> cloud-images.

[Ken Sharp (kennybobs)]

Noble on LXD works correctly too.

[Ken Sharp (kennybobs)]

Works fine on Debian Bookworm on GCP too.