Unable to run 4.14 stable kernel in 20.04 user space

[Terry Rudd]

Shuah Khan reported in IRC that she starting having problems with 4.14 only of the stable kernel after her upgrade to 20.04. 4.4 and 4.9 work just fine. She followed up with 4.19, 5.4 and 5.9 kernel, with no issue.

She suspects the issue is related to compress/decompress initramfs image. She tried all LZO, F2FS config options.

She reports that she made sure to have CONFIG_DECOMPRESS_LZ4 and CONFIG_F2FS_FS_LZ4 enabled. She has run into this once when she upgraded to 20.04, which she solved by enabling CONFIG_DECOMPRESS_LZ4 at that time. She is very willing to help debug.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1907156

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[dann frazier (dannf)]

What's the symptom?

[dann frazier (dannf)]

Note that we did switch to lz4 compression starting in 19.10. This is configurable via /etc/initramfs-tools/initramfs.conf, so you could workaround it by returning to gzip. Why 4.14.y would be a snowflake here isn't clear.

[Shuah Khan (shuah)]

Right. I see COMPRESS=lz4 in /etc/initramfs-tools/initramfs.conf. With the same config 4.9, 4.4 work just fine. Is CONFIG_F2FS_FS_LZO=y or CONFIG_F2FS_FS_LZ4=y in play by any chance?

It almost appears 4.14 boot sequence might be assuming decompressin. It just hangs after the first initramfs message and doesn't decompress

My 4.14 config is:
ONFIG_HAVE_KERNEL_LZ4=y
CONFIG_KERNEL_LZ4=y
CONFIG_RD_LZ4=y
CONFIG_SQUASHFS_LZ4=y
# CONFIG_PSTORE_LZ4_COMPRESS is not set
CONFIG_CRYPTO_LZ4=m
CONFIG_CRYPTO_LZ4HC=m
CONFIG_LZ4_COMPRESS=m
CONFIG_LZ4HC_COMPRESS=m
CONFIG_LZ4_DECOMPRESS=y
CONFIG_DECOMPRESS_LZ4=y

My 4.9 config is:
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_KERNEL_LZ4=y
CONFIG_RD_LZ4=y
CONFIG_SQUASHFS_LZ4=y
# CONFIG_PSTORE_LZ4_COMPRESS is not set
CONFIG_CRYPTO_LZ4=m
CONFIG_CRYPTO_LZ4HC=m
CONFIG_LZ4_COMPRESS=m
CONFIG_LZ4HC_COMPRESS=m
CONFIG_LZ4_DECOMPRESS=y
CONFIG_DECOMPRESS_LZ4=y

[Shuah Khan (shuah)]

More information. It doesn't appear to be compression related. Changing /etc/initramfs-tools/initramfs.conf to gzip didn't make a difference.

[dann frazier (dannf)]

It works fine for me:

ubuntu@ubuntu:~$ uname -a
Linux ubuntu 4.14.211 #3 SMP Wed Dec 9 00:40:44 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

I kept the default initramfs.conf w/ COMPRESS=lz4. I started with a defconfig and added what I needed to boot my (EFI-based) VM.

ubuntu@ubuntu:~$ sudo file /boot/initrd.img-4.14.211
/boot/initrd.img-4.14.211: LZ4 compressed data (v0.1-v0.9)

[Shuah Khan (shuah)]

I am testing on AMD Ryzen 7 4700G with Radeon Graphics system. I tried disabling CONFIG_FIRMWARE_IN_KERNEL. Anyway it doesn't appear to be compression/decompression issue. Maybe not even Ubuntu specific.

[dann frazier (dannf)]

ACK, I'll go ahead and close this bug then, but still happy to help if I can :)
I'd suggest adding some debugging command line arguments (debug, initcall_debug, etc) to see if you can get any more info about where it's hanging.