Change kernel compression method to improve boot speed

[Seth Forshee]

Colin King has done some analysis of kernel boot speed using different kernel compression methods. Results for x86 are at:

https://kernel.ubuntu.com/~cking/boot-speed-eoan-5.3/kernel-compression-method.txt
https://kernel.ubuntu.com/~cking/boot-speed-eoan-5.3/boot-speed-compression-5.3-rc4.ods

Testing of s390 gave the following:

GZIP 31528972
LZ4 192348049
LZO 85990145

From Colin: "I used the monotonic TOD timer using the stckf opcode to fetch a 64 bit time value. Not sure how this maps to 'real time' in seconds."

Conclusion: We should switch x86 to LZ4 and s390 to LZO. PPC and ARM do not support LZO or LZ4, so we will stick with gzip there.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1840934

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Dimitri John Ledkov (xnox)]

Should we match these defaults for initramfs too?

At the moment I have switched to lz4 across the board, but maybe it does make sense to use lzo on s390x and gzip on armhf, etc.

Should I escalate to IBM supporting lz4 on ppc64le?

[Seth Forshee (sforshee)]

I don't think Colin collected any data on initramfs compression, but I subscribed Colin to the bug so he can comment.

[Colin Ian King (colin-king)]

@Dimitri, so you set the defaults w/o any benchmarking stats?

I'll see if I can get some time to check what the sane options are per-arch.

Colin

[Colin Ian King (colin-king)]

@Dimitri,

Seems that for initramfs lz4 makes a lot of sense, see:

https://kernel.ubuntu.com/~cking/boot-speed-eoan-5.3/boot-speed-initramfs-decompression-eoan.ods

The load times for LZ4 is slower than the previous default, however, the decompression time makes up for this unless one is booting off really slow media sub-5400 RPM HDD such as slow flash.

So, the LZ4 default looks sane to me, lets see how it works out for Eaon.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 5.2.0-15.16

---------------
linux (5.2.0-15.16) eoan; urgency=medium

  * Packaging resync (LP: #1786013)
    - [Packaging] resync getabis

  * Change kernel compression method to improve boot speed (LP: #1840934)
    - [Packaging] add build dependencies for compression algorithms

linux (5.2.0-14.15) eoan; urgency=medium

  * eoan/linux: 5.2.0-14.15 -proposed tracker (LP: #1841159)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync getabis

  * Eoan update: v5.2.9 upstream stable release (LP: #1841110)
    - Revert "PCI: Add missing link delays required by the PCIe spec"
    - IIO: Ingenic JZ47xx: Set clock divider on probe
    - iio: cros_ec_accel_legacy: Fix incorrect channel setting
    - iio: imu: mpu6050: add missing available scan masks
    - iio: adc: gyroadc: fix uninitialized return code
    - iio: adc: max9611: Fix misuse of GENMASK macro
    - staging: gasket: apex: fix copy-paste typo
    - staging: wilc1000: flush the workqueue before deinit the host
    - staging: android: ion: Bail out upon SIGKILL when allocating memory.
    - Staging: fbtft: Fix probing of gpio descriptor
    - Staging: fbtft: Fix reset assertion when using gpio descriptor
    - crypto: ccp - Fix oops by properly managing allocated structures
    - crypto: ccp - Add support for valid authsize values less than 16
    - crypto: ccp - Ignore tag length when decrypting GCM ciphertext
    - driver core: platform: return -ENXIO for missing GpioInt
    - usb: usbfs: fix double-free of usb memory upon submiturb error
    - Revert "USB: rio500: simplify locking"
    - usb: iowarrior: fix deadlock on disconnect
    - sound: fix a memory leak bug
    - mmc: cavium: Set the correct dma max segment size for mmc_host
    - mmc: cavium: Add the missing dma unmap when the dma has finished.
    - loop: set PF_MEMALLOC_NOIO for the worker thread
    - bdev: Fixup error handling in blkdev_get()
    - Input: usbtouchscreen - initialize PM mutex before using it
    - Input: elantech - enable SMBus on new (2018+) systems
    - Input: synaptics - enable RMI mode for HP Spectre X360
    - coresight: Fix DEBUG_LOCKS_WARN_ON for uninitialized attribute
    - perf annotate: Fix s390 gap between kernel end and module start
    - perf db-export: Fix thread__exec_comm()
    - perf record: Fix module size on s390
    - x86/purgatory: Do not use __builtin_memcpy and __builtin_memset
    - x86/purgatory: Use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS
    - genirq/affinity: Create affinity mask for single vector
    - gfs2: gfs2_walk_metadata fix
    - usb: host: xhci-rcar: Fix timeout in xhci_suspend()
    - usb: yurex: Fix use-after-free in yurex_delete
    - usb: typec: ucsi: ccg: Fix uninitilized symbol error
    - usb: typec: tcpm: free log buf memory when remove debug file
    - usb: typec: tcpm: remove tcpm dir if no children
    - usb: typec: tcpm: Add NULL check before dereferencing config
    - usb: typec: tcpm: Ignore unsupported/unknown alternate mode requests
    - can: rcar_canfd: fix possible IRQ storm on high load
    - can: flexcan: fix stop mode acknowledgment
    - can: flexcan: fix an use-after-free in flexcan_setup...

[Colin Ian King (colin-king)]

Also enable LZ4 for s390x as IBM has provided us with some positive feedback about using this.

[Colin Ian King (colin-king)]

From "Comment bridged from LTC Bugzilla" in a bug discussion:

"FWIW, I verified this on z14, and there clearly lz4 is (as expected) the fastest decompression algorithm.

With vanilla 5.3-rc6 and defconfig I get the following kernel uncompression times:
lzo: 27us
lz4: 24us

An initrd (uncompressed size ~55MB) gets these uncompression times:
lzo: 62us
lz4: 49us

So I'd clearly vote to switch to lz4 on s390 as well."

Also:

"I also instrumented the kernel code to only measure the time to decompress the kernel. If its stckf or stcke doesn't matter in this case.
Note that if you shift a tod clock value 12 bits to the right will give you microseconds. (All numbers I posted were actually milliseconds not microseconds by the way).

I measured both runs (z13 + z14) when running within z/VM and IPL'ed from the punch card reader.

Times used for decompressing the initrd were just extracted from dmesg; no kernel instrumentation required here, since there are two messages provided before and after initrd decompression.

Find below an extract of the patch to measure decompression time.

diff --git a/arch/s390/boot/startup.c b/arch/s390/boot/startup.c
index 7b0d054..cee3d97 100644
--- a/arch/s390/boot/startup.c
+++ b/arch/s390/boot/startup.c
@@ -146,7 +146,10 @@ void startup_kernel(void)
}

if (!IS_ENABLED(CONFIG_KERNEL_UNCOMPRESSED)) {
+ start = get_tod_clock();
img = decompress_kernel();
+ end = get_tod_clock();
+ time = (end - start) >> 12;
memmove((void *)vmlinux.default_lma, img, vmlinux.image_size);
} else if (__kaslr_offset)
memcpy((void *)vmlinux.default_lma, img, vmlinux.image_size);
..."

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted linux-gcp-5.3 (5.3.0-1008.9~18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

linux-gcp-5.3/unknown (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-gcp-5.3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!