manual_add_modules is slow / consumes the most time

[Benjamin Drung]

I sprinkled mkinitramfs with "date -Ins" to see where most time is spent. I ran the [1] test case in a chroot on my laptop. mkinitramfs took 18.3 seconds. The most time consuming parts:

* 10.13 s (51.4%) auto_add_modules
* 7.20 s (36.5%) run_scripts_optional /usr/share/initramfs-tools/hooks
* 1.27 s (6.4%) final { cat; cpio; cpio|compress } > $outfile

The remaining 1.11 seconds are spread over the remaining parts.

Following hooks in /usr/share/initramfs-tools/hooks took the longest:

* 4.56s (63.3%) framebuffer
* 0.87s (12.1%) plymouth
* 0.81s (11.3%) cryptroot
* 0.23s (3.2%) lvm2
* 0.18s (2.5%) udev
* 0.17s (2.4%) mdadm

The remaining 0.38 seconds are spread over the remaining dozen of hooks.

The framebuffer hook just calls copy_modules_dir and manual_add_modules multiple times. auto_add_modules calls copy_modules_dir multiple times and manual_add_modules. copy_modules_dir calls find and then
manual_add_modules.

So most time will be spent in manual_add_modules. This function calls modprobe on the modules, copies the modules, and add_firmware on the firmwares from a "modinfo -F firmware" call.

Replacing manual_add_modules by a call to /usr/lib/dracut/dracut-install speeds up the initramfs building significantly. In the chroot on my laptop the mkinitramfs execution reduces from 18.3 seconds to 11.0 (40% reduction).

[1] schroot-wrapper -p linux-image-generic,initramfs-tools,zstd,busybox-initramfs,cryptsetup-initramfs,isc-dhcp-client,kbd,lvm2,mdadm,ntfs-3g,plymouth,plymouth-theme-spinner --ppa bdrung/ppa -c mantic -u root

[Benjamin Drung (bdrung)]

Attached the tested change (verified that the content of the initramfs stays the same).

I am not uploading this change, because dracut-core is in universe.

[Benjamin Drung (bdrung)]

More tuning can be done. The code can be found in the dracut-install branch in https://code.launchpad.net/~ubuntu-core-dev/ubuntu/+source/initramfs-tools/+git/initramfs-tools/+ref/dracut-install

And there is initramfs-tools 0.142ubuntu8bd1 in https://launchpad.net/~bdrung/+archive/ubuntu/ppa for testing.

[Launchpad Janitor (janitor)]

This bug was fixed in the package initramfs-tools - 0.142ubuntu12

---------------
initramfs-tools (0.142ubuntu12) mantic; urgency=medium

  * Change dracut-core dependency to newly created dracut-install package
    (LP: #2031304)
  * autopkgtest:
    - Explicitly specify supported architectures
    - Enable KVM if available (not available on Ubuntu runners)
    - log qemu parameters

initramfs-tools (0.142ubuntu11) mantic; urgency=medium

  * Use dracut-install in manual_add_modules to speed up mkinitramfs
    significantly (LP: #2031185)
  * Install tty/serial kernel modules. Dracut is doing the same. (LP: #2019435)
  * Harmonize kernel module installation with dracut's behavior (LP: #2031841):
    - Use the symbol regex from dracut-install to reduce the unnessary block
      and net kernel modules.
    - Install all usb/typec kernel modules instead of cherry-picking some.
    - Do not install ARM/RISCV specific modules on other architectures.

 -- Benjamin Drung <email address hidden> Wed, 23 Aug 2023 10:58:40 +0200