zfs icp has deselected all optimized aes & gcm impls

[Kent Ross]

In an upgrade from Jammy kernel 5.15.0-27-generic to 5.15.0-35-generic on x86_64 (AMD threadripper pro 39x5wx series), a 40x performance regression in the first read of cached writes to an encrypted dataset revealed that zfs is no longer configured to choose any implementations from advanced instruction sets:

$ grep . /sys/module/icp/parameters/*impl*
/sys/module/icp/parameters/icp_aes_impl:cycle [fastest] generic x86_64
/sys/module/icp/parameters/icp_gcm_impl:cycle [fastest] generic

With correct configuration, the output should read as follows:

$ grep . /sys/module/icp/parameters/*impl*
/sys/module/icp/parameters/icp_aes_impl:cycle [fastest] generic x86_64 aesni
/sys/module/icp/parameters/icp_gcm_impl:cycle [fastest] avx generic pclmulqdq

The immediate ill effects are the use of gcm_generic_mul instead of the dedicated instruction, consuming 50% CPU and slowing reads of data cached in ram to less than 20% of what they would be even reading directly from disk.

openzfs changed its configure process to detect cpu features differently recently to adapt to the kernel api change. It seems that the upstream changes that unexport the needed symbols and the downstream changes in openzfs that stop using them were not cherrypicked in sync.

https://github.com/openzfs/zfs/pull/13147
https://github.com/openzfs/zfs/pull/13236

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Nick O'Connor (nick-oconnor)]

I'm affected by this as well. This is not the first time this exact issue has occurred (see https://github.com/openzfs/zfs/issues/8259). I can confirm that the current issue is resolved in v2.1.4. Please merge v2.1.4 into the kernel ASAP. It would be awesome to have some testing around this!

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1977699

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Kent Ross (mumbleskates)]

Logs would not be relevant or helpful to this issue. The problem is completely described, and the solution is to update the ZFS module as in https://bugs.launchpad.net/bugs/1969482 .