Module aesni_intel not being loaded before mounting LVM2 stacked on LUKS

[Filiprino]

Hi,
I've made an installation of Ubuntu 11.10 on a system with AES-NI instructions only to find that the aesni_intel module is not being loaded before mounting the root file system, leaving the system with software encryption instead of using the hardware capabilities.

The problem I think it's due to the modules not existing on the initramfs. Naturally, aes_x86_64 and cryptd modules aren't loaded too.

Having the kernel compiled with options
CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_AES_X86_64=y
CONFIG_CRYPTO_AES_NI_INTEL=y

instead of

CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_AES_X86_64=m
CONFIG_CRYPTO_AES_NI_INTEL=m

as it appears on "config-3.0.0-14-generic" in the /boot directory should be OK too.

I can manually load those modules after the system boots, but then it's not useful for my fully encrypted filesystem.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-image (not installed)
ProcVersionSignature: Ubuntu 3.0.0-14.23-generic 3.0.9
Uname: Linux 3.0.0-14-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 1.23-0ubuntu4
Architecture: amd64
ArecordDevices:
 **** List of CAPTURE Hardware Devices ****
 card 0: PCH [HDA Intel PCH], device 0: ALC269VB Analog [ALC269VB Analog]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: raul 1706 F.... pulseaudio
CRDA: Error: [Errno 2] No existe el archivo o el directorio
Card0.Amixer.info:
 Card hw:0 'PCH'/'HDA Intel PCH at 0xdf000000 irq 49'
   Mixer name : 'Intel CougarPoint HDMI'
   Components : 'HDA:10ec0269,10431063,00100100 HDA:80862805,80860101,00100000'
   Controls : 22
   Simple ctrls : 12
Date: Sat Dec 24 14:06:12 2011
HibernationDevice: RESUME=UUID=968e95f3-010c-4925-976d-6af33e7c279b
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64 (20111011)
MachineType: ASUSTeK Computer Inc. N53SN
ProcEnviron:
 PATH=(custom, no user)
 LANG=es_ES.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.0.0-14-generic root=/dev/mapper/ubunturoot-raiz ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.0.0-14-generic N/A
 linux-backports-modules-3.0.0-14-generic N/A
 linux-firmware 1.60
SourcePackage: linux
StagingDrivers: mei
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 08/10/2011
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: N53SN.208
dmi.board.asset.tag: ATN12345678901234567
dmi.board.name: N53SN
dmi.board.vendor: ASUSTeK Computer Inc.
dmi.board.version: 1.0
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: ASUSTeK Computer Inc.
dmi.chassis.version: 1.0
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrN53SN.208:bd08/10/2011:svnASUSTeKComputerInc.:pnN53SN:pvr1.0:rvnASUSTeKComputerInc.:rnN53SN:rvr1.0:cvnASUSTeKComputerInc.:ct10:cvr1.0:
dmi.product.name: N53SN
dmi.product.version: 1.0
dmi.sys.vendor: ASUSTeK Computer Inc.

[Filiprino (filiprino)]

I've put temporaly the modules on /etc/initramfs-tools/modules and then updated the initramfs of the latest kernel installed:

# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax: module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
#
# Examples:
#
# raid1
# sd_mod

cryptd
aes_x86_64
aesni_intel

Now they are being loaded on startup as should be. Prove of that is that the aesni_intel module now can't be unloaded due to it being in use:

raul@n53sn:/etc/initramfs-tools$ sudo rmmod aesni_intel
[sudo] password for raul:
ERROR: Module aesni_intel is in use
raul@n53sn:/etc/initramfs-tools$

[Filiprino (filiprino)]

Where I wrote "Prove" should say "Proof". My current lsmod output with aesni loaded from the beginning:

Module Size Used by
bnep 18436 2
rfcomm 47946 0
parport_pc 36962 0
ppdev 17113 0
binfmt_misc 17540 1
snd_hda_codec_hdmi 32040 1
snd_hda_codec_realtek 330769 1
joydev 17693 0
arc4 12529 2
asus_nb_wmi 12517 0
asus_wmi 20035 1 asus_nb_wmi
sparse_keymap 13890 1 asus_wmi
uvcvideo 72711 0
videodev 92992 1 uvcvideo
v4l2_compat_ioctl32 17083 1 videodev
snd_hda_intel 33390 2
snd_hda_codec 104802 3 snd_hda_codec_hdmi,snd_hda_codec_realtek,snd_hda_intel
snd_hwdep 13668 1 snd_hda_codec
snd_pcm 96714 3 snd_hda_codec_hdmi,snd_hda_intel,snd_hda_codec
snd_seq_midi 13324 0
snd_rawmidi 30547 1 snd_seq_midi
snd_seq_midi_event 14899 1 snd_seq_midi
psmouse 73882 0
snd_seq 61896 2 snd_seq_midi,snd_seq_midi_event
serio_raw 13166 0
iwlagn 314213 0
snd_timer 29991 2 snd_pcm,snd_seq
snd_seq_device 14540 3 snd_seq_midi,snd_rawmidi,snd_seq
btusb 18600 0
bluetooth 166112 11 bnep,rfcomm,btusb
mac80211 462092 1 iwlagn
snd 68266 14 snd_hda_codec_hdmi,snd_hda_codec_realtek,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_pcm,snd_rawmidi,snd_seq,snd_timer,snd_seq_device
cfg80211 199587 2 iwlagn,mac80211
soundcore 12680 1 snd
snd_page_alloc 18529 2 snd_hda_intel,snd_pcm
mei 41480 0
lp 17799 0
parport 46562 3 parport_pc,ppdev,lp
usbhid 47198 0
hid 95463 1 usbhid
dm_crypt 23199 1
aesni_intel 55586 50
aes_x86_64 17208 1 aesni_intel
cryptd 20530 17 aesni_intel
i915 566827 3
r8169 52788 0
nouveau 728662 0
ttm 76805 1 nouveau
drm_kms_helper 42558 2 i915,nouveau
ahci 26002 2
libahci 26861 1 ahci
drm 236290 6 i915,nouveau,ttm,drm_kms_helper
xhci_hcd 82820 0
mxm_wmi 12979 1 nouveau
i2c_algo_bit 13423 2 i915,nouveau
video 19412 2 i915,nouveau
wmi 19256 2 asus_wmi,mxm_wmi

[Joseph Salisbury (jsalisbury)]

This config change can be reviewed for Precise.

[xor (xor)]

This also applies to encrypted root using the following configuration chosen during 11.10 setup:

- Partition disks - manual:
 sda:
  new partition table: yes
  parititions sorted by position on disk:
   partition 1 - 1GB, Name: <empty>, Use as: Reserved BIOS boot area, Bootable flag: off
   partition 2 - 4GB, Name: <empty>, Use as: Physical Volume for raid, Bootable flag: off
   partition 3 - remaining space (3TB), Name: <empty>, Use as: Physical Volume for raid, Bootable flag: off
 sdb:
  new partition table: yes
  parititions sorted by position on disk:
   partition 1 - 1GB, Name: <empty>, Use as: Reserved BIOS boot area, Bootable flag: off
   partition 2 - 4GB, Name: <empty>, Use as: Physical Volume for raid, Bootable flag: off
   partition 3 - remaining space (3TB), Name: <empty>, Use as: Physical Volume for raid, Bootable flag: off
 sdc:
  new partition table: yes
  parititions sorted by position on disk:
   partition 1 - 1GB, Name: <empty>, Use as: Reserved BIOS boot area, Bootable flag: off
   partition 2 - 4GB, Name: <empty>, Use as: Physical Volume for raid, Bootable flag: off
   partition 3 - remaining space (3TB), Name: <empty>, Use as: Physical Volume for raid, Bootable flag: off

 select "Configure software RAID":
  Create MD device - RAID1, active devices: 3, spare devices: 0, using the 4GB disks: sda2, sdb2, sdc2
  Create MD device - RAID5, active devices: 3, spare devices: 0, using the 3TB disks: sda3, sdb3, sdc3

 back to partitioning menu, now configure the raid devices' usage:
  under "RAID1 device #0 (4GB)" select part "#1 4.0GB":
    use as: ext3, mount point: /boot, mount options: noatime, label: none, reserved block: 5%, typical usage: standard

  RAID5 device #1, configure "#1 6.0TB":
    use as: physical volume for encryption,

    encryption parameters - leave those at default if you are no mathematician:
     encryption method: device-mapper (dm-crypt), encryption: aes, key size: 256, iv algorithm: cbc-essiv:sha256, encryption key: passphrase, erasa data: NO if disks were randomized before, YES otherwise

 select "Configure encrypted volumes:"
   select "Create encrypted volumes" - devices to encrypt: /dev/md1
   now select "Finish" and enter the passphrase when asked

 back to the partitioning menu again:
  NOW notice the follwing bug in the partitioner: the ext3 raid device for "/boot" is NOT marked as being used for "/boot" anymore!
  - correct its parameters, now you must also set "Format the partition: yes" because it was formated before when we first configured it and we want it to be clean.

  under "Enccrypted volume (md1_crypt)" select part "#1 6.0TB":
   use as: xfs, mount point: /, mount options: noatime, label: none

  finish partitioning and write changes to disk. boot the system when the raid becomes degraded: yes. return to the partitioning menu to create swap: no. write changes to disk: yes.

[David Holmer (odinguru)]

Please see https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1058896 for a proposed patch to fix this.

[David Holmer (odinguru)]

Moving my patch and discussion here instead of on the the duplicate bug.

In the other bug Steve Langesek (vorion) wrote:
Cryptsetup should not be manually loading any modules, the kernel is supposed to take care of this automatically for us (and this works for me). Reassigning to the kernel package.

Thanks for looking at my proposed patch.

I did some checking to see why the kernel is not auto-loading aesni-intel.ko in this case.

It looks like the initrd modules.alias is correct as it contains these lines:
alias aes aes_x86_64
alias aes aesni_intel

I believe this will load aesni_intel.ko if the command "modprobe aes" is run by the kernel. However, I do not believe the kernel ever runs this command.

Here is my analysis of the related kernel code:
* start with drivers/md/dm-crypt.c
* which calls crypto_alloc_cipher() in include/linux/crypto.h
* which calls crypto_alloc_base() in crypto/api.c
* which calls crypto_alg_mod_lookup()
* which calls crypto_larval_lookup()
* which calls crypto_alg_lookup()
* crypto_alg_lookup() returns the best matching algorithm currently loaded into the kernel, if any.
* crypto_larval_lookup() then calls request_module() if and only if crypto_alg_lookup() did not find a match.
* crypto_larvel_lookup() returns algorithm back through stack to start

Essentially, the kernel will not issue a "modprobe aes" on behalf of dm-crypt.ko if there is already an algorithm loaded that implements AES.

The default Ubuntu 12.04 kernel configuration (3.2.0-31-generic) contains the following:
CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_AES_X86_64=m
CONFIG_CRYPTO_AES_NI_INTEL=m
CONFIG_CRYPTO_DEV_PADLOCK_AES=m

So the generic implementation of "aes" is built-in to the kernel, but the higher priority implementations are built as modules.

As a result, the combination of the kernel auto-loading logic with the Ubuntu kernel configuration seems to be what results in the issue.

I view this as a serious issue as it significantly affects the performance of the SSD on my laptop (and likely would of anyone else with a similar setup). With the Ubuntu default mode of 256-bit aes-cbc-essiv:sha256, my average read performance as measured by Disk Utility is limited to 155.3 Mb/s. When this issue is fixed, I get an over three-fold improvement to 537.5 Mb/s, which is essentially the same as the unencrypted speed (538.3 Mb/s). Unnecessarily large performance hits like this will slow the adoption of full-disk encryption.

I see a few potential ways to fix this issue each with pros and cons:

1) Change the kernel so that it always issues "modprobe aes"
pro: keeps responsibility with loading the correct drivers in the kernel, where one might argue it rightfully belongs.
con: this would mean that EVERY call to crypto_alloc_* would result in a user space execution of "modprobe aes", which is potentially a serious performance issue and unlikely to be accepted by kernel developers.

2) Change the Ubuntu kernel configuration so that CONFIG_AES=m
pro: this would cause the kernel to execute "modprobe aes" once, which should load in the correct driver.
con: I haven't explicitly checked, but I would suspect CONFIG_AES=y ...

[Devin Lane (t-devin)]

I've been searching for this issue off and on for a couple years, presumably using the wrong search terms as I only just now found this bug posting. Applying the temporary fix from #2 worked fine for me and I now see correct crypto SSD speeds.

Is there a reason this hasn't been fixed? Its astonishing that such a critical bug with an easy workaround hasn't been fixed in over two years. All this does is make Ubuntu look slow on modern hardware. Best of all, it's very difficult to determine that this is the source of the problem, as all the config looks correct, the correct modules are installed, and there's no tool that tells you what crypto backend is actually being used.

Please patch this once and for all.

[gdm85 (gdm85)]

@t-devin and that was 2014. Bug is still present in 2018.

Additionally to the mentioned modules cryptd, aes_x86_64 and aesni_intel, I had to add the dm_crypt module.