Grub 2 fails to boot a kernel on a luks encrypted volume with Secure Boot enabled

[Chris Marks]

Booting in UEFI Secure Boot requires that all code loaded up to and including the OS kernel be signed. This includes all grub modules. This is accomplished by including selected modules in a single signed binary. However, the modules required for grub to use an encrypted volume have been omitted from the binary package and therefore Ubuntu cannot boot from an encrypted volume with Secure Boot enabled. This can be corrected as follows.

The debian/build-efi-images file needs to have lines 136-140 changed from:

GRUB_MODULES="$CD_MODULES
 lvm
 mdraid09
 mdraid1x
 "

to:

GRUB_MODULES="$CD_MODULES
 cryptodisk
 gcry_arcfour
 gcry_blowfish
 gcry_camellia
 gcry_cast5
 gcry_crc
 gcry_des
 gcry_dsa
 gcry_idea
 gcry_md4
 gcry_md5
 gcry_rfc2268
 gcry_rijndael
 gcry_rmd160
 gcry_rsa
 gcry_seed
 gcry_serpent
 gcry_sha1
 gcry_sha256
 gcry_sha512
 gcry_tiger
 gcry_twofish
 gcry_whirlpool
 luks
 lvm
 mdraid09
 mdraid1x
 "

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Marcin (marcincelebucki)]

This bug is also attached for any other grub2 module, example for: loopback
"error: Secure Boot forbids loading module from (hd0, msdos1)/EFI/ubuntu/x86_64-efi/loopback.mod."

[Chris Marks (christopher-l-marks)]

This is not a duplicate of bug #1062623. That other bug has to do with grub install and setting some switches to ease installation. This bug has to do with the modules that are included in the signed grub binary without which an encrypted boot partition cannot be loaded if the UEFI BIOS is set to require Secure Boot.

[Nazar Mokrynskyi (nazar-pc)]

For me the issue is even deeper.
I'm not striving for Secure Boot, just for UEFI with encrypted `/boot`.

However, even when `/boot/grub/grub.cfg` contains lines with `insmod luks` and `insmod cryptodisk` (I've tried even both `GRUB_ENABLE_CRYPTODISK=y` and `GRUB_PRELOAD_MODULES="luks cryptodisk"` at the same time), resulting `/boot/efi/EFI/ubuntu/grubx64.efi` (on ESP partition, of course) doesn't have those modules embedded and thus falls back to console without possibility to actually boot the system.

I feel like modules bundling is broken, since, for instance, BTRFS driver is always bundled independently from whether I use BTRFS or ext4.

[Chris Marks (christopher-l-marks)]

@Nazar, Are you saying that the modules are not on your ESP? If so, they need to be there. Mine are on the ESP at EFI/ubuntu/x86_64-efi/.

[Nazar Mokrynskyi (nazar-pc)]

No, they are not on ESP (I mean files like luks.mod, etc.), my assumption is that those files should be bundled into `grubx64.efi`, otherwise what is the point of having options like `GRUB_PRELOAD_MODULES`.

Is my assumption wrong and it works in a different way? I'd really like to avoid keeping anything besides `EFI\ubuntu\grubx64.efi` and `EFI\ubuntu\grub.cfg` on ESP partition.

[Chris Marks (christopher-l-marks)]

I'm not an expert on what the GRUB_PRELOAD_MODULES is used for, but I know that it only specifies that grub should load the specified modules as early as possible in the boot process. AFAIK, the modules you need to actually load and handoff to your chosen kernel must be in the ESP.

[Nazar Mokrynskyi (nazar-pc)]

I'm not an expert either. I'm just wondering why some modules are there and others not. I think it makes sense to bundle everything that is explicitly used in `/boot/grub/grub.cfg` automatically.

[Nazar Mokrynskyi (nazar-pc)]

I've tried to `insmod luks` manually (I've copied luks.mod, cryptodisk.mod and procfs.mod to ESP for this purpose).
2 issues here:
1) It is trying to find modules in `/@/boot/grub/*`, while that will clearly be inaccessible until decrypted (thud modules should be bundled with grubx64.efi
2) When manually loading necessary modules, `search.fs_uuid` from `ESP/EFI/ubuntu/grub.cfg` still results in `error: no such device: <UUID>`

No password prompt at any step.

I believe there should be a way to boot the system with encrypted `/boot`.

[Nazar Mokrynskyi (nazar-pc)]

Updated test setup to current 17.04 and after manual loading necessary modules alongside with `cryptomount -a` I was finally able to see `Ubuntu`, but didn't went any further because of https://forums.virtualbox.org/viewtopic.php?f=3&t=76791

[Nick B. (futurepilot)]

Just encountered this problem when trying to setup a fully encrypted system. I rebuilt the grub2 package with the changes listed in the first comment and tested in VirtualBox in UEFI mode and it does work.

You can't simply copy the modules to the ESP partition because it a) won't work with secure boot and b) grub seems to be hardcoded to try to load them from the /boot/grub/ directory which isn't accessible at that point in the boot process and there doesn't appear to be a way to specify them manually.

Can these extra modules please get included in the grubx64.efi binary?

[Nazar Mokrynskyi (nazar-pc)]

Yes, Secure Boot will not work, however, grubx64.efi happily uses grub.cfg that is placed in the same directory and you can instruct GRUB to load modules from other location, here is my config:
search.fs_uuid E495-1F0C boot
set prefix=($boot)'/EFI/ubuntu'
insmod luks
insmod gcry_sha256
cryptomount -u 739967f19770470aa0318d8b8bcdb350
search.fs_uuid 5170aca4-061a-4c6c-ab00-bd7fc8ae6030 root cryptouuid/739967f19770470aa0318d8b8bcdb350
set prefix=($root)'/root/boot/grub'
configfile $prefix/grub.cfg

[kay (kay-diam)]

If signed grub allows to load unsigned modules from encrypted partition - this is a high security issue. Spoofed module can store the decryption password.

[kay (kay-diam)]

s/encrypted/unencrypted efi/

[Phillip Susi (psusi)]

The whole point is that you can't do that when secure boot is enabled.

[Andreas Oberritter (mtdcr)]

Besides including the required modules into the binary, which would be very appreciated, please consider the following options:

1.) Install all required modules to the ESP. Although this doesn't help secure boot, this would allow booting insecurely in order to be able to uninstall signed grub.

2.) Refuse to install signed grub on systems unable to boot it, because of inaccessible filesystems.

3.) Install both signed and unsigned grub in parallel.

[Andrew Gunnerson (cxl)]

Yep, I had to chroot into the installation, remove the Ubuntu-signed GRUB, run grub-install, and sign the EFI binary with my own key for secure boot.

[Nazar Mokrynskyi (nazar-pc)]

This bug is almost 2 years old and no one seems to care about it unfortunately

[TJ (tj)]

I've just been caught out by this when the EFI -signed- packages somehow got themselves installed - despite having been removed in the past when they appeared due to a change in the depends - and replaced the locally built grubx64.efi that contains the cryptodisk, luks, etc., modules, leaving the system unbootable.

On another (BIOS-based) system I started an LXD 18.04 container, installed grub-amd64-efi, and then built a custom core image to transfer to a USB stick and then used the the EFI boot manager to add that to the boot menu. Once that started I used the grub shell to unlock the encrypted /boot/ file-system, reset 'root' and 'prefix' and started the usual menu.

Here's the commands I used to build the grubx64.efi inside the container.

apt install grub-amd64-efi

## I added the following to /etc/default/grub (just in case - I think this is only noticed by grub-install though)

GRUB_ENABLE_CRYPTODISK=y

mkdir -p /boot/efi/EFI/UBUNTU
grub-mkimage -v -p /boot -o /boot/efi/EFI/UBUNTU/grubx64.efi -O x86_64-efi part_gpt fat ext2 efi_gop cryptodisk luks gcry_rijndael gcry_sha1 usb usbms usb_keyboard efifwsetup efinet iso9660 help gcry_sha256 loopback lsefi lsefimmap lsefisystab lvm mdraid09 mdraid1x part_msdos ntfs ntfscomp squash4 tftp http udf ufs1 ufs2 exfat ehc
i uhci cat

Then from the host PC I wrote a GPT with a single 256MB partition to an memory card (SD-card in this case) and formatted it as FAT16 and copied the GRUB core image to it:

mkfs.vfat -F 16 /dev/mmcblk0p1
mkdir -p /mnt/boot
mount /dev/mmcblk0p1 /mnt/boot
mkdir -p /mnt/boot/EFI/BOOT
cp /var/lib/lxd/containers/u1804/rootfs/boot/efi/EFI/UBUNTU/grubx64.efi /mnt/boot/EFI/BOOT/BOOTX64.EFI
umount /mnt/boot

Then I rebooted the EFI PC with the SD-Card inserted in a USB card-adapter into its setup program and added a boot menu entry pointing to the BOOTX64.EFI, then booted to it.

Once in the GRUB rescue shell I identified the partition containing encrypted GRUB root file-system (Known as /boot/ on Linux) using:

> ls
> ls (hd1,gpt2)
Unknown filesytem

And then did:

> cryptomount hd1,gpt2

Typed the passphrase correctly and saw:

slot 0 opened

Then I did 'ls' to list the devices and saw a new (crypto0) device. I checked that GRUB could read the file-system inside it with:

> ls (crpyto0)/

Then reset the environment variables:

> set root=crypto0
> set prefix=($root)/grub
> ls $prefix/

Having seen the files in the GRUB directory I then loaded the menu-generating module and executed it:

> insmod normal
> normal

And booted normally.

Lastly I identified and removed the grub*signed packages and those that depend on them, and regenerated the correct grubx64.efi core image.

apt list --installed grub*
apt list --installed shim*

sudo apt remove shim shim-signed grub-efi-amd64-signed
sudo grub-install /dev/sda
sudo ls -latr /boot/efi/EFI/ubuntu/

I confirmed the EFI boot menu entry was now pointing to the newly generated grubx64.efi

sudo efibootmgr -v

[Paddy Landau (paddy-landau)]

I believe that bug #1401532 is related (but not a duplicate).

[Tom Reynolds (tomreyn)]

See bug #1762988 for a discussion of how
(a) this also affects Ubuntu installers and
(b) a non secureboot full disk encryption installation can be achieved with the alternative server installer (AKA debian-installer).

[Launchpad Janitor (janitor)]

This bug was fixed in the package grub2 - 2.02+dfsg1-12ubuntu1

---------------
grub2 (2.02+dfsg1-12ubuntu1) disco; urgency=medium

  * Merge against Debian unstable; remaining changes (LP: #564853):
    - debian/control: Update Vcs fields for code location on Ubuntu.
    - debian/control: Breaks shim (<< 13).
    - Secure Boot support: use newer patchset from rhboot repo:
      - many linuxefi_* patches added and modified
      - dropped debian/patches/linuxefi_require_shim.patch
      - renamed: debian/patches/no_insmod_on_sb.patch ->
        debian/patches/linuxefi_no_insmod_on_sb.patch
    - debian/patches/install_signed.patch, grub-install-extra-removable.patch:
      - Make sure if we install shim; it should also be exported as the default
        bootloader to install later to a removable path, if we do.
      - Rework grub-install-extra-removable.patch to reverse its logic: in the
        default case, install the bootloader to /EFI/BOOT, unless we're trying
        to install on a removable device, or explicitly telling grub *not* to
        do it.
      - Install a BOOT.CSV for fallback to use.
      - Make sure postinst and templates know about the replacement of
        --force-extra-removable with --no-extra-removable.
    - debian/patches/add-an-auto-nvram-option-to-grub-install.patch: Add the
      --auto-nvram option to grub-install for auto-detecting NVRAM availability
      before attempting NVRAM updates.
    - debian/build-efi-images: provide a new grub EFI image which enforces that
      loaded kernels are signed for Secure Boot: build gsb$arch.efi; which is
      the same as grub$arch.efi minus the 'linux' module. Without fallback to
      'linux' for unsigned loading, this makes it effectively enforce having a
      signed kernel.
    - Verify that the current and newer kernels are signed when grub is
      updated, to make sure people do not accidentally shutdown without a
      signed kernel.
    - debian/default/grub: replace GRUB_HIDDEN_* variables with the less
      confusing GRUB_TIMEOUT_STYLE=hidden.
    - debian/patches/support_initrd-less_boot.patch: Added knobs to allow
      non-initrd boot config.
    - Disable os-prober for ppc64el on the PowerNV platform, to reduce the
      number of entries/clutter from other OSes in Petitboot
    - debian/patches/shorter_version_info.patch: Only show the upstream version
      in menu and console, and hide the package one in a package_version
      variable.
    - debian/patches/skip_text_gfxpayload_where_not_supported.patch: Skip the
      'text' payload if it's not supported but present in gfxpayload, such as
      on EFI systems.
    - debian/patches/bufio_sensible_block_sizes.patch: Don't use arbitrary file
      fizes as block sizes in bufio: this avoids potentially seeking back in
      the files unnecessarily, which may require re-open files that cannot be
      seeked into, such as via TFTP.
    - debian/patches/ofnet-init-structs-in-bootpath-parser.patch: initialize
      structs in bootpath parser.
    - debian/rules: shuffle files around for now to keep build artefacts
      for signing at the same location as they were expected by Launchpad.
    - debian/ru...

[Tom Reynolds (tomreyn)]

Will this also make it into Bionic, Xenial (and maybe Cosmic, too)?

[Mathieu Trudel-Lapierre (cyphermox)]

There currently are no plans to that effect.

For now; shipping in the devel series, and the installer does not set up any partitioning automatically that would make use of this. It's "only" a convenience for people who do their own partitioning, which requires some prior knowledge to set this up correctly.

I'm not against having this landed as stable updates for bionic and cosmic; but it would greatly help if people signed up to do the SRU verification, as this is often what is the bottleneck for releasing updates.

[Tom Reynolds (tomreyn)]

I'd be very happy to help testing cryptodisk / luks / lvm / md raid configurations as part of a joint SRU effort.

[semreh (launchpad-via-forwarder)]

This could affect people using 'non-standard' modules not included in the standard distribution. For example, I use NILFS2 for all my filesystems, and if the signed grubx64.efi does not include that, I won't be able to boot. It might be a bit esoteric, but it points to a general problem: either all possible modules are included in the signed grub64x.efi, which some people might object to on size and/or attack surface grounds, or some are omitted, causing problems for people who wish to boot securely with modules not included in the standard distribution.

I vote for including all modules, but I have no problems with the size grub64x.efi becomes, and I'm less worried about the attack surface than some others might be.

[Nazar Mokrynskyi (nazar-pc)]

Another option would be to have current build and build with all modules with ability to chose one or the other in configuration file. However, I would argue crypto-related modules are.not as esoteric to not be included by default.

[Niklas Sombert (ytvwld)]

I just ran into this bug on bionic with a pretty custom configuration. Is there any chance that this can be backported?