grub fails to install bootloader for zesty LVM with Encryption

[Jeremy Bícha]

Ubuntu GNOME 17.04 Alpha 2 candidate. Entire disk. Verified on both i386 and amd64. I am testing in VirtualBox. Only an 8GB virtual disk which should be just barely enough for Ubuntu GNOME (see also LP: #1614810 )

Install completes successfully if I chose LVM, but if I chose to "Encrypt the Ubuntu installation for security", I get this error:
"Unable to install GRUB in /dev/sda
 Executing 'grub-install /dev/sda1' failed.
 This is a fatal error."

After clicking ok, I get another popup with three options (Choose a different device to install the bootloader on, continue without a bootloader, cancel the instllation). None of the options appear to do anything.

I'm attaching screenshots.

From UbiquitySyslog.txt
-----------------------
Jan 26 01:11:01 ubuntu-gnome partman: PV /dev/mapper/sda1_crypt VG ubuntu-gnome-vg lvm2 [8.49 GiB / 24.00 MiB free]
Jan 26 01:11:01 ubuntu-gnome partman: Total: 1 [8.49 GiB] / in use: 1 [8.49 GiB] / in no VG: 0 [0 ]
Jan 26 01:11:01 ubuntu-gnome partman: Reading volume groups from cache.
Jan 26 01:11:01 ubuntu-gnome partman: Found volume group "ubuntu-gnome-vg" using metadata type lvm2
Jan 26 01:11:01 ubuntu-gnome partman-lvm: 2 logical volume(s) in volume group "ubuntu-gnome-vg" now active
Jan 26 01:11:02 ubuntu-gnome ubiquity: mount: unknown filesystem type 'crypto_LUKS'
Jan 26 01:11:03 ubuntu-gnome ubiquity: mount:
Jan 26 01:11:03 ubuntu-gnome ubiquity: unknown filesystem type 'crypto_LUKS'
Jan 26 01:11:03 ubuntu-gnome ubiquity:
Jan 26 01:11:03 ubuntu-gnome ubiquity: mount:
Jan 26 01:11:03 ubuntu-gnome ubiquity: unknown filesystem type 'crypto_LUKS'
Jan 26 01:11:03 ubuntu-gnome ubiquity:
Jan 26 01:11:03 ubuntu-gnome ubiquity: mount:
Jan 26 01:11:03 ubuntu-gnome ubiquity: unknown filesystem type 'crypto_LUKS'
Jan 26 01:11:03 ubuntu-gnome ubiquity:
Jan 26 01:11:03 ubuntu-gnome ubiquity[4392]: Device /dev/mapper/ubuntu--gnome--vg-root not found in os-prober output
Jan 26 01:11:03 ubuntu-gnome ubiquity[4392]: Device /dev/sda1 not found in os-prober output
Jan 26 01:11:03 ubuntu-gnome ubiquity[4392]: Device /dev/mapper/ubuntu--gnome--vg-swap_1 not found in os-prober output
Jan 26 01:11:03 ubuntu-gnome ubiquity[4392]: switched to page partman

ProblemType: Bug
DistroRelease: Ubuntu 17.04
Package: ubiquity 17.04.4 [modified: lib/partman/automatically_partition/question]
ProcVersionSignature: Ubuntu 4.9.0-12.13-generic 4.9.2
Uname: Linux 4.9.0-12-generic x86_64
ApportVersion: 2.20.4-0ubuntu1
Architecture: amd64
CasperVersion: 1.380
CurrentDesktop: GNOME
Date: Wed Jan 25 20:12:08 2017
InstallCmdLine: file=/cdrom/preseed/ubuntu-gnome.seed boot=casper initrd=/casper/initrd.lz quiet splash --- maybe-ubiquity
LiveMediaBuild: Ubuntu-GNOME 17.04 "Zesty Zapus" - Alpha amd64 (20170125)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: grub-installer
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[sudodus (nio-wiklund)]

Iso testing Lubuntu Zesty, testcase 'Alternate Install (Encryption) in Lubuntu Alternate i386 in Zesty Daily'

http://iso.qa.ubuntu.com/qatracker/milestones/371/builds/141408/testcases/1439/results

The grub bootloader could not be installed (red screen). I assume the same bug as reported by Ubuntu Gnome.

[Mathieu Trudel-Lapierre (cyphermox)]

This applies to all flavors using full-disk encryption; os-prober won't be able to deal with that partition directly, it needs to dig in -- that should have worked already since we do ask dm and lvm for the partitions, but somehow fails in the installer.

It's a side-effect of bug 1639801; I'll work to fix it.

[Mathieu Trudel-Lapierre (cyphermox)]

It appears to be because since November, we changed partman to create LVM installs without a separate /boot. Unfortunately, /boot is still required for LVM+crypto, as grub2 by itself can't load cryptodisks (unless the option is enabled, that is), and doing so would make it necessary to unlock the cryptodisk twice -- once for grub, and another time for the kernel.

For now, let's just make sure crypto still uses a /boot partition.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-auto-lvm - 59ubuntu3

---------------
partman-auto-lvm (59ubuntu3) zesty; urgency=medium

  * lib/auto-lvm.sh: Pass method on to choose_recipe() and decode_recipe()
    when auto_lvm_prepare() is called. This allows crypto installs to be
    run as "crypto" method (as they should), rather than "lvm", and thus
    possibly different partitioning recipes. (LP: #1659448)

 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 26 Jan 2017 16:32:06 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 17.04.5

---------------
ubiquity (17.04.5) zesty; urgency=medium

  * Automatic update of included source packages: clock-setup
    0.131ubuntu1, partman-auto-lvm 59ubuntu3. (LP: #1659448)

 -- Mathieu Trudel-Lapierre <email address hidden> Mon, 30 Jan 2017 22:45:07 -0500

[Jeremy Bícha (jbicha)]

Thank you. I confirm that an "LVM with encryption" install works fine with the February 5 Ubuntu GNOME 17.04 daily iso.