System doesn't boot after installation - Legacy mode / 2 disks

[Jean-Baptiste Lallement]

[Impact]

 * Sometimes, installer was not installing the bootloader on the right drive, if there were more than two to pick from.

[Test Case]

 * Install onto a machine with two different drives, i.e. sda & nvme

* Ideally wipe both drives clean, to prevent bootloader from previous installation making it seem that installation works, when it does not.

 * Pick one drive or the other, during guided installation

 * Before rebooting, inspect /var/log/syslog, to ensure that "grub-installer" was run against the same drive as the one used for installation (i.e. mounted in /target)

 * At the end of both installs, machine should reboot off the target drive just fine.

[Regression Potential]

 * The currently behaviour of choosing the "first" drive to be the bootloader device, is flawed logic under BIOS, as the same drive as the rest of the installation should normally be used for installation. Some may have been relying on this behaviour and installing bootloaders on one drive, to boot off the other. Such users can still go into "Something else" and precisely pick the desired drive for the bootloader installation. (See the workaround below)

===

Workaround installing on SATA drive (sda)

* Click Something Else custom first
* Change boot loader drop down from nvme to sda
* Click back
* Use wipe & install
* Select sda for install
* Complete install & reboot

--

Ubuntu 19.10 "Eoan Ermine" - Beta amd64 (20191012)

Dell Inspiron 7472 with 2 disks: sata and nvme
Boot in legacy mode
System installed on sda

After installation the system doesn't find a bootable device.
The same problem occurs if I install Ubuntu on the nvme drive instead.

If I perform the installation in uefi/no secure boot mode, it boots fine.

ProblemType: Bug
DistroRelease: Ubuntu 19.10
Package: ubiquity 19.10.19
ProcVersionSignature: Ubuntu 5.3.0-18.19-generic 5.3.1
Uname: Linux 5.3.0-18-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia zfs zunicode zavl icp zcommon znvpair
ApportVersion: 2.20.11-0ubuntu8
Architecture: amd64
CasperVersion: 1.424
CurrentDesktop: ubuntu:GNOME
Date: Sun Oct 13 05:00:22 2019
InstallCmdLine: file=/cdrom/preseed/ubuntu.seed initrd=/casper/initrd quiet splash nomodeset ---
LiveMediaBuild: Ubuntu 19.10 "Eoan Ermine" - Beta amd64 (20191012)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C.UTF-8
 SHELL=/bin/bash
SourcePackage: ubiquity
UpgradeStatus: No upgrade log present (probably fresh install)

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1847898

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 19.10.20

---------------
ubiquity (19.10.20) eoan; urgency=medium

  [ Jean-Baptiste Lallement ]
  * zsys-setup: Force a re-read of the partition table before using the disk
    (LP: #1847748)

  [ Iain Lane ]
  * zsys-setup: Mark packages to kept in the init phase, rather than finalize.
    Finalize runs after ubiquity has removed files from packages which it has
    decided the target system isn't going to need. It's too late for us to try
    to save things then. (LP: #1847898)

  [ Dimitri John Ledkov ]
  * Automatic update of included source packages: partman-auto
    134ubuntu12, partman-base 206ubuntu6 (LP: #1847944)

 -- Dimitri John Ledkov <email address hidden> Mon, 14 Oct 2019 14:31:19 +0100

[Jean-Baptiste Lallement (jibel)]

Reopening, the wrong bug has been closed.

[Adam Conrad (adconrad)]

We reproduced this in eoan, and then went on to also reproduce it in disco, meaning it's not a regression, and therefor not RC for eoan.

It is, however, really bad UX, and I want to make sure we do this better for 20.04.

[Dimitri John Ledkov (xnox)]

For future reference qemu reproducer is:

qemu-system-x86_64 -m 4096 -cdrom ./ubuntu-19.04-desktop-amd64.iso -drive file=generic.qcow2,if=none,id=D22 -device nvme,drive=D22,serial=1234 --enable-kvm -smp 2 -drive id=disk,file=generic-uefi-1.qcow2,if=none -device ahci,id=ahci -device ide-drive,drive=disk,bus=ahci.0 -boot menu=on

[Dimitri John Ledkov (xnox)]

It appears that we go by alphabetical order for both select_disk option and the bootdev, and never update grub-installer/bootdev to partman-auto/select_disk value post the "choose disk" screen.

[oldfred (oldfred)]

Related bugs:

Ubuntu Installer uses wrong bootloader location for USB/sdb UEFI installs
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1173457
Posted work around to manually unmount & mount correct ESP during install
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1396379
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1229488
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1229488
EFI doesn't show multiple installs of the same operating system
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1561712

[Herb Theriault (herbt)]

My primary nvme drive shows up as sdb. Installing with 2 drives, and the nvme as sdb fails. Machine will not boot, post install. Pulled the secondary drive, reinstalled, and it does boot.

[Michael Hudson-Doyle (mwhudson)]

I think the confusion is in this function (and the similarly shaped kde one):

    def get_grub_choice(self):
        i = self.grub_device_entry.get_active_iter()
        if i:
            return self.grub_device_entry.get_model().get_value(i, 0)
        else:
            self.debug('No active iterator for grub device entry.')
            disk = self.get_current_disk_partman_id()
            if isinstance(disk, str) and disk:
                disk_path = disk.replace("=", "/")
                if os.path.exists(disk_path):
                    return misc.grub_default(boot=disk_path)

            return misc.grub_default()

(or https://git.launchpad.net/ubiquity/tree/ubiquity/plugins/ubi-partman.py#n681)

This figures out which device has been selected as the target disk and passes it as the boot= argument to misc.grub_default(), clearly expecting that grub_default() will return an appropriate path to install grub too. But grub_default() does not interpret the boot= argument in that way at all, and (modulo some fussing about installing from writable media) returns the first line from grub-mkdevicemap.

[Michael Hudson-Doyle (mwhudson)]

Untested maybe-fix at https://code.launchpad.net/~mwhudson/ubiquity/+git/ubiquity/+merge/383762

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 20.10.1

---------------
ubiquity (20.10.1) groovy; urgency=medium

  [ Sebastien Bacher ]
  * debian/real-po:
    - updated translations from launchpad, including strings from the new
      subpage for RST (lp: #1874103)

  [ Jean-Baptiste Lallement ]
  * zsys-setup: Enable autotrim on all pools.

  [ Dimitri John Ledkov ]
  * Automatic update of included source packages: console-setup
    1.195ubuntu2, localechooser 2.71ubuntu5, partman-efi 84ubuntu1,
    partman-lvm 133.

  [ Michael Hudson-Doyle ]
  * make grub_default consider the boot argument in the non-removable case
    (LP: #1847898)

 -- Dimitri John Ledkov <email address hidden> Wed, 27 May 2020 11:32:08 +0100

[Dimitri John Ledkov (xnox)]

This is now on daily groovy.

Jibel can you please test this?

[Brian Murray (brian-murray)]

Hello Jean-Baptiste, or anyone else affected,

Accepted ubiquity into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubiquity/20.04.15.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Jean-Baptiste Lallement (jibel)]

SRU verification for Focal:
I have reproduced the problem with ubiquity 20.04.15 in focal and have verified that the version of ubiquity 20.04.15.1 in -proposed fixes the issue.

I tested on a VM with 3 disks, and it boots from 3rd disk when I set it as the first disk to try. Otherwise it doesn't find any bootable device but I suppose it's because the VM doesn't try the other disks in sequence.

Marking as verification-done

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 20.04.15.1

---------------
ubiquity (20.04.15.1) focal; urgency=medium

  [ Sebastien Bacher ]
  * debian/real-po: updated translations from launchpad, including strings
    from the new subpage for RST (LP: #1874103)

  [ Michael Hudson-Doyle ]
  * make grub_default consider the boot argument in the non-removable case
    (LP: #1847898)

  [ Jean-Baptiste Lallement ]
  * Enable ZFS autotrim (LP: #1883686)
  * Use persistent path for pool's vdevs (LP: #1880869)
  * Do not export all pools (LP: #1875045)

  [ Dimitri John Ledkov ]
  * ubiquity-dm: start gsd-keyboard for keyboard indicator (LP: #1847307)

 -- Dimitri John Ledkov <email address hidden> Tue, 16 Jun 2020 11:17:42 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for ubiquity has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.