lubuntu noble install failure - encryption

[Chris Guiver]

possibly a duplicate... (install attempt a few days back report is here - https://bugs.launchpad.net/ubuntu/+source/calamares/+bug/2043161)

but error message doesn't quite match.. thus new report but likely related

** details

install to

- sony vaio ultrabook svp11216cgb (i5-9400u, 4gb, intel haswell-ULT)

testcase: full disk, encryption, EFI, no-internet, no-swap, autologin

** expected result

clean install, reboot & login

** actual result

calamares ended with message
---
The bootloader could not be installed. The installation command <pre>grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu --force</pre> returned error code 1.
---

Installs with options
- testcase: full disk, no-encryption, EFI, no-internet, no-swap
- testcase: full disk, no-encryption, EFI, no-internet, no-swap, autologin

This failed install used options
- testcase: full disk, encryption, EFI, no-internet, no-swap, autologin

ie. issue is with ENCRYPTION being used I believe

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: calamares 3.3.0-alpha4-0ubuntu1
ProcVersionSignature: Ubuntu 6.5.0-9.9-generic 6.5.3
Uname: Linux 6.5.0-9-generic x86_64
.etc.calamares.modules.after_bootloader_context.conf:
 ---
 dontChroot: false
 timeout: 120
 firmwareType:
     "*": "-rm ${ROOT}/home/${USER}/Desktop/lubuntu-calamares.desktop"
.etc.calamares.modules.finished.conf:
 ---
 restartNowMode: user-checked
 restartNowCommand: "systemctl -i reboot"
.etc.calamares.modules.fstab.conf:
 crypttabOptions: luks,keyscript=/bin/cat
 efiMountOptions: umask=0077
.etc.calamares.modules.shellprocess_logs.conf:
 ---
 dontChroot: true
 timeout: 30
 script:
     - calamares-logs-helper ${ROOT}
.etc.calamares.modules.unpackfs.conf:
 ---
 unpack:
     - source: "/cdrom/casper/filesystem.squashfs"
         sourcefs: "squashfs"
         destination: ""
ApportVersion: 2.27.0-0ubuntu6
Architecture: amd64
CasperMD5CheckResult: pass
CasperVersion: 1.486
CurrentDesktop: LXQt
Date: Sun Nov 12 17:36:59 2023
LiveMediaBuild: Lubuntu 24.04 "Noble Numbat" - Daily amd64 (20231111)
RelatedPackageVersions:
 calamares-settings-ubuntu-common 1:23.10.4
 calamares-settings-lubuntu 1:23.10.4
 xfsprogs 6.3.0-1ubuntu1
 btrfs-progs 6.3.2-1
SourcePackage: calamares
UpgradeStatus: No upgrade log present (probably fresh install)

[Chris Guiver (guiverc)]

This is the second install today to the sony.vaio.thingy

First install used these options
- testcase: full disk, no-encryption, EFI, no-internet, no-swap

This failed install used options
- testcase: full disk, encryption, EFI, no-internet, no-swap, autologin

IE...

It worked with NO-ENCRYPTION and no-autologin
but failed with ENCRYPTION and autologin.

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
https://iso.qa.ubuntu.com/qatracker/reports/bugs/2043301

[Chris Guiver (guiverc)]

Another install to sony.vaio.thingy

- testcase: full disk, NO-encryption, EFI, no-internet, no-swap, autologin

so FAILURE only with ENCRYPTION

(works with/without autologin)

iso.qa link being https://iso.qa.ubuntu.com/qatracker/milestones/450/builds/289541/testcases/1701/results/

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[sudodus (nio-wiklund)]

Tested in a Toshiba Satellite Pro C850-19W with Intel i5-3210M CPU in BIOS (alias legacy) mode, full disk, encryption, internet, swap.

The install stopped with a popup window with the following message:

Starthanterare kunde inte installeras. Installationskommandot <pre>grub-install --target=i386-pc --recheck --force /dev/sda</pre>returnerade felkod 1.

which I think you can understand without translation. (The start handler could not be installed ... returned error code 1.)

I can confirm the bug, and it affects also boot in BIOS mode.

[Aaron Rainbolt (arraybolt3)]

Looking at the Calamares installation log in /root/.cache/calamares/session.log, I see the following additional info:

    grub-install: error: attempt to install to encrypted disk without cryptodisk enabled. Set `GRUB_ENABLE_CRYPTODISK=y' in file `/etc/default/grub'.

As expected the specified option is missing in the file. Adding the option to /etc/default/grub doesn't change anything though, the same error appears... of course, because Calamares runs grub-install within the already-installed system via a chroot (if I'm understanding correctly), meaning it's using its own /etc/default/grub, probably copying whatever's on the squashfs.img of the ISO.

If I chroot into the installed system (i.e., `sudo chroot /tmp/calamares-root-<TAB>`), then add the option to /etc/default/grub within the installed system and THEN run grub-install, it seems to work. After that stunt, I attempted to boot into the installed system. Unlocking the disk worked, but I didn't manage to get to a graphical environment thereafter since I forgot to generate a GRUB configuration file and my GRUB-fu and BusyBox-fu failed me. But I think the missing option is the root of the problem.

[Aaron Rainbolt (arraybolt3)]

After much trial and error, I have a workaround.

When you first boot the ISO, before beginning the installation, open QTerminal, and edit /etc/calamares/modules/before_bootloader_context.conf so that it has these contents underneath the comments:

    ---
    firmwareType:
        bios:
            - command: bash -c "echo \"GRUB_ENABLE_CRYPTODISK=y\" > /etc/default/grub.d/lubuntu-enable-crypto.cfg"
                 timeout: 10
        "*":
            ...<a bunch of apt and sed related stuff goes here>...
            - command: bash -c "echo \"GRUB_ENABLE_CRYPTODISK=y\" > /etc/default/grub.d/lubuntu-enable-crypto.cfg"
                 timeout: 10

(obviously the ...<a bunch of apt and sed related stuff goes here>... is a placeholder, you'll put the last two lines at the very bottom of that section)

Save the file, then run an encrypted installation. It should install successfully and you should be able to boot into the installed system and get a graphical environment.

[Chris Guiver (guiverc)]

i attempted to follow instruction in comment #8 and failure during install
- sony vaio thingy

failure reported by calamares
---
The bootloader could not be installed. The installation command <pre>grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu --force</pre> returned error code 1.
---

modified file is
---
lubuntu@lubuntu:/etc/calamares/modules$ cat before_bootloader_context.conf
# Make sure the correct bootloader package is installed for EFI.
# Also pull in shim so secureboot has a chance at working.
# Because of edge cases, we ignore BIOS, and do the same
# procedure for all EFI types.
---
firmwareType:
    bios: "-/bin/true"
    "*":
        - command: apt-cdrom add -m -d=/media/cdrom/
             timeout: 10
        - command: sed -i '/deb http/d' /etc/apt/sources.list
             timeout: 10
        - command: apt-get update
             timeout: 120
        - command: apt install -y --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict grub-efi-$(if grep -q 64 /sys/firmware/efi/fw_platform_size; then echo amd64-signed; else echo ia32; fi)
             timeout: 300
        - command: apt install -y --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict shim-signed
             timeout: 300

#firmwareType:
# bios: "-/bin/true"
# # "*":
# - command: apt-cdrom add -m -d=/media/cdrom/
# timeout: 10
# - command: sed -i '/deb http/d' /etc/apt/sources.list
# timeout: 10
# - command: apt-get update
# timeout: 120
# - command: apt install -y --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict grub-efi-$(if grep -q 64 /sys/firmware/efi/fw_platform_size; then echo amd64-signed; else echo ia32; fi)
# timeout: 300
# - command: apt install -y --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict shim-signed
           # # timeout: 300
---

note I didn't remove the prior contents; just commented them out, then attempted to copy & paste the details from comment 8 above into the file.

a diff however I hoped would show me a difference (I didn't see any), but I didn't get any..

---
lubuntu@lubuntu:/etc/calamares/modules$ diff compare before_bootloader_context.conf
0a1,5
> # Make sure the correct bootloader package is installed for EFI.
> # Also pull in shim so secureboot has a chance at working.
> # Because of edge cases, we ignore BIOS, and do the same
> # procedure for all EFI types.
> ---
14a20,32
> #firmwareType:
> # bios: "-/bin/true"
> # # "*":
> # - command: apt-cdrom add -m -d=/media/cdrom/
> # timeout: 10
> # - command: sed -i '/deb http/d' /etc/apt/sources.list
> # timeout: 10
> # - command: apt-get update
> # timeout: 120
> # - command: apt install -y --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict grub-efi-$(if grep -q 64 /sys/firmware/efi/fw_platform_size; then echo amd64-signed; else echo ia32; fi)
> # timeout: 300
> # - command: apt install -y --no-upgrade -o Acquire::gpgv::Options::=--ignore-time-conflict shim-signed
> ...

[Aaron Rainbolt (arraybolt3)]

@guiverc I think something must have gone wrong since I don't see any `bash -c` call being done anywhere in your modified config file.

However, this workaround is very much a hack - the problem has been tracked down to an issue in the Python code of the grubcfg module. I have a working patch, and have submitted it (and a bug report) upstream. I'll hopefully be patching our package soon-ish.

[Launchpad Janitor (janitor)]

This bug was fixed in the package calamares - 3.3.0-alpha4-0ubuntu2

---------------
calamares (3.3.0-alpha4-0ubuntu2) noble; urgency=medium

  * Fixed encrypted installations. (LP: #2043301)
  * Fixed watch file.
  * Swapped out an obsolete build dependency (poilcykit-1).
  * Created upstream/metadata file.

 -- Aaron Rainbolt <email address hidden> Tue, 14 Nov 2023 17:32:18 -0600

[sudodus (nio-wiklund)]

I can confirm that it works to create an installed system with encryption. See the attached file.