Cannot configure 'cryptsetup luksFormat' at install time

[Mauricio Faria de Oliveira]

[Impact]

 * Users cannot specify options for 'cryptsetup luksFormat'
   that is used by the installer.

 * Some deployments need the installed disks in LUKS1 format
   for backward compatibility with older releases that don't
   support LUKS2, for backup/audit/management purposes.

 * However, on Focal and later, cryptsetup defaults to LUKS2,
   which broke that functionality.

 * Currently it's not possible to request the LUKS format in
   the installer, so this patch allows for that w/ a preseed
   option ('partman-crypto/luksformat_options') for the user.

[Test Case]

 * Default behavior: LUKS2

   - Install Ubuntu (Focal/later); check LUKS header version:

     $ sudo cryptsetup luksDump /dev/vda4
     LUKS header information
     Version: 2
     ...

 * Opt-in behavior: LUKS1 (for example; can use other options)

   - Install Ubuntu (Focal/later) with preseed file/option:

     ubiquity partman-crypto/luksformat_options string \
       --type luks1

   - Check LUKS header version:

     $ sudo cryptsetup luksDump /dev/vda4
     LUKS header information for /dev/vda4
     Version: 1
     ...

   - Check install logs for confirmation:

     $ grep luksFormat /var/log/partman
     /usr/bin/autopartition-crypto: Additional options for luksFormat: '--type luks1'

[Where problems could occur]

 * The changes are contained within the partman-crypto functionality,
   so only install with encrypted disks should be affected by issues.

 * Any additional options specified to 'cryptsetup luksFormat' are
   opt-in _and_ specified by the user via the preseed option, thus
   errors are probably tied to particular options (mis) used.

 * If the preseed option is not specified, original behavior remains.

[Other Info]

 * This patch is applied in Hirsute.
 * This patch is not needed in Groovy (rationale in comment #15.)
 * This patch is targeted at Focal (cryptsetup defaulted to LUKS2.)
 * This patch is not needed in Bionic/earlier (^defaults to LUKS1.)

[Original Description]
Most users should be fine with the options to
'cryptsetup luksFormat' used by the installer.

However, some users may have reasons to use
other options, and that is not possible now.

Let's provide a new preseed option for that:
'partman-crypto/luksformat_options'

[Mauricio Faria de Oliveira (mfo)]

Attaching a working patch; for reference.

[Mauricio Faria de Oliveira (mfo)]

Testing in Groovy Desktop ISO (daily build 2020-09-30)

Example: usage to specify luks header version,
as LUKS1; the default is LUKS2 on Focal/later.

$ sudo dpkg -i --force-depends --force-overwrite partman-crypto_101ubuntu4+luksopts1_amd64.udeb

Default (no option specified / no changes):
---

$ sudo cryptsetup luksDump /dev/vda4
LUKS header information
Version: 2
...

Option:
---

Preseed file:

"""
ubiquity partman-crypto/luksformat_options string \
  --type luks1
"""

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1

$ grep luksFormat /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat: '--type luks1'

$ sudo cryptsetup luksDump /dev/vda4
LUKS header information for /dev/vda4

Version: 1
...

[Steve Langasek (vorlon)]

In groovy and later, ubiquity vendorizes the udeb sources instead of including them from the archive at build time; so reassigning this to ubiquity since it appears you are targeting fixing this in the desktop installer.

[Mauricio Faria de Oliveira (mfo)]

Thanks for mentioning; I wasn't aware of this shipping method.

I'll update the patch for ubiquity/d-i/sources/partman-crypto.

[Mauricio Faria de Oliveira (mfo)]

Reported bug on Debian #971599

https://bugs.debian.org/971599

[Mauricio Faria de Oliveira (mfo)]

Submitted the change to the two repos (per ubiquity/d-i/source/README)
as I cannot pull into/upload to them. Hopefully this should be correct.

1) the stand-alone repo (~ubuntu-installer/partman-crypto) [1].
2) the bundled code in ubiquity (~ubuntu-installer/ubiquity) [2].

[1] https://code.launchpad.net/~mfo/partman-crypto/+git/partman-crypto/+merge/391766
[2] https://code.launchpad.net/~mfo/ubiquity/+git/ubiquity/+merge/391767

[Oleg "Nightwing" Lomakin (nightwing666)]

Tested this patch with netinstall by legacy installer(aka debian-installer).
Works well!
Just added into preseed:

d-i preseed/early_command string \
wget -q http://myhost.com/files/packages/partman-crypto_101ubuntu4+luksopts1_amd64.udeb ; \
udpkg -i partman-crypto_101ubuntu4\+luksopts1_amd64.udeb
d-i partman-crypto/luksformat_options string --type luks1

[Łukasz Zemczak (sil2100)]

Merged all the required branches and pushed those out to hirsute.

[Mauricio Faria de Oliveira (mfo)]

This has been fixed on Hirsute; changing to Fix Released.

"""
ubiquity (21.04.3) hirsute; urgency=medium

  [ Mauricio Faria de Oliveira ]
  * Introduce preseed option partman-crypto/luksformat_options

  [ Łukasz 'sil2100' Zemczak ]
  * Automatic update of included source packages: partman-crypto
    101ubuntu5.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Thu, 03 Dec 2020 15:21:45 +0100
"""

[Mauricio Faria de Oliveira (mfo)]

In order to fix this in Focal for the 20.04.3 point release:

Groovy is a no-op, since it has no partman-crypto, and its
ubiquity doesn't need the patch as there won't be new ISOs.
[thus: partman-crypto / Invalid; ubiquity / Won't Fix.]

Focal needs the patch on partman-crypto only (as it is not
yet shipped via ubiquity), then ubiquity installs it later.
[thus: partman-crypto / In Progress; ubiquity / Invalid.]

[Mauricio Faria de Oliveira (mfo)]

Actually in Focal partman-crypto is present in ubiquity
(in d-i/source/partman-crypto) so we'll change it again.

However, ubiquity in Focal doesn't yet use git subtrees;
it just `apt-get source` the d-i components, so we need:

1) patch partman-crypto/focal.
2) no-change-rebuild ubiquity/focal, so to pick it up.

I'll try that in a PPA.

[Mauricio Faria de Oliveira (mfo)]

Actually, step 2) is not a no-change-rebuild, but running 'debian/rules update', which adds to the changelog properly.

[Mauricio Faria de Oliveira (mfo)]

All looks good with the PPA build.

After patch/build partman-crypto, and updating its sources
in ubiquity (adding an apt source to the PPA, of course),
the ubiquitu .deb package includes the partman-crypto changes
in '/lib/partman/lib/crypto-base.sh' as expected.

[Mauricio Faria de Oliveira (mfo)]

Next Steps:

1) Upload partman-crypto to focal.
2) Wait for it to be available in focal-proposed.
3) Upload ubiquity to focal (gets focal-proposed.)

We could upload both and coordinate the time that
they are accepted and published (ubiquity must be
after partman-crypto), but plans might go wrong :)

[Mauricio Faria de Oliveira (mfo)]

In order to fix this for Focal / 20.04.2 point release:

Groovy is a no-op, because:

1) it has no partman-crypto as stand-alone src pkg anymore.
2) its ubiquity (w/ bundled partman-crypto) doesn't need a
   patch as there won't be new ISOs/Desktop install for it.
[thus, partman-crypto: Invalid / ubiquity: Won't Fix.]

Focal needs the fix in both, in two steps:

1) patch partman-crypto (stills exists as stand-alone src pkg.)
2) update ubiquity w/ the patched partman-crypto (in proposed.)
[thus, partman-crypto: In Progress / ubiquity: In Progress.]

[Mauricio Faria de Oliveira (mfo)]

Uploaded partman-crypto to focal.

After it makes it to focal-proposed,
updated ubiquity upload will follow.

[Brian Murray (brian-murray)]

Hello Mauricio, or anyone else affected,

Accepted partman-crypto into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/partman-crypto/101ubuntu4.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Mauricio Faria de Oliveira (mfo)]

Verification done for focal-proposed (partman-crypto)
---

1) Booted the Ubuntu 20.04.1 Desktop ISO.
2) Launched a terminal, extracted the .udeb and copied over the crypto-base.sh file.
3) Performed installation to encrypted disk without the preseed option (default).
4) Performed installation to encrypted disk with the preseed option (testing).

On both cases, the installation finishes successfully, and the system can boot.
The LUKS header version is used as expected (LUKS2 by default, LUKS1 w/ option)

Details:
---

Launch terminal:

$ wget https://launchpad.net/ubuntu/+archive/primary/+files/partman-crypto_101ubuntu4.1_amd64.udeb
$ dpkg-deb -x partman-crypto_101ubuntu4.1_amd64.udeb udeb
$ sudo cp udeb/lib/partman/lib/crypto-base.sh /lib/partman/lib/crypto-base.sh

$ grep luksopts /lib/partman/lib/crypto-base.sh
 local mapping device cipher iv size pass luksopts
   luksopts="$RET"
   log "Additional options for luksFormat: '$luksopts'"
  log-output -t partman-crypto /sbin/cryptsetup -c $cipher-$iv -h $hash -s $size $luksopts luksFormat $device $pass

Launch ubiquity / Install Ubuntu 20.04.1 Desktop to Encrypted LVM device.
(In 'Installation type', select 'Erase disk and install Ubuntu', click in 'Advanced features', select 'Use LVM ...', select 'Encrypt ...', and move on to 'Install Now')

Launch terminal:

Without the option:

$ sudo debconf-get partman-crypto/luksformat_options
$

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n2
LUKS header information
Version: 2

With the option:

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n3
LUKS header information for /dev/vda6

Version: 1

[Mauricio Faria de Oliveira (mfo)]

Based on good test results with partman-crypto (comment #19)
now moving on with ubiquity (to include the updated sources).

Tests with the patched packaged built from a PPA are positive.
The results are identical to the reported for partman-crypto
(LUKS2/LUKS1 by default/as requested; installed system boots.)

Testing ubiquity from PPA:
-------------------------

Booted the Ubuntu 20.04.1 Desktop ISO.
Launch terminal:

$ sudo add-apt-repository ppa:mfo/lp1898129prop2
$ sudo apt install -y ubiquity

$ dpkg -s ubiquity | grep Version:
Version: 20.04.15.3

$ grep luksopts /lib/partman/lib/crypto-base.sh
 local mapping device cipher iv size pass luksopts
   luksopts="$RET"
   log "Additional options for luksFormat: '$luksopts'"
  log-output -t partman-crypto /sbin/cryptsetup -c $cipher-$iv -h $hash -s $size $luksopts luksFormat $device $pass

Without the option:

$ sudo debconf-get partman-crypto/luksformat_options

$

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n2
LUKS header information
Version: 2

With the option:

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n3
LUKS header information for /dev/vda6

Version: 1

[Mauricio Faria de Oliveira (mfo)]

Uploaded ubiquity to focal.

Two notes:

1) The 'debian/rules update' process picked up the new partman-crypto from focal-proposed, and also shim-signed.
Discussed this with Laney and it shouldn't be an issue, as the ISO .list file already shows the new version, so it's being used for a while now.
Nonetheless, I'll test a secure boot scenario w/ a VM and OVMF/UEFI.

2) There's new pyflakes in focal-proposed that cause ubiquity to FTBFS.
This is a simple change already in Groovy, so just picking that up too.

[Łukasz Zemczak (sil2100)]

Thanks for including the FTBFS fix!

[Łukasz Zemczak (sil2100)]

Hello Mauricio, or anyone else affected,

Accepted ubiquity into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubiquity/20.04.15.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Mauricio Faria de Oliveira (mfo)]

Verification done for focal-proposed
---

All good, the package from -proposed works correctly in both scenarios -- without the option (ie, default behavior) and with the option (ie, opt-in behavior.)

Note: tested on VM with UEFI OVMF firmware with secure boot enabled (OVMF_CODE_4M.ms.fd), as shim-signed is also updated in the upload.
All good -- both scenarios install/boot to login screen w/ secboot.

Steps:
=====

On install, select Try Ubuntu, and launch Terminal.

$ sudo add-apt-repository 'deb http://archive.ubuntu.com/ubuntu focal-proposed main' && sudo apt install -y ubiquity && apt policy ubiquity
...
ubiquity:
  Installed: 20.04.15.3
  Candidate: 20.04.15.3
  Version table:
 *** 20.04.15.3 500
        500 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
...

$ grep -c luksopts /lib/partman/lib/crypto-base.sh
4

$ dmesg | grep -i secure
[ 0.000000] secureboot: Secure boot enabled
[ 0.000000] Kernel is locked down from EFI Secure Boot mode; see man kernel_lockdown.7
[ 0.008398] secureboot: Secure boot enabled

Move on with installer, select install to LVM/Encrypt.

Check on Terminal:

$ lsblk --ascii | grep -B1 crypt
`-vda3 252:3 0 8.8G 0 part
  `-vda3_crypt 253:0 0 8.8G 0 crypt

Without option (default)
---

$ sudo debconf-get partman-crypto/luksformat_options

$

$ sudo cryptsetup luksDump /dev/vda3 | head -n2
LUKS header information
Version: 2

With option (opt-in)
---

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1
$

$ sudo cryptsetup luksDump /dev/vda3 | head -n3
LUKS header information for /dev/vda3

Version: 1

$ grep luks /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat: '--type luks1'

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for partman-crypto has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-crypto - 101ubuntu4.1

---------------
partman-crypto (101ubuntu4.1) focal; urgency=medium

  * Add preseed option 'partman-crypto/luksformat_options' to
    provide more options for 'cryptsetup luksFormat' (LP: #1898129)
    - d/partman-crypto.templates: add preseed option.
    - lib/crypto-base.sh: check for, log, and use it.

 -- Mauricio Faria de Oliveira <email address hidden> Thu, 07 Jan 2021 16:51:37 -0300

[Launchpad Janitor (janitor)]

This bug was fixed in the package ubiquity - 20.04.15.3

---------------
ubiquity (20.04.15.3) focal; urgency=medium

  * Automatic update of included source packages:
    partman-crypto 101ubuntu4.1, shim-signed 1.40.4. (LP: #1898129)
  * Fix FTBFS with new pyflakes (fixed in Groovy/20.10.4, no LP bug):
    autopilot/ubiquity_autopilot_tests/configs/english_label_conf.py:48:
    .format(...) has unused arguments at position(s): 0

 -- Mauricio Faria de Oliveira <email address hidden> Wed, 13 Jan 2021 11:42:57 -0300

[Mauricio Faria de Oliveira (mfo)]

The functionality is present in the desktop ISO daily build of 2021-01-21 [1] , per the ISO manifest file [2]:

    ubiquity 20.04.15.3

It has been verified to work correctly on testing of that ISO, with the following in a preseed file:

    ubiquity partman-crypto/luksformat_options string \
        --type luks1

cheers,
Mauricio

[1] https://cdimage.ubuntu.com/focal/daily-live/20210121/
[2] https://cdimage.ubuntu.com/focal/daily-live/20210121/focal-desktop-amd64.manifest