| 2022-08-17 09:10:14 |
Lorenzhuether@hotmail.de |
description |
During boot, cryptsetup fails to decrypt the root partition in a, seemingly, non-deterministic fashion. I know that the password is correct and that the keymap is not a fault either, because I have specifically chosen the very weak password "123456" for testing purposes. A hardware defect seems also rather unlikely as this behavior does not affect other Linux distributions or FreeBSD. Earlier Ubuntu versions do not seem to be affected either, as this bug appears to have been introduced during a kernel update in 20.04 and persists throughout 20.04-22.04. Unfortunately I cannot pinpoint the exact kernel update that introduced this bug. I have appended the output of cryptsetup when manually called from the initramfs shell. Here the second attempt succeeded in decrypting the root partition, however, it usually takes a lot more attempts to do so.
As for some additional information, I can decrypt the same luks partition from a live USB without any problems whatsoever.
echo 123456 | cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt
# cryptsetup 2.4.3 processing "cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt"
# Running command open.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating context for crypt device /dev/nvme0n1p3.
# Trying to open and read device /dev/nvme0n1p3 with direct-io.
# Initialising device-mapper backend library.
# Trying to load any crypt type from device /dev/nvme0n1p3.
# Crypto backend (OpenSSL 3.0.2 15 Mar 2022 [default]) initialized in cryptsetup library version 2.4.3.
# Detected kernel Linux 5.15.0-46-generic x86_64.
# Loading LUKS2 header (repair disabled).
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Trying to read primary LUKS2 header at offset 0x0.
# Opening locked device /dev/nvme0n1p3
# Verifying locked device handle (bdev)
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (on-disk)
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (in-memory)
# Trying to read secondary LUKS2 header at offset 0x4000.
# Reusing open ro fd on device /dev/nvme0n1p3
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (on-disk)
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (in-memory)
# Device size 497776852992, offset 16777216.
# Device /dev/nvme0n1p3 READ lock released.
# PBKDF argon2id, time_ms 2000 (iterations 0), max_memory_kb 1048576, parallel_threads 4.
# Activating volume nvme0n1p3_crypt using token (any type) -1.
# dm version [ opencount flush ] [16384] (*1)
# dm versions [ opencount flush ] [16384] (*1)
# Detected dm-ioctl version 4.45.0.
# Detected dm-crypt version 1.23.0.
# Device-mapper backend running with UDEV support enabled.
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
No usable token is available.
# STDIN descriptor passphrase entry requested.
# Activating volume nvme0n1p3_crypt [keyslot -1] using passphrase.
# dm versions [ opencount flush ] [16384] (*1)
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
# Keyslot 0 priority 1 != 2 (required), skipped.
# Keyslot 1 priority 1 != 2 (required), skipped.
# Trying to open LUKS2 keyslot 0.
# Running keyslot key derivation.
# Reading keyslot area [0x8000].
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Reusing open ro fd on device /dev/nvme0n1p3
# Device /dev/nvme0n1p3 READ lock released.
# Verifying key from keyslot 0, digest 0.
# Digest 0 (pbkdf2) verify failed with -1.
# Trying to open LUKS2 keyslot 1.
# Running keyslot key derivation.
# Reading keyslot area [0x47000].
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Reusing open ro fd on device /dev/nvme0n1p3
# Device /dev/nvme0n1p3 READ lock released.
# Verifying key from keyslot 1, digest 0.
# Digest 0 (pbkdf2) verify failed with -1.
# Releasing crypt device /dev/nvme0n1p3 context.
# Releasing device-mapper backend.
# Closing read only fd for /dev/nvme0n1p3.
# Unlocking memory.
Command failed with code -2 (no permission or bad passphrase).
echo 123456 | cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt
# cryptsetup 2.4.3 processing "cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt"
# Running command open.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating context for crypt device /dev/nvme0n1p3.
# Trying to open and read device /dev/nvme0n1p3 with direct-io.
# Initialising device-mapper backend library.
# Trying to load any crypt type from device /dev/nvme0n1p3.
# Crypto backend (OpenSSL 3.0.2 15 Mar 2022 [default]) initialized in cryptsetup library version 2.4.3.
# Detected kernel Linux 5.15.0-46-generic x86_64.
# Loading LUKS2 header (repair disabled).
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Trying to read primary LUKS2 header at offset 0x0.
# Opening locked device /dev/nvme0n1p3
# Verifying locked device handle (bdev)
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (on-disk)
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (in-memory)
# Trying to read secondary LUKS2 header at offset 0x4000.
# Reusing open ro fd on device /dev/nvme0n1p3
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (on-disk)
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (in-memory)
# Device size 497776852992, offset 16777216.
# Device /dev/nvme0n1p3 READ lock released.
# PBKDF argon2id, time_ms 2000 (iterations 0), max_memory_kb 1048576, parallel_threads 4.
# Activating volume nvme0n1p3_crypt using token (any type) -1.
# dm version [ opencount flush ] [16384] (*1)
# dm versions [ opencount flush ] [16384] (*1)
# Detected dm-ioctl version 4.45.0.
# Detected dm-crypt version 1.23.0.
# Device-mapper backend running with UDEV support enabled.
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
No usable token is available.
# STDIN descriptor passphrase entry requested.
# Activating volume nvme0n1p3_crypt [keyslot -1] using passphrase.
# dm versions [ opencount flush ] [16384] (*1)
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
# Keyslot 0 priority 1 != 2 (required), skipped.
# Keyslot 1 priority 1 != 2 (required), skipped.
# Trying to open LUKS2 keyslot 0.
# Running keyslot key derivation.
# Reading keyslot area [0x8000].
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Reusing open ro fd on device /dev/nvme0n1p3
# Device /dev/nvme0n1p3 READ lock released.
# Verifying key from keyslot 0, digest 0.
# Loading key (64 bytes, type logon) in thread keyring.
# dm versions [ opencount flush ] [16384] (*1)
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
# Calculated device size is 972187648 sectors (RW), offset 32768.
# DM-UUID is CRYPT-LUKS2-a41dd71155cc4b6ca29d391c500c546d-nvme0n1p3_crypt
# Udev cookie 0xd4d65d7 (semid 0) created
# Udev cookie 0xd4d65d7 (semid 0) incremented to 1
# Udev cookie 0xd4d65d7 (semid 0) incremented to 2
# Udev cookie 0xd4d65d7 (semid 0) assigned to CREATE task(0) with flags DISABLE_LIBRARY_FALLBACK (0x20)
# dm create nvme0n1p3_crypt CRYPT-LUKS2-a41dd71155cc4b6ca29d391c500c546d-nvme0n1p3_crypt [ opencount flush ] [16384] (*1)
# dm reload (253:0) [ opencount flush securedata ] [16384] (*1)
# dm resume nvme0n1p3_crypt [ opencount flush securedata ] [16384] (*1)
# nvme0n1p3_crypt: Stacking NODE_ADD (253,0) 0:6 0660 [trust_udev]
# nvme0n1p3_crypt: Stacking NODE_READ_AHEAD 256 (flags=1)
# Udev cookie 0xd4d65d7 (semid 0) decremented to 1
# Udev cookie 0xd4d65d7 (semid 0) waiting for zero
# Udev cookie 0xd4d65d7 (semid 0) destroyed
# nvme0n1p3_crypt: Skipping NODE_ADD (253,0) 0:6 0660 [trust_udev]
# nvme0n1p3_crypt: Processing NODE_READ_AHEAD 256 (flags=1)
# nvme0n1p3_crypt (253:0): read ahead is 256
# nvme0n1p3_crypt: retaining kernel read ahead of 256 (requested 256)
Key slot 0 unlocked.
# Releasing crypt device /dev/nvme0n1p3 context.
# Releasing device-mapper backend.
# Closing read only fd for /dev/nvme0n1p3.
# Unlocking memory.
Command successful.
Release and package version below:
lsb_release -rd
Description: Ubuntu 22.04.1 LTS
Release: 22.04
apt-cache policy cryptsetup
cryptsetup:
Installed: 2:2.4.3-1ubuntu1.1
Candidate: 2:2.4.3-1ubuntu1.1
Version table:
*** 2:2.4.3-1ubuntu1.1 500
500 http://de.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
100 /var/lib/dpkg/status
2:2.4.3-1ubuntu1 500
500 http://de.archive.ubuntu.com/ubuntu jammy/main amd64 Packages |
During boot, cryptsetup fails to decrypt the root partition in a, seemingly, non-deterministic fashion. I know that the password is correct and that the keymap is not a fault either, because I have specifically chosen the very weak password "123456" for testing purposes. A hardware defect seems also rather unlikely as this behavior does not affect other Linux distributions or FreeBSD. Earlier Ubuntu versions do not seem to be affected either, as this bug appears to have been introduced during a kernel update in 20.04 and persists throughout 20.04-22.04. Unfortunately I cannot pinpoint the exact kernel update that introduced this bug. I have appended the output of cryptsetup when manually called from the initramfs shell. Here the second attempt succeeded in decrypting the root partition, however, it usually takes a lot more attempts to do so.
As for some additional information, I can decrypt the same luks partition from a live USB without any problems whatsoever.
echo 123456 | cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt
# cryptsetup 2.4.3 processing "cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt"
# Running command open.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating context for crypt device /dev/nvme0n1p3.
# Trying to open and read device /dev/nvme0n1p3 with direct-io.
# Initialising device-mapper backend library.
# Trying to load any crypt type from device /dev/nvme0n1p3.
# Crypto backend (OpenSSL 3.0.2 15 Mar 2022 [default]) initialized in cryptsetup library version 2.4.3.
# Detected kernel Linux 5.15.0-46-generic x86_64.
# Loading LUKS2 header (repair disabled).
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Trying to read primary LUKS2 header at offset 0x0.
# Opening locked device /dev/nvme0n1p3
# Verifying locked device handle (bdev)
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (on-disk)
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (in-memory)
# Trying to read secondary LUKS2 header at offset 0x4000.
# Reusing open ro fd on device /dev/nvme0n1p3
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (on-disk)
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (in-memory)
# Device size 497776852992, offset 16777216.
# Device /dev/nvme0n1p3 READ lock released.
# PBKDF argon2id, time_ms 2000 (iterations 0), max_memory_kb 1048576, parallel_threads 4.
# Activating volume nvme0n1p3_crypt using token (any type) -1.
# dm version [ opencount flush ] [16384] (*1)
# dm versions [ opencount flush ] [16384] (*1)
# Detected dm-ioctl version 4.45.0.
# Detected dm-crypt version 1.23.0.
# Device-mapper backend running with UDEV support enabled.
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
No usable token is available.
# STDIN descriptor passphrase entry requested.
# Activating volume nvme0n1p3_crypt [keyslot -1] using passphrase.
# dm versions [ opencount flush ] [16384] (*1)
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
# Keyslot 0 priority 1 != 2 (required), skipped.
# Keyslot 1 priority 1 != 2 (required), skipped.
# Trying to open LUKS2 keyslot 0.
# Running keyslot key derivation.
# Reading keyslot area [0x8000].
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Reusing open ro fd on device /dev/nvme0n1p3
# Device /dev/nvme0n1p3 READ lock released.
# Verifying key from keyslot 0, digest 0.
# Digest 0 (pbkdf2) verify failed with -1.
# Trying to open LUKS2 keyslot 1.
# Running keyslot key derivation.
# Reading keyslot area [0x47000].
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Reusing open ro fd on device /dev/nvme0n1p3
# Device /dev/nvme0n1p3 READ lock released.
# Verifying key from keyslot 1, digest 0.
# Digest 0 (pbkdf2) verify failed with -1.
# Releasing crypt device /dev/nvme0n1p3 context.
# Releasing device-mapper backend.
# Closing read only fd for /dev/nvme0n1p3.
# Unlocking memory.
Command failed with code -2 (no permission or bad passphrase).
echo 123456 | cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt
# cryptsetup 2.4.3 processing "cryptsetup open --type luks --debug /dev/nvme0n1p3 nvme0n1p3_crypt"
# Running command open.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating context for crypt device /dev/nvme0n1p3.
# Trying to open and read device /dev/nvme0n1p3 with direct-io.
# Initialising device-mapper backend library.
# Trying to load any crypt type from device /dev/nvme0n1p3.
# Crypto backend (OpenSSL 3.0.2 15 Mar 2022 [default]) initialized in cryptsetup library version 2.4.3.
# Detected kernel Linux 5.15.0-46-generic x86_64.
# Loading LUKS2 header (repair disabled).
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Trying to read primary LUKS2 header at offset 0x0.
# Opening locked device /dev/nvme0n1p3
# Verifying locked device handle (bdev)
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (on-disk)
# Checksum:99172356e66a2fec247b1e5c758af8bc1338a3fb8bd973aab5e1512a93b2dbdc (in-memory)
# Trying to read secondary LUKS2 header at offset 0x4000.
# Reusing open ro fd on device /dev/nvme0n1p3
# LUKS2 header version 2 of size 16384 bytes, checksum sha256.
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (on-disk)
# Checksum:655a50aef64b6fd4e10b8863df72d7fc62a7020f74684cb3419d4e22adb6fd9c (in-memory)
# Device size 497776852992, offset 16777216.
# Device /dev/nvme0n1p3 READ lock released.
# PBKDF argon2id, time_ms 2000 (iterations 0), max_memory_kb 1048576, parallel_threads 4.
# Activating volume nvme0n1p3_crypt using token (any type) -1.
# dm version [ opencount flush ] [16384] (*1)
# dm versions [ opencount flush ] [16384] (*1)
# Detected dm-ioctl version 4.45.0.
# Detected dm-crypt version 1.23.0.
# Device-mapper backend running with UDEV support enabled.
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
No usable token is available.
# STDIN descriptor passphrase entry requested.
# Activating volume nvme0n1p3_crypt [keyslot -1] using passphrase.
# dm versions [ opencount flush ] [16384] (*1)
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
# Keyslot 0 priority 1 != 2 (required), skipped.
# Keyslot 1 priority 1 != 2 (required), skipped.
# Trying to open LUKS2 keyslot 0.
# Running keyslot key derivation.
# Reading keyslot area [0x8000].
# Acquiring read lock for device /dev/nvme0n1p3.
# Opening lock resource file /run/cryptsetup/L_259:3
# Verifying lock handle for /dev/nvme0n1p3.
# Device /dev/nvme0n1p3 READ lock taken.
# Reusing open ro fd on device /dev/nvme0n1p3
# Device /dev/nvme0n1p3 READ lock released.
# Verifying key from keyslot 0, digest 0.
# Loading key (64 bytes, type logon) in thread keyring.
# dm versions [ opencount flush ] [16384] (*1)
# dm status nvme0n1p3_crypt [ opencount noflush ] [16384] (*1)
# Calculated device size is 972187648 sectors (RW), offset 32768.
# DM-UUID is CRYPT-LUKS2-a41dd71155cc4b6ca29d391c500c546d-nvme0n1p3_crypt
# Udev cookie 0xd4d65d7 (semid 0) created
# Udev cookie 0xd4d65d7 (semid 0) incremented to 1
# Udev cookie 0xd4d65d7 (semid 0) incremented to 2
# Udev cookie 0xd4d65d7 (semid 0) assigned to CREATE task(0) with flags DISABLE_LIBRARY_FALLBACK (0x20)
# dm create nvme0n1p3_crypt CRYPT-LUKS2-a41dd71155cc4b6ca29d391c500c546d-nvme0n1p3_crypt [ opencount flush ] [16384] (*1)
# dm reload (253:0) [ opencount flush securedata ] [16384] (*1)
# dm resume nvme0n1p3_crypt [ opencount flush securedata ] [16384] (*1)
# nvme0n1p3_crypt: Stacking NODE_ADD (253,0) 0:6 0660 [trust_udev]
# nvme0n1p3_crypt: Stacking NODE_READ_AHEAD 256 (flags=1)
# Udev cookie 0xd4d65d7 (semid 0) decremented to 1
# Udev cookie 0xd4d65d7 (semid 0) waiting for zero
# Udev cookie 0xd4d65d7 (semid 0) destroyed
# nvme0n1p3_crypt: Skipping NODE_ADD (253,0) 0:6 0660 [trust_udev]
# nvme0n1p3_crypt: Processing NODE_READ_AHEAD 256 (flags=1)
# nvme0n1p3_crypt (253:0): read ahead is 256
# nvme0n1p3_crypt: retaining kernel read ahead of 256 (requested 256)
Key slot 0 unlocked.
# Releasing crypt device /dev/nvme0n1p3 context.
# Releasing device-mapper backend.
# Closing read only fd for /dev/nvme0n1p3.
# Unlocking memory.
Command successful.
Release and package version below:
lsb_release -rd
Description: Ubuntu 22.04.1 LTS
Release: 22.04
apt-cache policy cryptsetup
cryptsetup:
Installed: 2:2.4.3-1ubuntu1.1
Candidate: 2:2.4.3-1ubuntu1.1
Version table:
*** 2:2.4.3-1ubuntu1.1 500
500 http://de.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
100 /var/lib/dpkg/status
2:2.4.3-1ubuntu1 500
500 http://de.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu82.1
Architecture: amd64
CasperMD5CheckResult: pass
DistroRelease: Ubuntu 22.04
InstallationDate: Installed on 2022-08-15 (1 days ago)
InstallationMedia: Ubuntu 22.04.1 LTS "Jammy Jellyfish" - Release amd64 (20220809.1)
Package: linux
PackageArchitecture: amd64
ProcVersionSignature: Ubuntu 5.15.0-46.49-generic 5.15.39
Tags: jammy
Uname: Linux 5.15.0-46-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
cmdline: BOOT_IMAGE=/vmlinuz-5.15.0-46-generic root=/dev/mapper/vgubuntu-root ro quiet splash break=mountroot
crypttab: nvme0n1p3_crypt UUID=a41dd711-55cc-4b6c-a29d-391c500c546d none luks,discard |
|
