NBDE with multiple disks does not work with clevis-initramfs

[Jaimes Joschko]

Hi,

I'm unable to get automatic Network-Bound Disk Encryption (NBDE) with multiple disks to work when using clevis-initramfs.
I see Bug "#1948598 Can't unlock multiple devices in initramfs" is the same symptom but I'm not certain it is the same cause and it says this has been fixed in jammy.

I've done the following testing:

Note: all machines (tang server, dracut client, initramfs client) are on Ubuntu 22.04 (Jammy Jellyfish).

# Set up tang server

```
sudo apt update && sudo apt upgrade -y
sudo apt install tang
sudo systemctl enable tangd.socket --now
systemctl show tangd.socket -p Listen
reboot
```

Note: The tang server is at 192.168.122.40.

# Set up clevis-dracut client (works as expected)

start with ubuntu-22.04.2-desktop-amd64.iso and install it in a VM with Virtual Machine Manager (i.e. libvirt)
    - Configure to use LVM with disk encryption
    - Otherwise just the defaults

```
sudo apt update && sudo apt upgrade -y
sudo apt install clevis clevis-luks clevis-udisks2 clevis-systemd clevis-dracut
sudo clevis luks bind -d /dev/vda4 tang '{"url": "http://192.168.122.40:80"}'
sudo dracut -fv --regenerate-all --hostonly-cmdline
sudo reboot
```

The disk is automatically unlocked as expected.
Powering off, and adding a second virtual disk. Then rebooting.

```
ubuntu@ubuntu-01:~$ lsblk -e 7
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sr0 11:0 1 1024M 0 rom
vda 252:0 0 25G 0 disk
\u251c\u2500vda1 252:1 0 1M 0 part
\u251c\u2500vda2 252:2 0 513M 0 part /boot/efi
\u251c\u2500vda3 252:3 0 1.7G 0 part /boot
\u2514\u2500vda4 252:4 0 22.8G 0 part
  \u2514\u2500vda4_crypt 253:0 0 22.8G 0 crypt
    \u251c\u2500vgubuntu-swap_1 253:1 0 2.5G 0 lvm [SWAP]
    \u2514\u2500vgubuntu-root 253:2 0 20.3G 0 lvm /var/snap/firefox/common/host-hunspell
                                                /
vdb 252:16 0 20G 0 disk
```

Setting up the second disk:

```
sudo cryptsetup luksFormat /dev/vdb
sudo cryptsetup luksOpen /dev/vdb disk2
sudo mkfs.ext4 /dev/mapper/disk2
sudo mkdir /disk2
```

Added `/dev/mapper/disk2 /disk2 ext4 defaults 0 0` to `/etc/fstab`
Added `disk2 /dev/vdb none` to `/etc/crypttab`

```
sudo reboot
```

The first disk vda4 decrypts automatically and the second disk prompts for the password, as expected.

Setting up clevis for the second disk:

```
sudo clevis luks bind -d /dev/vdb tang '{"url": "http://192.168.122.40:80"}'
sudo dracut -fv --regenerate-all --hostonly-cmdline
```

```
sudo reboot
```

Both disks decrypt automatically (starting with vda4, then vdb), as expected.
Note: You need to wait about 20 seconds

So with clevis-dracut this all seem to work fine.

# Set up clevis-initramfs client (does not work as expected)

start with ubuntu-22.04.2-desktop-amd64.iso and install it in a VM with Virtual Machine Manager (i.e. libvirt)
    - Configure to use LVM with disk encryption
    - Otherwise just the defaults

```
sudo apt update && sudo apt upgrade -y
sudo apt install clevis clevis-luks clevis-udisks2 clevis-systemd clevis-initramfs
sudo clevis luks bind -d /dev/vda4 tang '{"url": "http://192.168.122.40:80"}'
sudo update-initramfs -u -k 'all'
sudo reboot
```

The disk is automatically unlocked as expected.
Powering off, and adding a second virtual disk. Then rebooting.

```
ubuntu@ubuntu-01:~$ lsblk -e 7
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sr0 11:0 1 1024M 0 rom
vda 252:0 0 25G 0 disk
\u251c\u2500vda1 252:1 0 1M 0 part
\u251c\u2500vda2 252:2 0 513M 0 part /boot/efi
\u251c\u2500vda3 252:3 0 1.7G 0 part /boot
\u2514\u2500vda4 252:4 0 22.8G 0 part
  \u2514\u2500vda4_crypt 253:0 0 22.8G 0 crypt
    \u251c\u2500vgubuntu-swap_1 253:1 0 2.5G 0 lvm [SWAP]
    \u2514\u2500vgubuntu-root 253:2 0 20.3G 0 lvm /var/snap/firefox/common/host-hunspell
                                                /
vdb 252:16 0 20G 0 disk
```

Setting up the second disk:

```
sudo cryptsetup luksFormat /dev/vdb
sudo cryptsetup luksOpen /dev/vdb disk2
sudo mkfs.ext4 /dev/mapper/disk2
sudo mkdir /disk2
```

Added `/dev/mapper/disk2 /disk2 ext4 defaults 0 0` to `/etc/fstab`
Added `disk2 /dev/vdb none` to `/etc/crypttab`

```
sudo reboot
```

Neither disk decrypts automatically and you must enter the passphrase for both disks. Not expected.

Setting up clevis for the second disk:

```
sudo clevis luks bind -d /dev/vdb tang '{"url": "http://192.168.122.40:80"}'
sudo update-initramfs -u -k 'all'
```

```
sudo reboot
```

Neither disk decrypts automatically and you must enter the passphrase for both disks. Not expected.

So with clevis-initramfs, multi disk decryption does not seem to work as expected.
I dug through the logs but I don't really see much of a difference between dracut and initramfs across the different boots.

I'm happy to help test a fix but I don't have the knowledge/skills yet to dig much further into this on my own.

Thanks for spending the time reading through all this. It is very much appreciated.

Cheers,
Jaimes Joschko

[Niclas Morin (brydling)]

Did you append 'initramfs' to the disks in /etc/crypttab before doing 'sudo update-initramfs ...'?

Don't know if it is needed in your case, or if it is added automatically when performing the 'sudo clevis luks bind ...'. I'm just reading my old bug report and seeing I mention this, as if I had to do it manually.

Regards,
Niclas

[Niclas Morin (brydling)]

My two lines in /etc/crypttab looks like this:
luks-ata-WDC_....-partX /dev/disk/by-uuid/abcdef-.... none luks,discard,initramfs
luks-ata-WDC_....-partX /dev/disk/by-uuid/123456-.... none luks,discard,initramfs

[Jaimes Joschko (jjoschko)]

@brydling++

That's the issue. I was missing the initramfs option in /etc/crypttab. Adding that then sudo update-initramfs -u -k 'all' fixed the issue.

I must have missed that in your old bug report.

Thanks you for the clarification!

Very much appreciated!
Jaimes Joschko

[Niclas Morin (brydling)]

@jjoschko No problem! Glad I could help!