Cryptsetup-initramfs cant deal with tpm2-device option

[Christopher Hall]

In order to boot an encrypted system and autounlock with tpm2, the tpm2-device= option must be specified in /etc/crypttab. This works for non-root filesystems for some reason, but when applied to root filesystems it doesnt. Tested working on both arch and fedora, so the method is good, something is off in the background.

root@test:~# update-initramfs -u
update-initramfs: Generating /boot/initrd.img-5.15.0-40-generic
cryptsetup: WARNING: sda3_crypt: ignoring unknown option 'tpm2-device'

Manually adding it to /lib/cryptsetup/functions produces this

root@test:~# update-initramfs -u
update-initramfs: Generating /boot/initrd.img-5.15.0-40-generic
/usr/share/initramfs-tools/hooks/cryptroot: 1: eval: CRYPTTAB_OPTION_tpm2-device=auto: not found

That file belongs to cryptsetup-initramfs

[Christopher Hall (christopher88hall)]

putting correct package on the bug

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Stewart Shearer (stewartshearer)]

Hey, just wanted to add my 2c here:
I'm closely following along -- I want to do automatic unlock on a laptop with TPM2. I tried a similar thing of adding tpm2-devices to the /lib/cryptsetup/functions file. There are a number of locations you need to change it; if you follow along one of the other options that takes arguments (like keyfile-size or keyfile-offset), you can eventually get update-initramfs to work without complaint.
However, on reboot, it was effectively bricked with an "unknown option" for cryptsetup (not exactly sure the wording).
I'm wondering if that means it's in cryptsetup-bin, not just the cryptsetup scripts? Running cryptsetup without any options shows it doesn't have that tpm2-device option, and if I look at cryptsetup.c, I can see it defines a bunch of variables like
static uint64_t opt_keyfile_offset = 0;

But tpm2-devices is not one of them.

All of which is to say -- it's not just that /lib/cryptsetup/functions file.... :)

[Christopher Hall (christopher88hall)]

@stewartshearer

I found two things yesterday that are relevant to what you're saying.

The option works for non-root file systems. I'm not sure why or how but something correctly interprets them and uses the TPM2 chip to unlock on boot.

Fedora, has this working for root file systems, and is running the same version of crypt setup. That doesn't mean they're compiled with the same options, but that the capability should already there in cryptsetup itself

[asi (gmazyland)]

Unlocking of TPM tokens is implemented in systemd LUKS2 token handlers, not in cryptsetup itself.
It can be compiled as plugins (then check cryptsetup --help "LUKS2 external token plugin path" and check that in this directory in initramfs are systemd plugins installed).

But systemd can also implement it directly in systemd-cryptsetup/cryptenroll - then you need only the systemd binaries. I am really not sure what Ubuntu uses now, the first option should be preferred for future.

[W McElderry (wmcelderry)]

Hi All,

This has been (rightly) divided in to two issues in launchpad, but I addressed it as one in my scripts. I've published them at:

https://github.com/wmcelderry/systemd_with_tpm2

I went all out to get this working a bit ago, I was in my final phase of testing when I ran out of time, so this may only be 90% of the way there - i.e. forgive/expect typos!

If you can successfully use cryptenroll with tpm2 then I suspect the bits you need for this to work are in the 'patches' directory of that repo.

If you cannot, then you may have joy using the full install.sh script from a fresh install of 22.04

Let me know how you get on & Good luck!

Will.

[W McElderry (wmcelderry)]

P.S. I've just re-read my last comment, and it's cryptic, so clarifying:
The two issues my repo addresses are:
  this issue
and
  https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1969375

Unfortunately my docs don't get much better in the repo README, but I believe the code speaks for itself... I hope so anyway, and if not ask here.

W.

[W McElderry (wmcelderry)]

@stewartshearer
I know you were trying to get this working - if you give the scripts I published ago it literally does (or instructs you to do) everything you need. Just run `./install.sh`

I've just re-run it on a system to upgrade to '249.11-0ubuntu3.4'

I'd love to hear how you get on.

[W McElderry (wmcelderry)]

I see that @gmazyland was saying there are two ways to implement this and neither is 'cryptsetup'.

Having got it working, my solution requires modification to two files which are packaged in `cryptsetup` and `cryptsetup-initramfs` in Ubuntu, and adding one file to `/etc/initramfs-tools/hooks` (or the platform equivalent location)

I've generated minimal patches of what needs doing in my repo under the 'patches' directory.
  https://github.com/wmcelderry/systemd_with_tpm2/tree/main/patches

and the extra hook script is under 'scripts':
  https://github.com/wmcelderry/systemd_with_tpm2/tree/main/scripts

I'm not claiming these are the best way to implement the solution, but it makes it very clear what I have done to get it working.

If there's a better way to implement this, I'd love to hear your thoughts and integrate it.

[sfc (sfc-0)]

Any update on this? This is really annoying on tablets such as surface go.

[Steve Langasek (vorlon)]

Using tpm-backed encryption with classic Ubuntu has known vulnerabilities and is largely security theater. There is work in progress on a secure tpm-backed implementation, which requires changes to the delivery of the kernel. In the meantime, bugs such as this are not a priority for the university developers.

[Steve Langasek (vorlon)]

*for the Ubuntu developers

[W McElderry (wmcelderry)]

@vorlon

Thanks for the comment. I'm interested in TPMs and have been for a while, so I'd love to hear more about the vulnerabilities you mention. Can you recommend somewhere to get more info?

[Steve Langasek (vorlon)]

On Thu, Sep 08, 2022 at 04:54:33PM -0000, W McElderry wrote:
> @vorlon

> Thanks for the comment. I'm interested in TPMs and have been for a
> while, so I'd love to hear more about the vulnerabilities you mention.
> Can you recommend somewhere to get more info?

I don't know of anything specifically published about this. But the root
issue is that if you load an initramfs, the initramfs is not measured, so
can be modified to steal control of the encrypted disk.

[W McElderry (wmcelderry)]

@vorlon

Thanks for that!

I've heard of the issue you raise and a couple of solutions:
 - unified kernel image that is booted directly from the BIOS.
 - sign the initrd and kernel with keys and instruct grub to respects those keys

I'm more interested in the unified kernel image approach as I like the minimal approach and am keen on using the secure boot functionality with my own keys.

Admittedly it's more of a pain when you update the kernel, but I've heard there are hooks that can be used to recreate the image after the initrd or kernel is updated. I've prototyped all of the above on an Arch system, but I haven't written all the hook scripts.

In conclusion: I'd encourage not to de-prioritise this as the other issues are not insurmountable, and this needs to be resolved to get the system running with encrypted root (at rest).

[xenago (xenago)]

I would also like to echo that this should not be de-prioritized... having usable TPM functionality is rather critical and it's a big pain point currently

[Christopher Hall (christopher88hall)]

Even if it is 'security theater', to store/fetch/use decryption keys in a tpm2 chip would allow users meet full disk encryption requirements and reboot remote machines without having someone go physically type in boot passwords, or relying on other tools

As of now, the only way to make this happen on Ubuntu is clevis. Clevis is Ok but I'd much rather use core programs like systemd-cryptenroll and cryptsetup-initramfs

[Steve Langasek (vorlon)]

tpm2-backed encryption without a signed initramfs is LESS SECURE than passphrase-based encryption, because anyone who can write to the disk can compromise the root filesystem without the admin ever typing the passphrase. Until we do have unified kernel images integrated into Ubuntu that address this, no, we are not going to be doing work to enable tpm2-backed encryption options.

[W McElderry (wmcelderry)]

@vorlon I understand the position you are taking. What you have said is true: without measuring the initrd, it does degrade security compared to passphrase-based encryption.

While I'm sure there are those who are disappointed, perhaps disagree even for their use, I'd suggest we avoid debating the relative merit or belief in how useful it may still be to have the functionality while the initrd vulnerability persists, and spend the energy on removing the vulnerability (as it doesn't seem like it's too much work anyway)!

To that end I'd propose we create a new ticket that discusses the issue of creating a Unified Kernel Image (unless someone knows of an appropriate one already?), and leave this thread to focus on the scripts that are used to unlock the LUKS container using the TPM.

Given that I have created quite small patches for files that implement a solution to unlocking the LUKS container using the TPM in the initramfs and that is what this thread is about, perhaps someone can either comment on the patches, or tell me where & how to submit the patches for review?

Thanks!

[Christopher Hall (christopher88hall)]

>It does degrade security compared to passphrase-based encryption.

So does allowing luks key retrieval from other volumes, like network, block devices, or USB keys, which have been mainstays for years. The odds someone is going to just walk off with your decryption keys when they are stored in a processor embedded tpm2 (intel Gen 8 or higher) is very very low. That usb or disk drive could easily grow legs and suffers the same vulnerabilities described above.

Ranting aside, I tried wmcelderry's patches today on a fresh 22.04 host. Looks like about 10 lines of code in two files to get the tpm2-device option working and an initramfs hook. It works well. I can reboot and watch it fetch the systemd-cryptenrolled key off the tpm2 and unlock itself. I did install the compiled systemd with tpm2 packages, but I think 22.04 has all of that working alredy so that may have been unneccesary. Thanks for putting that together

edit: Confirming this works out of the box with 22.04 with just the patches and the initramfs hook

[W McElderry (wmcelderry)]

@christopher88hall: Glad you got the scripts working, thanks for the feedback!

I take your point that this may be a narrowing of attack vectors over the best suitable alternatives for your use.

[W McElderry (wmcelderry)]

To anyone thinking about using the scripts:

Be sure you understand the impact of the initrd not being measured before you deploy this solution on valuable data!

My explanation is that it means anyone who can write to your /boot directory can replace your initrd with a modified/compromised version and then access unencrypted files without any password.

As I understand it, the attacker would need to run the attack in a device with access to the TPM initially (they cannot just clone the disk and use another computer to decrypt it) [as christopher88hall commented] . And it could be said that this same attack would compromise many common system deployments!

Some may think of this as an upgrade over the next best suitable option for them (which could be an unencrypted filesystem) and others will think of it as a major downgrade just to avoid typing a password. It's a choice for each of us to take in our own way, but make sure you know what you're decision is based on!

I'd hate for you to get a nasty surprise /after/ some malicious attacker just walked off with all your valuable data and you've potentially lost everything...

Much better to read about it first if there's any doubt in your mind and work out if this is better or worse than you currently have!

From christopher88hall's comments, I suspect he has it straight already, but ultimately his decision is between him and his data security policies.

[W McElderry (wmcelderry)]

Hi All,

I've made a very rough and ready script that creates a Unified Kernel Image (thanks again to the Arch Wiki authors!) that mitigates (to some degree) the issue raised in the last post (and earlier by @vorlon).

https://github.com/wmcelderry/unified_kernel_image

I've tested it on my laptop and it works for me to capture the current kernel, initrd, cmdline etc. into one file that is then measured. I've done very brief testing that a new key is added and only automatically unlocked when booting that UKI. Seems to work, but I'll not be surprised if there are some important features that need to be added still.

It would be great if someone is willing to confirm it works for them, then I'll look at how to get these two components put in to Ubuntu (I don't expect that'll be quick/easy!)

I'd not be surprised to find that there are more security issues that need to be addressed, but short of going full on 'secure boot' (which I may still do in time) I doubt anything will be perfect. In short, it's another small step in the right direction.

Thanks!

Will.

[David Manouchehri (manouchehri)]

@vorlon: initramfs should be measured by PCR9 already, no? (Assuming you're running >=Linux 5.17.)

> tpm2-backed encryption without a signed initramfs is LESS SECURE than passphrase-based encryption

tpm2-backed encryption without a signed initramfs is MORE SECURE than no encryption.

There's a very common use-case here: servers with no (easy) physical access, that are currently left unencrypted. Having encryption on everything is still nice, as you can be less paranoid when it comes to recycling drives.

[Christopher Hall (christopher88hall)]

>Having encryption on everything is still nice, as you can be less paranoid when it comes to recycling drives.

Its not just convenience. There is a big increase in security because there is a MASSIVE technical skill gap between someone being able to exploit a situation where someone has to manipulate an initramfs to fetch tpm stored keys, and someone simply walking off with unencrypted drives or booting a usb and mounting them, finding you key file for encrypted drives sitting around on other more-removable media. How many people can do the latter and how many people can do the former?

If you think of it on a gradient, its like 0-10 security are allowed, but its not worth properly implementing 8 because its not 10. Even if it's not a 10 right now, make it a proper 8 and then push to 10.

Looking over Mcelderderry's code it seemed trivial to do.

[geppi (tgeppert-digitx)]

I can also confirm that this works with a recent 22.04 by just applying the two patches and the single initramfs hook from W McElderry. Cudos and thank you very much for this straightforward fix!

[Bruno Redondi (brunor73)]

The two patches and the hook work with 22.10 (systemd 251.4), but fail with 23.04 preview.
Tried with 22.10 and systemd 252.4 and it also fail.

[Ari (ari-reads)]

I've been using the @wmcelderry patches for some time on 22.04.2 and they work great - so much better than using Clevis (clevis also works but takes like 20s to open the disk on every boot).

Looking forward to seeing these patches adopted by ubuntu for the next release to make tpm2 usable

[Bruno Redondi (brunor73)]

Found a fork that works with systemd 252.4.

With the new version of systemd the initramfs should contain also the file /lib/x86_64-linux-gnu/cryptsetup/libcryptsetup-token-systemd-tpm2.so

https://github.com/larssilven/systemd_with_tpm2

[Petr Mandys (peteatpassengera)]

Just one note to the patches (both from Lars and WM). There is a bug in systemd_cryptsetup_hook. Line

copy_exec /lib/systemd/systemd-cryptsetup /lib/systemd

It creates a file /lib/systemd unless the directory /lib/systemd already exists. It should be

copy_exec /lib/systemd/systemd-cryptsetup /lib/systemd/systemd-cryptsetup

[Grumpus (thegrumpus)]

Steve Langasek (@vorlon) wrote on 2022-09-08:

"I don't know of anything specifically published about this. But the root
issue is that if you load an initramfs, the initramfs is not measured, so
can be modified to steal control of the encrypted disk"

Is there a source to support this statement? As far as I can tell Ubuntu does measure the initramfs into the PCRs, which are in turn utilised to ensure the boot is unmodified.

If it is in fact measured (as also appears the other various boot components as far as I understand) then can it really be designated a "security theater"?

And if it’s not a "security theater" then this bug should be prioritised surely? (or at least not deprioritised).

There needs to be something authoritative to support the statements being made around “security theatre” and requirements for Unified Kernel Images. As far as I understand the individual components of the boot can (and are already in Ubuntu?) measured.

[Steve Langasek (vorlon)]

On Sun, Oct 29, 2023 at 11:47:41PM -0000, Grumpus wrote:
> Is there a source to support this statement? As far as I can tell
> Ubuntu does measure the initramfs into the PCRs

What gives you that impression? What PCR do you see being extended by GRUB
with a hash of the initramfs when loaded?

[Christopher Hall (christopher88hall)]

The security theater argument is vacuous anyways, because you can already boot from a luks key stored on a removable media like a USB drive. Tpm is way more secure than that.

It sounds like they're going to do an implementation of it with a lot more security around it in a future Ubuntu release, but there's no real compelling security reason this couldn't be implemented as it is today, and improved later.

[Steve Langasek (vorlon)]

https://ubuntu.com/blog/tpm-backed-full-disk-encryption-is-coming-to-ubuntu

Not "a future Ubuntu release". The implementation is there today.

[Christopher Hall (christopher88hall)]

Cool. I guess the future is today. Are there plans to backport any of that functionality?

If not, I may not be able to use this in prod until LTS 24.04 gets fips approved, which may not be until the year 2026

[Grumpus (thegrumpus)]

"What gives you that impression? What PCR do you see being extended by GRUB
with a hash of the initramfs when loaded?"

I found if I update initramfs on Ubuntu 22.04 then PCR9 changes.

I only tested this as below lead me to believe this was an intended behaviour:

https://uapi-group.org/specifications/specs/linux_tpm_pcr_registry/
https://wiki.archlinux.org/title/Trusted_Platform_Module#Accessing_PCR_registers
https://www.gnu.org/software/grub/manual/grub/html_node/Measured-Boot.html
https://github.com/rhboot/shim/blob/main/README.tpm

Hence when I read your original comment it left me wondering whether I'm misunderstanding something.

[Steve Langasek (vorlon)]

On Mon, Oct 30, 2023 at 01:47:49AM -0000, Grumpus wrote:
> "What gives you that impression? What PCR do you see being extended by GRUB
> with a hash of the initramfs when loaded?"

> I found if I update initramfs on Ubuntu 22.04 then PCR9 changes.

> I only tested this as below lead me to believe this was an intended
> behaviour:

> https://uapi-group.org/specifications/specs/linux_tpm_pcr_registry/
> https://wiki.archlinux.org/title/Trusted_Platform_Module#Accessing_PCR_registers
> https://www.gnu.org/software/grub/manual/grub/html_node/Measured-Boot.html
> https://github.com/rhboot/shim/blob/main/README.tpm

> Hence when I read your original comment it left me wondering whether I'm
> misunderstanding something.

Thanks. I didn't know that GRUB was measuring into PCR9. But I also think
this is not a PCR we are sealing against in our implementation due to
fragility, and I would expect others not to want to seal against it either
(except for the fact that you would need to, to be secure against initramfs
bypass).

[Steve Langasek (vorlon)]

On Mon, Oct 30, 2023 at 01:37:47AM -0000, Christopher Hall wrote:
> Cool. I guess the future is today. Are there plans to backport any of
> that functionality?

No, it's very much not backportable.

[Grumpus (thegrumpus)]

@enr0n

In the light of the above clarifications can we have this moved up from "Wishlist" importance? I think the original assessment was made with misleading information.

[Christopher Hall (christopher88hall)]

The whole thing may not be backportable, but it would seem trivial to backport the part that allows fetching of keys from the TPM.

I imagine it would look a lot like McElderry's patches