Merely booting Ubuntu 24.04 beta live CD breaks BitLocker & booting anything using Shim

[Ratchanan Srirattanamet]

On my ASUS Transformer Mini T103HAF, it seems like the dbx in Ubuntu 24.04 is too big or something, it breaks TPM's PCR7 measurement. And since secureboot-db.service runs in a live CD, simply booting one will apply this dbx. This causes BitLocker in the existing Windows installation to fails to unlock & require recovery key.

Step to reproduce:
1. Boot into pre-installed Windows installation. Run "msinfo32.exe" as administrator, go to "System Summary", and verify that "PCR7 Configuration" is shown as "Bound".
2. Before proceed further, make sure that you have BitLocker recovery key available (or have it suspended). I have one backed up in Microsoft Account [1], or I think you can also open a terminal as administrator and run `manage-bde -protectors -get "C:"`.
3. Restart the machine into USB flashed with Ubuntu 24.04 live CD. You don't have to finish setting up language, keyboard layout etc.; you can restart the system as soon as a GUI appear. By that point, "secureboot-db.service" should have run.
4. Boot back to Windows. At this point, Windows will fail to boot asking for BitLocker recovery key. Input the key you have from step 2. After that, the system will reboot itself again.
5. Run "msinfo32.exe" as administrator again. Go to "System Summary", and notice that "PCR7 Configuration" is now shown as "Binding not possible".
6. Restart into the live USB again. This time, it won't boot, failing with:

```
Could not create MokListRT: Volume full
Could not create MokListXRT: Volume full
Could not create SbatlevelRT: Volume full
Could not create MokListTrustedRT: Volume full
Something has gone seriously wrong: import_mok_state() failed : Volume Full
```

And shortly after that, the machine turns off.

[1]: https://account.microsoft.com/devices/recoverykey

---

Now, to verify that dbx is indeed the source of the problem, I did the following:

1. Reboot the system to the UEFI settings, go to Security > Secure Boot > Key Management, select Forbidden Signature > Set new key > Yes (restore to factory).
2. Reboot to Windows again. It will probably ask for BitLocker recovery key again. But once booted, run "msinfo32.exe" as administrator, go to "System Summary". "PCR7 Configuration" is now shown as "Bound" again.
3. Reboot into live USB again. This time it will boot. For debugging purpose, go to terminal and run `sudo mokutil --set-verbosity true`. Reboot to live USB.
4. Shim will now prints verbose message. Because I can't screencapture the boot process, I can't really transcribe the whole log to text (and my photo is of low quality). However, there are repeating lines along the line of (by now I'm using another Shim binary from Fedora 39 boot USB):

```
mok.c:798:mirror_one_mok_variable() tpm_log_event(0x73207F18, 76, 14, "MokListX")->Volume Full
Could not create MokListXRT: Volume Full
mok.c:926:import_one_mok_state() returning Volume Full
```

Or:

```
mok.c:762:mirror_one_mok_variable() tpm_measure_variable("SbatLevel",46,0x73207F98)->Volume Full
Could not create SbatLevelRT: Volume Full
mok.c:926:import_one_mok_state() returning Volume Full
```

---

I think there might be 2 problems that has to both be solved:

1. The dbx update causes breakage to TPM measured boot on this particular firmware.
2. Shim considers failure in TPM measured boot to be fatal and refuses to boot at all (as oppose to Windows which will still at least boot even if it will have to ask for recovery key later on).

---

Information about my tablet:

Make & model: ASUS Transformer Mini T103HAF
Firmware (BIOS) make & version: American Megatrends Inc. T103HAF.309, 22/4/2019
TPM manufacturer: INTC
TPM manufacturer version: 2.0.5.3015
TPM specification version: 2.0

[Mate Kukri (mkukri)]

> 1. The dbx update causes breakage to TPM measured boot on this particular firmware.

The dbx update causes the measurements in PCR 7 to change yes. This is not breakage and is required and expected.
BitLocker asking for a recovery key in this scenario is normal behavior. Entering the recovery key once should resolve that issue.
The reason you clearing the dbx "resolves" this is because it resets to the outdated, insecure configuration the bitlocker key was sealed against.

> 2. Shim considers failure in TPM measured boot to be fatal and refuses to boot at all (as oppose to Windows which will still at least boot even if it will have to ask for recovery key later on).

What shim considers fatal here is the failure to write required UEFI variables to the system's variable store. These are used by shim to communicate important information to the kernel, hence this failure has to be fatal.
This stems from the firmware's failure to perform UEFI variable garbage collection, leading to the variable store filling up and shim not being able to install the required variables.
If the vendor in question has provided newer firmware updates, try applying that, otherwise there is little we can do here.

[Mate Kukri (mkukri)]

The only workaround for a system with such broken firmware (in absence of a firmware update that resolves the issue) is to disable UEFI Secure Boot entirely.

[Mate Kukri (mkukri)]

Perhaps we could remove secureboot-db from the ISO, so that only installing Ubuntu causes the varstore filling up on such systems, but Secure Boot isn't easily supportable on buggy firmware unfortunately.

[Ratchanan Srirattanamet (peat-new)]

> The dbx update causes the measurements in PCR 7 to change yes. This is not breakage and is required and expected.
> BitLocker asking for a recovery key in this scenario is normal behavior. Entering the recovery key once should resolve that issue.

Hmm Ok. Indeed the fact that updating dbx causes PCR7 measurement to change is expected. But it should not cause the measurement to "fail".

See, with subsequent testing, in the state that dbx is populated and PCR7 is broken, now my Windows installation asks for recovery key on _every_ boot. And as I mentioned, now msinfo32.exe complains that it's not possible to form a new PCR7 binding. Isn't BitLocker supposeed to re-seal the key after the recovery key is entered?

> What shim considers fatal here is the failure to write required UEFI variables to the system's variable store. These are used by shim to communicate important information to the kernel, hence this failure has to be fatal.

Not exactly. One detail I forgot to mention is that, when booted to Ubuntu, there's _plenty_ of space available in `efivars` filesystem, even after secureboot-db.service has done its job.

```
$ df -h /sys/firmware/efi/efivars
Filesystem Size Used Avail Use% Mounted on
efivarfs 128K 67K 57K 55% /sys/firmware/efi/efivars
```

So it's implausible that the failure actually comes from UEFI variables. And according to one of verbose Shim boot, it doesn't. Rather, the failure actually comes from `tpm_log_event()` and `tpm_measure_variable()`, as mentioned. (Side Note: This is one of the reason debugging this issue took me so long).

Please see my attempt to use a phone camera to capture a verbose boot log from Shim [1]. Unfortunately the quality isn't so good, but you should be able to make out that the failure doesn't come from `SetVariable()`.

Oh, and cherry-on-top: I forgot to mention that disable Secure Boot does NOT fix booting with Shim. The boot I took [1] from, I have Secure Boot disabled, yet the boot still fails. The only way to fix booting with Shim is resetting the dbx.

[1]: https://ibb.co/album/RzXY28

[Mate Kukri (mkukri)]

I am still pretty sure it is the variable storage failure mode we are seeing in this firmware. I also personally own an ASUS device with firmware from 2019 and I managed to reach the same failure mode on it even when using Mantic.

Also let me try to explain why errors crop up in other places too:

You can see multiple failures besides 'tpm_measure_variable()' including:
- 'Create SbatLevelRT : Volume Full',
- 'Create MokListRT: Volume Full', and
- 'import_mok_state() failed Volume Full' on the last image.

These all internally call `SetVariable()`.

The reason I suspect 'tpm_measure_variable()' also fails is because presumably it tries to internally write to the varstore and then propagates this error.

Also keep in mind the used space and percentage indicator provided by the kernel for efivarfs might very well not be accurate at all.

If disabling Secure Boot really isn't enough, you can try removing shim-signed and then installing just GRUB on this device.

[Mate Kukri (mkukri)]

@peat-new

After some discussion about the upstream bug, I'm re-opening this as 'New'.

It seems that 'Volume Full' here indeed only refers to the event log filling up and not the entire variable store.

The shim debug log was confusing me into thinking it was the full variable store.

This could possible be fixed by allowing shim to drive forward in the face of errors, because as mentioned the PCRs are indeed extended just not the event log.

Could you possibly verify this theory by disabling TPM support in the firmware and seeing if the device boots via shim with that?

[Ratchanan Srirattanamet (peat-new)]

Unfortunately, I can't seem to find a setting to disable TPM in my device's firmware settings, so I can't test that.

[Mate Kukri (mkukri)]

I have proposed a patch for this now: https://github.com/rhboot/shim/pull/657