IMA Hashes keep changing on every reboot (PCR10)

[Magesh GV]

In my understanding (not an expert on this) the linux integrity measure hash should not change unless there is a real change to kernel/modules or the aggregate boot measure.

We are tying to use IMA for trusted boot and attestation. However, on 6.5.0-35-generic (ubuntu 22.04), the IMA hash keeps changing on every reboot without any software updates.

I may be wrong about the root cause, but it may be related to the order of evaluation of the kernel module files ?

Diff of /sys/kernel/security/ima/ascii_runtime_measurements between 2 reboots:

--- ascii_runtime_measurements22 2024-06-06 14:00:23.440000000 +0000
+++ ascii_runtime_measurements21 2024-06-06 13:58:33.229038384 +0000
@@ -2,14 +2,14 @@
 10 b1d60291291154dcef902e2a8c23772d48798148 ima-ng sha1:b952f8331430d08db2931db38713342a45dcb9e1 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/usb/host/xhci-pci-renesas.ko
 10 6489a4f054c3d0c4df0f645a74f8f730dec9af7f ima-ng sha1:01f17ddccffb8cbc8651b46f91916b21258ba82b /usr/lib/modules/6.5.0-35-generic/kernel/drivers/char/hw_random/virtio-rng.ko
 10 dc9529d9c1a17ea7d7ada8218068c975bad1153f ima-ng sha1:e84dbae74b7f246b964d3b073b9a8847dd8e408f /usr/lib/modules/6.5.0-35-generic/kernel/drivers/usb/host/xhci-pci.ko
-10 3f5e368749dbff84d3a7410a1b4c4a7fab66b559 ima-ng sha1:0bc18fb894d2f5b04331b239e0e6073b51354211 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/gpu/drm/drm.ko
 10 17cccb8cb394efb7efbee1aca74c79c1d2f8a38e ima-ng sha1:2f902f7314e44bba2d3056e6340d587c376f641a /usr/lib/modules/6.5.0-35-generic/kernel/drivers/i2c/i2c-smbus.ko
-10 d19437485bf5540a30de2cca2de936fd73580369 ima-ng sha1:99e31489a8d3a958411ffd6e99c8ea0d0d01c210 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/ata/libahci.ko
-10 591a35a9de40e752cfc9f85194a31ef97d0b1623 ima-ng sha1:a0919355cf28b07a7ec1a1f641cf1a4ed4219691 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/i2c/busses/i2c-i801.ko
+10 3f5e368749dbff84d3a7410a1b4c4a7fab66b559 ima-ng sha1:0bc18fb894d2f5b04331b239e0e6073b51354211 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/gpu/drm/drm.ko
 10 4d2af98b6a28806abe7e47ac7e830f81fa43878f ima-ng sha1:4190f2cc17a89dac6afae4575910487409a47b29 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/mfd/lpc_ich.ko
+10 d19437485bf5540a30de2cca2de936fd73580369 ima-ng sha1:99e31489a8d3a958411ffd6e99c8ea0d0d01c210 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/ata/libahci.ko
 10 2a4b0265d5807763cd7784617d61ab8dd97d4844 ima-ng sha1:32a600680fd22682c12fb34d2e16ceb4c6415fd6 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/input/mouse/psmouse.ko
-10 6239ce08348df615bc4056538fc98543c4ccb33b ima-ng sha1:4b0216c96c99bfbab72daa30df057e378029123b /usr/lib/modules/6.5.0-35-generic/kernel/crypto/cryptd.ko
 10 41b5ef647a337225aff73c125320db0101f87825 ima-ng sha1:27856f6182f8e76688055184db94e1756d77da59 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/ata/ahci.ko
+10 591a35a9de40e752cfc9f85194a31ef97d0b1623 ima-ng sha1:a0919355cf28b07a7ec1a1f641cf1a4ed4219691 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/i2c/busses/i2c-i801.ko
+10 6239ce08348df615bc4056538fc98543c4ccb33b ima-ng sha1:4b0216c96c99bfbab72daa30df057e378029123b /usr/lib/modules/6.5.0-35-generic/kernel/crypto/cryptd.ko
 10 1d2d52cc82f2ff0943dc00008c43bf6a78722247 ima-ng sha1:f0c245e28ca906a8b3ced94eaaf872175095c24e /usr/lib/modules/6.5.0-35-generic/kernel/crypto/crypto_simd.ko
 10 9d233b196dac726c5e188f18b6efb38d24066917 ima-ng sha1:0b4ba623e888760dee0d1227d820058ff7e3e9d2 /usr/lib/modules/6.5.0-35-generic/kernel/drivers/gpu/drm/drm_kms_helper.ko
 10 8283f095fbd71a3fd6ea8ee96299a8697386b6fa ima-ng sha1:c4c9542d63c603275d08468045e56d45a3f06dee /usr/lib/modules/6.5.0-35-generic/kernel/arch/x86/crypto/aesni-intel.ko
--snipped-----

Diff of tpm2_pcrread:
--- pcr22 2024-06-06 14:00:20.196000000 +0000
+++ pcr21 2024-06-06 13:58:27.795767357 +0000
@@ -10,7 +10,7 @@
     7 : 0x9A4E36070648A8DF6FCE7CA435446C541729BF6D93E4C41915655C77D152CEBB
     8 : 0xC3BFD21B69B10AD69421BBF0B5DD649A99B8C45BB4025A096FDCD300C71193C5
     9 : 0x32D3F1ABC6F853A521F985CD516AB2ED5FE12D8E3F3E40CD60E6D613A70B53EC
- 10: 0xE1781081A5D620A58482C5F81B71B22198D535B66B6F490DF5CFC5D4D2009522
+ 10: 0xA35CF31BFAC4A1E547CAFE9CFE5BFCFC6D5E0E9EA8E6C520D93156A8C743CA1E
     11: 0x0000000000000000000000000000000000000000000000000000000000000000
     12: 0x0000000000000000000000000000000000000000000000000000000000000000
     13: 0x0000000000000000000000000000000000000000000000000000000000000000

[Hector CAO (hectorcao)]

I see only a change in the order of the measurements but not on the measurement values, for example, drm.ko has the exact hash value for both boots

[Magesh GV (magesh-gv)]

Sorry, I have now updated the report addding the diff of tpm2_pcrread

[Hector CAO (hectorcao)]

Thanks can you give more details on your setup for "trusted boot and attestation" ?

[Magesh GV (magesh-gv)]

Hi Hector,

LF Edge Measured Boot and Remote Attestation document is a good picture of what we are trying to do:
https://wiki.lfedge.org/spaces/flyingpdf/pdfpageexport.action?pageId=27722830

While our specific setup uses some non standard stuff like iso boot, please find a simple setup to reproduce this.

1. Enable Secure Boot in Bios if using a PC with TPM or use a Virtual machine with vtpm and Secure boot:
<tpm model="tpm-crb">
  <backend type="emulator" version="2.0"/>
  <alias name="tpm0"/>
</tpm>
  <os firmware="efi">
    <type arch="x86_64" machine="pc-q35-8.2">hvm</type>
    <firmware>
      <feature enabled="yes" name="enrolled-keys"/>
      <feature enabled="yes" name="secure-boot"/>
    </firmware>
    <loader readonly="yes" secure="yes" type="pflash">/usr/share/OVMF/OVMF_CODE_4M.ms.fd</loader>
    <nvram template="/usr/share/OVMF/OVMF_VARS_4M.ms.fd">/var/lib/libvirt/qemu/nvram/ubuntu24.04_VARS.fd</nvram>
    <boot dev="hd"/>
  </os>

2. Install ubuntu 22.04. The default 5.15 kernel does not perform kernel module integrity measurements as seen from /sys/kernel/security/ima/ascii_runtime_measurements. Install hwe kernel package ( linux-image-generic-hwe-22.04 ) to upgrade to 6.15 where the kernel module integrity is checked as well. I see some minor build flags changed between the two for CONFIG_IMA and CONFIG_INTEGRITY. But, at this step, PCR10 changes on every reboot.