[FFe] secure boot: TPM version '2.0' is not supported

[Andrea Righi]

Dear Release Team,

Please accept this update to swtpm to version 0.7.3 as a Lunar FFe.

PPA: https://launchpad.net/~lvoytek/+archive/ubuntu/swtpm-update-lunar

[Rationale]

Virtual machines with secure boot capabilities currently can not be created in Lunar. This includes Windows 11 and other vms secured with TPM. This is caused by the current version of swtpm not reporting that it has TPM 1.0 and TPM 2.0 capabilities. The best way to fix this alongside lunar's version of Libvirt is to update swtpm to the supported upstream version 0.7.3 from 0.6.3.

[Regression Potential]

Since this is a version update, issues can be caused by upstream changes. These would most likely be related to changes in swtpm_setup and swtpm_localca, which have seen a decent amount of updates and fixes between versions. swtpm itself has also had various bug fixes between versions that may change behavior.

[Proposed upload]

Code: https://code.launchpad.net/~lvoytek/ubuntu/+source/swtpm/+git/swtpm/+merge/439532

Build: https://launchpad.net/~lvoytek/+archive/ubuntu/swtpm-update-lunar/+packages

[Tests]

autopkgtest output:

============================================================================
Testsuite summary for swtpm 0.7.3
============================================================================
# TOTAL: 68
# PASS: 57
# SKIP: 11
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0
============================================================================
make[3]: Leaving directory '/tmp/autopkgtest.cw6xcl/build.KoO/src/tests'
make[2]: Leaving directory '/tmp/autopkgtest.cw6xcl/build.KoO/src/tests'
make[1]: Leaving directory '/tmp/autopkgtest.cw6xcl/build.KoO/src/tests'
make[1]: Entering directory '/tmp/autopkgtest.cw6xcl/build.KoO/src'
make[1]: Leaving directory '/tmp/autopkgtest.cw6xcl/build.KoO/src'
autopkgtest [10:05:55]: test run-tests: -----------------------]
run-tests PASS
autopkgtest [10:05:56]: test run-tests: - - - - - - - - - - results - - - - - - - - - -
autopkgtest [10:05:57]: @@@@@@@@@@@@@@@@@@@@ summary
run-tests PASS

[Original Description]

[Impact]

Trying to create a VM with secure boot enabled in lunar always returns the following error:

  ERROR unsupported configuration: TPM version '2.0' is not supported

This is quite critical, because it makes impossible to test secure boot inside VMs, using lunar as host.

[Test case]

$ virt-install --name lunar_secure --arch x86_64 --feature smm=on --boot loader=/usr/share/OVMF/OVMF_CODE_4M.ms.fd,loader_ro=yes,loader_type=pflash --import --disk path=lunar_secure.img --disk path=lunar_secure-seed.img --memory 2048 --vcpus 2 --osinfo ubuntu22.10 --graphics none --console pty,target_type=serial --network network:default

Starting install...
ERROR unsupported configuration: TPM version '2.0' is not supported
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start lunar_secure
otherwise, please restart your installation.

ProblemType: Bug
DistroRelease: Ubuntu 23.04
Package: libvirt-daemon 9.0.0-2ubuntu1
ProcVersionSignature: Ubuntu 6.2.0-17.17-generic 6.2.6
Uname: Linux 6.2.0-17-generic x86_64
ApportVersion: 2.26.0-0ubuntu2
Architecture: amd64
CasperMD5CheckResult: pass
Date: Fri Mar 17 07:31:37 2023
InstallationDate: Installed on 2022-07-25 (234 days ago)
InstallationMedia: Ubuntu 22.10 "Kinetic Kudu" - Alpha amd64 (20220724)
SourcePackage: libvirt
UpgradeStatus: Upgraded to lunar on 2023-02-11 (33 days ago)
modified.conffile..etc.libvirt.qemu.conf: [inaccessible: [Errno 13] Permission denied: '/etc/libvirt/qemu.conf']

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Ehrhardt  (paelzer)]

The section created by jammy and lunar in regard to the tpm is the same in both cases:

    <tpm model="tpm-crb">
      <backend type="emulator"/>
    </tpm>

With that known here a minimal testcase:

# cat minimal.xml
<domain type="kvm">
  <name>minimal</name>
  <memory>2097152</memory>
  <features>
    <acpi/>
   </features>
  <os>
    <type arch="x86_64" machine="q35">hvm</type>
  </os>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <tpm model="tpm-crb">
      <backend type="emulator"/>
    </tpm>
  </devices>
</domain>

# virsh define minimal.xml
error: Failed to define domain from minimal.xml
error: unsupported configuration: TPM version '2.0' is not supported

So we know this is very early on (e.g. before actually spawning the guest).
Next: Need to look into the definition step in detail why it is failing.

[Christian Ehrhardt  (paelzer)]

This is as early as when doing the the domain capability checks.

Jammy:
    <tpm supported='yes'>
      <enum name='model'>
        <value>tpm-tis</value>
        <value>tpm-crb</value>
      </enum>
      <enum name='backendModel'>
        <value>passthrough</value>
        <value>emulator</value>
      </enum>
      <enum name='backendVersion'>
        <value>1.2</value>
        <value>2.0</value>
      </enum>
    </tpm>

Lunar (and also Kinetic):
    <tpm supported='yes'>
      <enum name='model'>
        <value>tpm-tis</value>
        <value>tpm-crb</value>
      </enum>
      <enum name='backendModel'>
        <value>passthrough</value>
        <value>emulator</value>
        <value>external</value>
      </enum>
    </tpm>

We see "external" is added (we know that from libvirt changelog), but at the same time the versions vanished in Lunar.

We need to check changes in that regard and what is executed to probe those.

[Christian Ehrhardt  (paelzer)]

Interesting, jammy in a container also lacks the versions but works fine:

    <tpm supported='yes'>
      <enum name='model'>
        <value>tpm-tis</value>
        <value>tpm-crb</value>
      </enum>
      <enum name='backendModel'>
        <value>passthrough</value>
        <value>emulator</value>
      </enum>
    </tpm>

But the define of that minimal guest XML works.
That also explains why I thought I've seen both outputs on jammy, one in the container one on my laptop.
So the disappearing of the versions from domcapabilities might have been a red herring.

I've now settled on a test env that uses Kinetic and Lunar (closest in versions) both in containers (same environment) and just using the definition of that minimal xml (simpler case for debugging).

[Christian Ehrhardt  (paelzer)]

This will get some debug info on the initialization of capabilities and the definition of the domina:

$ rm /var/cache/libvirt/qemu/capabilities/*; systemctl restart libvirtd; echo "" > /var/log/libvirt/libvirtd.log; virt-admin daemon-log-outputs "1:file:/var/log/libvirt/libvirtd.log"; virt-admin daemon-log-filters "1:qemu 1:libvirt 4:object 4:json 4:event 1:util"; virsh domcapabilities >/dev/null; virsh define minimal.xml; virt-admin daemon-log-filters ""; virt-admin daemon-log-outputs ""; virsh undefine minimal

In Kinetic that just shows a single probe command directly from libvirt and later the definition using tpm

root@k:~# grep -n -i tpm /var/log/libvirt/libvirtd.log
86:2023-03-22 09:06:49.172+0000: 9645: debug : virCommandRunAsync:2579 : About to run /usr/bin/swtpm_setup --print-capabilities
88:2023-03-22 09:06:49.179+0000: 9645: debug : virCommandRun:2425 : Result exit status 0, stdout: '{ "type": "swtpm_setup", "features": [ "cmdarg-keyfile-fd", "cmdarg-pwdfile-fd", "tpm12-not-need-root", "tpm2-rsa-keysize-2048", "tpm2-rsa-keysize-3072" ] }
301: <tpm supported='yes'>
303: <value>tpm-tis</value>
304: <value>tpm-crb</value>
310: </tpm>
415: <tpm model="tpm-crb">
417: </tpm>

In Lunar OTOH there is a lot more in the early initialization, the followed by the same to finally be sorted out in qemuValidateDomainDeviceDefTPM

root@l:~# grep -n -i tpm /var/log/libvirt/libvirtd.log

...
1153:2023-03-22 09:05:06.224+0000: 31712: debug : virCommandRunAsync:2607 : About to run /usr/bin/swtpm_setup --print-capabilities
1155:2023-03-22 09:05:06.230+0000: 31712: debug : virCommandRun:2453 : Result exit status 0, stdout: '{ "type": "swtpm_setup", "features": [ "cmdarg-keyfile-fd", "cmdarg-pwdfile-fd", "tpm12-not-need-root", "tpm2-rsa-keysize-2048", "tpm2-rsa-keysize-3072" ] }
1366: <tpm supported='yes'>
1368: <value>tpm-tis</value>
1369: <value>tpm-crb</value>
1376: </tpm>
1513: <tpm model="tpm-crb">
1515: </tpm>
1550:2023-03-22 09:05:06.298+0000: 31712: error : qemuValidateDomainDeviceDefTPM:4616 : unsupported configuration: TPM version '2.0' is not supported

[Christian Ehrhardt  (paelzer)]

The libvirt code handling these is
1. initialization
  qemuMonitorGetTPMModels
  qemuMonitorGetTPMTypes
2. later blocking the creation
  qemuValidateDomainDeviceDefTPM

The change that added the check of the tpm version is in 8.10 and later, so it is new in Lunar:
=> https://gitlab.com/libvirt/libvirt/-/commit/b91dbf4851d57f536e4de4f3e8496e4e119e22bb

But the check would be ok to be done, what is wrong IMHO is the detection that does think we have no versions at all. That initialization is the long list of log that we see and it uses a query to qemu using the verbs "query-tpm-models" "query-tpm-types". That probing is older, just so far no component cared much what it returned.

[Christian Ehrhardt  (paelzer)]

In qmp-shell [1] kinetic and lunar both deliver the same.

$ qemu-system-x86_64 -S -qmp unix:./qmp-sock,server --monitor stdio -display none
$ ./qmp-shell ~/qmp-sock
(QEMU) query-tpm
{"return": []}
(QEMU) query-tpm-models
{"return": ["tpm-crb", "tpm-tis"]}
(QEMU) query-tpm-types
{"return": ["passthrough", "emulator"]}

That matches what we see in virsh domcapabilities.
The above is even the same on my laptop where the domcapabilities report versions.

At least everything else is stable and didn't change, but we actually want to know where tpmCaps.backendVersion is populated.
That is done in libvirt in virQEMUCapsFillDomainDeviceTPMCaps [2]
As usual after many indirections this is handled in

virTPMSwtpmSetupCapsGet(VIR_TPM_SWTPM_SETUP_FEATURE_TPM_1_2)
 -> virTPMBinaryGetCaps(VIR_TPM_BINARY_SWTPM_SETUP, cap
    -> virTPMGetCaps (fills a structure which is then checked)

This leads to the call I mentioned way above:

root@k:~# /usr/bin/swtpm_setup --print-capabilities
{ "type": "swtpm_setup", "features": [ "cmdarg-keyfile-fd", "cmdarg-pwdfile-fd", "tpm12-not-need-root", "tpm2-rsa-keysize-2048", "tpm2-rsa-keysize-3072" ] }

Here the same from my system which reports versions:

$ /usr/bin/swtpm_setup --print-capabilities
{ "type": "swtpm_setup", "features": [ "tpm-1.2", "tpm-2.0", "cmdarg-keyfile-fd", "cmdarg-pwdfile-fd", "tpm12-not-need-root", "cmdarg-write-ek-cert-files", "cmdarg-create-config-files", "cmdarg-reconfigure-pcr-banks", "tpm2-rsa-keysize-2048", "tpm2-rsa-keysize-3072" ], "version": "0.7.3" }

[1]: https://wiki.qemu.org/Documentation/QMP#qmp-shell_script
[2]: https://gitlab.com/libvirt/libvirt/-/blob/master/src/qemu/qemu_capabilities.c#L6544

[Christian Ehrhardt  (paelzer)]

I found the following:
- jammy-lunar have 0.6.3-0ubuntu3-0.6.3-0ubuntu5 => failing
- I have once experimented with that and use 0.7.3-1 from [1] => working

I think everyone has forgotten about updating this so far and this is now exposing an issue with older swtpm versions. There is 8.x and 9.x upstream, but for Lunar maybe just getting the 0.7.1-1+b1 from Debian would be most stable.

This seems to work fine:
root@d10-sid:~# /usr/bin/swtpm_setup --print-capabilities
{ "type": "swtpm_setup", "features": [ "tpm-1.2", "tpm-2.0", "cmdarg-keyfile-fd", "cmdarg-pwdfile-fd", "tpm12-not-need-root", "cmdarg-write-ek-cert-files", "cmdarg-create-config-files", "cmdarg-reconfigure-pcr-banks", "tpm2-rsa-keysize-2048", "tpm2-rsa-keysize-3072" ], "version": "0.7.1" }

Tasks from here:
- [lena] merge swtpm 0.7.1 from Debian
- [me] then make the swtpm dependency in libvirt versioned
- [bryce] ensure swtpm is in our list of packages to be merged each cycle

[1]: https://launchpad.net/~stefanberger/+archive/ubuntu/swtpm

[Andrea Righi (arighi)]

@paelzer thank you so much for the detailed analysis!

FWIW, I've tried to install swtpm-0.7.1 from Debian sid and I confirm that everything is working now also in my specific test case.

[Lena Voytek (lvoytek)]

Hello,

I created a ppa with an update to 0.7.3 in lunar that should fix the issue, located here: https://launchpad.net/~lvoytek/+archive/ubuntu/swtpm-update-lunar

If you would like to test it you can run:

sudo add-apt-repository ppa:lvoytek/swtpm-update-lunar
sudo apt update
sudo apt upgrade

On my end swtpm now shows the proper capabilities:

# swtpm_setup --print-capabilities
{ "type": "swtpm_setup", "features": [ "tpm-1.2", "tpm-2.0", "cmdarg-keyfile-fd", "cmdarg-pwdfile-fd", "tpm12-not-need-root", "cmdarg-write-ek-cert-files", "cmdarg-create-config-files", "cmdarg-reconfigure-pcr-banks", "tpm2-rsa-keysize-2048", "tpm2-rsa-keysize-3072" ], "version": "0.7.3" }

[Paride Legovini (paride)]

Hello Lena, I installed swtpm from your ppa verified that `swtpm_setup --print-capabilities` lists tpm-2.0. I also tried an ISO install of Ubuntu via:

  virt-install [...] --features smm=on --boot uefi,loader_secure=yes

and verified that it works and that `mokutil --sb-state` reports SecureBoot enabled. Everything looks good. Previously virt-install failed with the

  TPM version '2.0' is not supported

error.

[Lena Voytek (lvoytek)]

Thanks for testing that, I'll get a merge request up then

[Brian Murray (brian-murray)]

On behalf of the Ubuntu Release team I approve this FFe.

[Launchpad Janitor (janitor)]

This bug was fixed in the package swtpm - 0.7.3-0ubuntu1

---------------
swtpm (0.7.3-0ubuntu1) lunar; urgency=medium

  * New upstream release 0.7.3:
    - Bug fixes include:
      + Fix secure boot failure - TPM 2.0 not supported (LP: #2012028)
  * Add new debian/ files from upstream
    - d/clean: Clean man and gch files from source tree during build
    - d/not-installed: Do not install .la lib files with package
    - d/swtpm-libs.install: Install swtpm .so files with swtpm-libs package
  * d/rules: Add dh_clean and dh_makeshlibs overrides from upstream
  * d/swtpm-tools.install: Update installation of swtpm-tools files for 0.7
  * d/control: Remove unneeded dependencies for 0.7
  * Remove d/p/0001-Install-swtpm-localca-to-the-correct-path.patch as it is
    no longer needed to change swtpm-localca's path
  * d/p/no-autoconf-in-debian.patch: Refresh to clean fuzz
  * d/p/openssl-not-certtool.patch: Update and refresh to apply with 0.7

 -- Lena Voytek <email address hidden> Wed, 22 Mar 2023 14:03:19 -0700