Enabling vTPM in Victoria on CentOS 8 causes SELinux denials

[Kevin Jones]

Description
===========
I recently worked through enabling vTPM on a TripleO based Victoria deployment on CentOS 8. After modifying the openstack-nova-compute and openstack-nova-libvirt containers to get swtpm, trousers, and swtpm-tools installed, launching an instance threw SELinux denials on the compute host.

Steps to reproduce
==================
Enabled vTPM for Nova on my TripleO deployment. Steps documented at:
https://kdjlab.com/enabling-virtual-tpm-in-openstack-victoria/

Expected result
===============
Instance launched using a flavor or image with the vTPM metadata properties will complete successfully and the tpm device will be available in the instance.

Actual result
=============
Instance errors out on spawning with SELinux denials on the compute host.

Environment
===========
1. OpenStack Victoria (RDO TripleO)
2. Ceph Nautilus
3. Neutron OVN

Logs & Configs
==============
sealert output:

found 4 alerts in /var/log/audit/audit.log
--------------------------------------------------------------------------------

SELinux is preventing swtpm from write access on the directory swtpm.

***** Plugin qemu_file_image (98.8 confidence) suggests *******************

If swtpm is a virtualization target
Then you need to change the label on swtpm'
Do
# semanage fcontext -a -t virt_image_t 'swtpm'
# restorecon -v 'swtpm'

***** Plugin catchall (2.13 confidence) suggests **************************

If you believe that swtpm should be allowed write access on the swtpm directory by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'swtpm' --raw | audit2allow -M my-swtpm
# semodule -X 300 -i my-swtpm.pp

Additional Information:
Source Context system_u:system_r:svirt_t:s0:c38,c260
Target Context system_u:object_r:container_file_t:s0
Target Objects swtpm [ dir ]
Source swtpm
Source Path swtpm
Port <Unknown>
Host <Unknown>
Source RPM Packages
Target RPM Packages
Policy RPM selinux-policy-3.14.3-41.el8_2.6.noarch
Selinux Enabled True
Policy Type targeted
Enforcing Mode Permissive
Host Name overcloud-controller-1
Platform Linux overcloud-controller-1
                              4.18.0-193.19.1.el8_2.x86_64 #1 SMP Mon Sep 14
                              14:37:00 UTC 2020 x86_64 x86_64
Alert Count 1
First Seen 2020-11-01 06:25:41 EST
Last Seen 2020-11-01 06:25:41 EST
Local ID b088198d-ac2b-45a0-ac8a-354bcc3c51ac

Raw Audit Messages
type=AVC msg=audit(1604229941.636:2391008): avc: denied { write } for pid=452433 comm="swtpm" name="swtpm" dev="tmpfs" ino=510589704 scontext=system_u:system_r:svirt_t:s0:c38,c260 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=1

type=AVC msg=audit(1604229941.636:2391008): avc: denied { add_name } for pid=452433 comm="swtpm" name="1-instance-0000005a-swtpm.sock" scontext=system_u:system_r:svirt_t:s0:c38,c260 tcontext=system_u:object_r:container_file_t:s0 tclass=dir permissive=1

type=AVC msg=audit(1604229941.636:2391008): avc: denied { create } for pid=452433 comm="swtpm" name="1-instance-0000005a-swtpm.sock" scontext=system_u:system_r:svirt_t:s0:c38,c260 tcontext=system_u:object_r:container_file_t:s0 tclass=sock_file permissive=1

Hash: swtpm,svirt_t,container_file_t,dir,write

--------------------------------------------------------------------------------

SELinux is preventing swtpm from setattr access on the sock_file 1-instance-0000005a-swtpm.sock.

***** Plugin catchall (100. confidence) suggests **************************

If you believe that swtpm should be allowed setattr access on the 1-instance-0000005a-swtpm.sock sock_file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'swtpm' --raw | audit2allow -M my-swtpm
# semodule -X 300 -i my-swtpm.pp

Additional Information:
Source Context system_u:system_r:svirt_t:s0:c38,c260
Target Context system_u:object_r:container_file_t:s0
Target Objects 1-instance-0000005a-swtpm.sock [ sock_file ]
Source swtpm
Source Path swtpm
Port <Unknown>
Host <Unknown>
Source RPM Packages
Target RPM Packages
Policy RPM selinux-policy-3.14.3-41.el8_2.6.noarch
Selinux Enabled True
Policy Type targeted
Enforcing Mode Permissive
Host Name overcloud-controller-1
Platform Linux overcloud-controller-1
                              4.18.0-193.19.1.el8_2.x86_64 #1 SMP Mon Sep 14
                              14:37:00 UTC 2020 x86_64 x86_64
Alert Count 1
First Seen 2020-11-01 06:25:41 EST
Last Seen 2020-11-01 06:25:41 EST
Local ID 6ea3df16-9bf2-463e-bc0c-bd1f892eae4e

Raw Audit Messages
type=AVC msg=audit(1604229941.636:2391009): avc: denied { setattr } for pid=452433 comm="swtpm" name="1-instance-0000005a-swtpm.sock" dev="tmpfs" ino=510590763 scontext=system_u:system_r:svirt_t:s0:c38,c260 tcontext=system_u:object_r:container_file_t:s0 tclass=sock_file permissive=1

Hash: swtpm,svirt_t,container_file_t,sock_file,setattr

--------------------------------------------------------------------------------

SELinux is preventing swtpm from append access on the file instance-0000005a-swtpm.log.

***** Plugin qemu_file_image (98.8 confidence) suggests *******************

If instance-0000005a-swtpm.log is a virtualization target
Then you need to change the label on instance-0000005a-swtpm.log'
Do
# semanage fcontext -a -t virt_image_t 'instance-0000005a-swtpm.log'
# restorecon -v 'instance-0000005a-swtpm.log'

***** Plugin catchall (2.13 confidence) suggests **************************

If you believe that swtpm should be allowed append access on the instance-0000005a-swtpm.log file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'swtpm' --raw | audit2allow -M my-swtpm
# semodule -X 300 -i my-swtpm.pp

Additional Information:
Source Context system_u:system_r:svirt_t:s0:c38,c260
Target Context system_u:object_r:container_ro_file_t:s0
Target Objects instance-0000005a-swtpm.log [ file ]
Source swtpm
Source Path swtpm
Port <Unknown>
Host <Unknown>
Source RPM Packages
Target RPM Packages
Policy RPM selinux-policy-3.14.3-41.el8_2.6.noarch
Selinux Enabled True
Policy Type targeted
Enforcing Mode Permissive
Host Name overcloud-controller-1
Platform Linux overcloud-controller-1
                              4.18.0-193.19.1.el8_2.x86_64 #1 SMP Mon Sep 14
                              14:37:00 UTC 2020 x86_64 x86_64
Alert Count 1
First Seen 2020-11-01 06:25:41 EST
Last Seen 2020-11-01 06:25:41 EST
Local ID 0bbe7b10-0e15-4b86-a019-5bd9ec884b3b

Raw Audit Messages
type=AVC msg=audit(1604229941.636:2391010): avc: denied { append } for pid=452433 comm="swtpm" name="instance-0000005a-swtpm.log" dev="overlay" ino=510589703 scontext=system_u:system_r:svirt_t:s0:c38,c260 tcontext=system_u:object_r:container_ro_file_t:s0 tclass=file permissive=1

Hash: swtpm,svirt_t,container_ro_file_t,file,append

--------------------------------------------------------------------------------

SELinux is preventing swtpm from create access on the file 1-instance-0000005a-swtpm.pid.

***** Plugin catchall (100. confidence) suggests **************************

If you believe that swtpm should be allowed create access on the 1-instance-0000005a-swtpm.pid file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'swtpm' --raw | audit2allow -M my-swtpm
# semodule -X 300 -i my-swtpm.pp

Additional Information:
Source Context system_u:system_r:svirt_t:s0:c38,c260
Target Context system_u:object_r:container_file_t:s0
Target Objects 1-instance-0000005a-swtpm.pid [ file ]
Source swtpm
Source Path swtpm
Port <Unknown>
Host <Unknown>
Source RPM Packages
Target RPM Packages
Policy RPM selinux-policy-3.14.3-41.el8_2.6.noarch
Selinux Enabled True
Policy Type targeted
Enforcing Mode Permissive
Host Name overcloud-controller-1
Platform Linux overcloud-controller-1
                              4.18.0-193.19.1.el8_2.x86_64 #1 SMP Mon Sep 14
                              14:37:00 UTC 2020 x86_64 x86_64
Alert Count 1
First Seen 2020-11-01 06:25:41 EST
Last Seen 2020-11-01 06:25:41 EST
Local ID e43e45e0-3b04-4962-8fd5-21993166df9e

Raw Audit Messages
type=AVC msg=audit(1604229941.639:2391011): avc: denied { create } for pid=452434 comm="swtpm" name="1-instance-0000005a-swtpm.pid" scontext=system_u:system_r:svirt_t:s0:c38,c260 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=1

type=AVC msg=audit(1604229941.639:2391011): avc: denied { open } for pid=452434 comm="swtpm" path="/run/libvirt/qemu/swtpm/1-instance-0000005a-swtpm.pid" dev="tmpfs" ino=510592344 scontext=system_u:system_r:svirt_t:s0:c38,c260 tcontext=system_u:object_r:container_file_t:s0 tclass=file permissive=1

Hash: swtpm,svirt_t,container_file_t,file,create

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/puppet-tripleo/+/811103

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/puppet-tripleo/+/811103
Committed: https://opendev.org/openstack/puppet-tripleo/commit/98af2c581bce9228e2e35e29f2511dce9de34457
Submitter: "Zuul (22348)"
Branch: master

commit 98af2c581bce9228e2e35e29f2511dce9de34457
Author: Cédric Jeanneret <email address hidden>
Date: Mon Sep 27 11:47:51 2021 +0200

    Add a new bind-mount for vTPM logs

    swtpm has its own log directory, and it's hardcoded in libvirt[1].
    For the records, this location is currently set to:
    /var/log/swtpm/libvirt/qemu

    In order to keep some kind of consistency with the current log structure
    in /var/log/containers/libvirt, we will keep the "qemu" subdirectory -
    it already exists for other qemu-related services, therefore it makes
    sense to keep that subdirectory.

    This is possible since the swtpm log filename is composed of the
    instance ID suffixed by "-swtpm.log", leading to a clear view.

    [1] https://gitlab.com/libvirt/libvirt/-/commit/f9cd29a2e44d61da0fc94f245d27206ea66e1161

    Related: rhbz#2007314
    Related-Bug: #1902468
    Change-Id: Ibc80621a622a4eb4ef31520439ded8a08ce50c96

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/puppet-tripleo/+/812430

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/puppet-tripleo/+/812430
Committed: https://opendev.org/openstack/puppet-tripleo/commit/a4750e1660a22de0605d9fc46546aca78fa49824
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit a4750e1660a22de0605d9fc46546aca78fa49824
Author: Cédric Jeanneret <email address hidden>
Date: Mon Sep 27 11:47:51 2021 +0200

    Add a new bind-mount for vTPM logs

    swtpm has its own log directory, and it's hardcoded in libvirt[1].
    For the records, this location is currently set to:
    /var/log/swtpm/libvirt/qemu

    In order to keep some kind of consistency with the current log structure
    in /var/log/containers/libvirt, we will keep the "qemu" subdirectory -
    it already exists for other qemu-related services, therefore it makes
    sense to keep that subdirectory.

    This is possible since the swtpm log filename is composed of the
    instance ID suffixed by "-swtpm.log", leading to a clear view.

    [1] https://gitlab.com/libvirt/libvirt/-/commit/f9cd29a2e44d61da0fc94f245d27206ea66e1161

    Related: rhbz#2007314
    Related-Bug: #1902468
    Change-Id: Ibc80621a622a4eb4ef31520439ded8a08ce50c96
    (cherry picked from commit 98af2c581bce9228e2e35e29f2511dce9de34457)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/813431

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/813432

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/813431
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/f664302c3deb539c0f59fa5b1eff48449acf6b85
Submitter: "Zuul (22348)"
Branch: master

commit f664302c3deb539c0f59fa5b1eff48449acf6b85
Author: Cédric Jeanneret <email address hidden>
Date: Mon Oct 11 15:41:35 2021 +0200

    Enable new SELinux boolean for vTPM support

    In order to get a working vTPM support in containers, we need to enable
    a new SELinux boolean provided by openstack-selinux[1].

    This patch affects only the deprecated
    nova-libvirt-container-puppet.yaml template in order to do a clean
    backport to stable/Wallaby and stable/Victoria.

    [1] https://github.com/redhat-openstack/openstack-selinux/pull/80

    Change-Id: I1d2368135f7b0a83dec2192c242c081e2f5127c1
    Closes-Bug: #1902468
    Resolves: rhbz#2007314

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/814359

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/814360

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/814359
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/2133864d900ce034d8e300f9e397e3da81920201
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 2133864d900ce034d8e300f9e397e3da81920201
Author: Cédric Jeanneret <email address hidden>
Date: Mon Oct 11 15:41:35 2021 +0200

    Enable new SELinux boolean for vTPM support

    In order to get a working vTPM support in containers, we need to enable
    a new SELinux boolean provided by openstack-selinux[1].

    This patch affects only the deprecated
    nova-libvirt-container-puppet.yaml template in order to do a clean
    backport to stable/Wallaby and stable/Victoria.

    [1] https://github.com/redhat-openstack/openstack-selinux/pull/80

    Change-Id: I1d2368135f7b0a83dec2192c242c081e2f5127c1
    Closes-Bug: #1902468
    Resolves: rhbz#2007314
    (cherry picked from commit f664302c3deb539c0f59fa5b1eff48449acf6b85)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/814360
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/47e9740676e0d3e8ad3935f0a369639c5d975a23
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 47e9740676e0d3e8ad3935f0a369639c5d975a23
Author: Cédric Jeanneret <email address hidden>
Date: Mon Oct 11 15:41:35 2021 +0200

    Enable new SELinux boolean for vTPM support

    In order to get a working vTPM support in containers, we need to enable
    a new SELinux boolean provided by openstack-selinux[1].

    This patch affects only the deprecated
    nova-libvirt-container-puppet.yaml template in order to do a clean
    backport to stable/Wallaby and stable/Victoria.

    [1] https://github.com/redhat-openstack/openstack-selinux/pull/80

    Change-Id: I1d2368135f7b0a83dec2192c242c081e2f5127c1
    Closes-Bug: #1902468
    Resolves: rhbz#2007314
    (cherry picked from commit f664302c3deb539c0f59fa5b1eff48449acf6b85)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/813432
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/f834c26d59e87b958928d287e3623b374e0fd94d
Submitter: "Zuul (22348)"
Branch: master

commit f834c26d59e87b958928d287e3623b374e0fd94d
Author: Cédric Jeanneret <email address hidden>
Date: Mon Oct 11 15:44:35 2021 +0200

    Enable new SELinux boolean for vTPM support

    In order to get a working vTPM support in containers, we need to enable
    a new SELinux boolean provided by openstack-selinux[1]

    This patch affects only nova-modular-libvirt-container-puppet.yaml for
    master and future releases.

    [1] https://github.com/redhat-openstack/openstack-selinux/pull/80

    Change-Id: I0db66dd124e3e02fd2fe3c729dc0fb3eeafec7a0
    Closes-Bug: #1902468
    Resolves: rhbz#2007314

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 16.0.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 13.6.0 release.