nova_compute continuous restart if enabling NovaEnableVTPM

Bug #1901530 reported by Kevin Jones
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Bogdan Dobrelya

Bug Description

Description
===========
If you set NovaEnableVTPM true, nova_compute container continuously restarts because swtpm and swtpm_setup are not available on path.

nova_libvirt does have swtpm and swtpm_setup on path and does not crash.

Steps to reproduce
==================
* Deploy overcloud with NovaEnableVTPM set to true
* Observe deployment failure
* Observe that nova_compute container continuously restarts

Expected result
===============
vTPM would be configured correctly and nova_compute would be stable

Actual result
=============
vTPM is configured but swtpm and swtpm_setup are not available on path in nova_compute container

Environment
===========
1. RDO Victoria

2. TripleO deployed Ceph Nautilus

3. Neutron OVN

Additional Info
==============

[root@overcloud-compute-0 ~]# podman exec -it nova_libvirt /bin/bash
[root@overcloud-compute-0 /]# swtpm --version
TPM emulator version 0.2.0, Copyright (c) 2014 IBM Corp.

[root@overcloud-compute-0 ~]# podman exec -it nova_compute /bin/bash
bash-4.4$ swtpm
bash: swtpm: command not found
bash-4.4$ Error: non zero exit code: 137: OCI runtime error

020-10-26 07:26:49.378 8 ERROR oslo_service.service [-] Error starting thread.: nova.exception.InvalidConfiguration: vTPM support is configured but the 'swtpm' and 'swtpm_setup' binaries could not be found on PATH.
2020-10-26 07:26:49.378 8 ERROR oslo_service.service Traceback (most recent call last):
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 807, in run_service
2020-10-26 07:26:49.378 8 ERROR oslo_service.service service.start()
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/service.py", line 159, in start
2020-10-26 07:26:49.378 8 ERROR oslo_service.service self.manager.init_host()
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 1414, in init_host
2020-10-26 07:26:49.378 8 ERROR oslo_service.service self.driver.init_host(host=self.host)
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 691, in init_host
2020-10-26 07:26:49.378 8 ERROR oslo_service.service self._check_vtpm_support()
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 772, in _check_vtpm_support
2020-10-26 07:26:49.378 8 ERROR oslo_service.service raise exception.InvalidConfiguration(msg)
2020-10-26 07:26:49.378 8 ERROR oslo_service.service nova.exception.InvalidConfiguration: vTPM support is configured but the 'swtpm' and 'swtpm_setup' binaries could not be found on PATH.
2020-10-26 07:26:49.378 8 ERROR oslo_service.service

Revision history for this message
Kevin Jones (kevindjones) wrote :

Deployment failure error:

2020-10-26 03:11:39.242490 | ea78fe08-222d-1182-0241-000000000203 | TASK | Discovering nova hosts
2020-10-26 03:11:40.856732 | ea78fe08-222d-1182-0241-000000000203 | FATAL | Discovering nova hosts | undercloud -> 10.100.4.83 | error={"changed": false, "cmd": ["podman", "exec", "nova_compute", "nova-manage", "cell_v2", "discover_hosts
", "--by-service"], "delta": "0:00:01.324516", "end": "2020-10-25 23:11:40.812879", "msg": "non-zero return code", "rc": 137, "start": "2020-10-25 23:11:39.488363", "stderr": "Error: non zero exit code: 137: OCI runtime error", "stderr_lines"
: ["Error: non zero exit code: 137: OCI runtime error"], "stdout": "", "stdout_lines": []}

Revision history for this message
Kevin Jones (kevindjones) wrote :

I finally got through enabling vTPM in OpenStack Victoria. It involves modifying openstack-nova-compute and openstack-nova-libvirt, as well as disabling SELinux on the computes.

https://kdjlab.com/enabling-virtual-tpm-in-openstack-victoria/

Revision history for this message
Kevin Jones (kevindjones) wrote :

https://hub.docker.com/repository/docker/kjw3/openstack-nova-compute

Dockerfile for openstack-nova-compute:

[cloud-user@rhel8-image-builder ~]$ cat openstack-nova-compute/Dockerfile
# This will be replaced in the file Dockerfile.modified
FROM openstack-nova-compute

# switch to root to install packages
USER root

# install packages
RUN curl "https://content.kdjlab.com/public/advanced-virtualization.repo" -o "/etc/yum.repos.d/advanced-virtualization.repo"
RUN dnf install -y swtpm trousers swtpm-tools && dnf clean all && rm -rf /var/cache/dnf

#Add user and group for swtpm
USER tss

# switch the container back to the default user
USER nova

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (master)
Changed in tripleo:
status: New → In Progress
Changed in tripleo:
status: In Progress → Triaged
importance: Undecided → High
tags: added: victoria-backport-potential wallaby-backport-potential
Changed in tripleo:
status: Triaged → In Progress
Changed in tripleo:
assignee: nobody → Bogdan Dobrelya (bogdando)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/811609
Committed: https://opendev.org/openstack/tripleo-common/commit/8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a
Submitter: "Zuul (22348)"
Branch: master

commit 8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Sep 29 10:10:12 2021 +0200

    Align swtpm packages for nova-libvirt/compute

    Both containers require swtpm/tools and trousers packages installed.
    Also add tss user/group.

    Closes-bug: #1901530

    Change-Id: Id5b83ad4e5a4e223f43adac95d0079d5cdcc2749
    Signed-off-by: Bogdan Dobrelya <email address hidden>

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-common/+/812107

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-common/+/812019

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/812107
Committed: https://opendev.org/openstack/tripleo-common/commit/323cfa6db309bd33be746968b77ecc1c92433299
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 323cfa6db309bd33be746968b77ecc1c92433299
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Sep 29 10:10:12 2021 +0200

    Align swtpm packages for nova-libvirt/compute

    Both containers require swtpm/tools and trousers packages installed.
    Also add tss user/group.

    Closes-bug: #1901530

    Change-Id: Id5b83ad4e5a4e223f43adac95d0079d5cdcc2749
    Signed-off-by: Bogdan Dobrelya <email address hidden>
    (cherry picked from commit 8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/812019
Committed: https://opendev.org/openstack/tripleo-common/commit/cf231fb265a795c03218f3c889dea892dd1d1d3a
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit cf231fb265a795c03218f3c889dea892dd1d1d3a
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Sep 29 10:10:12 2021 +0200

    Align swtpm packages for nova-libvirt/compute

    Both containers require swtpm/tools and trousers packages installed.
    Also add tss user/group.

    Closes-bug: #1901530

    Change-Id: Id5b83ad4e5a4e223f43adac95d0079d5cdcc2749
    Signed-off-by: Bogdan Dobrelya <email address hidden>
    (cherry picked from commit 8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 16.2.0

This issue was fixed in the openstack/tripleo-common 16.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 15.4.0

This issue was fixed in the openstack/tripleo-common 15.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 13.3.0

This issue was fixed in the openstack/tripleo-common 13.3.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.