nova_compute continuous restart if enabling NovaEnableVTPM

Bug #1901530 reported by Kevin Jones
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
High
Bogdan Dobrelya

Bug Description

Description
===========
If you set NovaEnableVTPM true, nova_compute container continuously restarts because swtpm and swtpm_setup are not available on path.

nova_libvirt does have swtpm and swtpm_setup on path and does not crash.

Steps to reproduce
==================
* Deploy overcloud with NovaEnableVTPM set to true
* Observe deployment failure
* Observe that nova_compute container continuously restarts

Expected result
===============
vTPM would be configured correctly and nova_compute would be stable

Actual result
=============
vTPM is configured but swtpm and swtpm_setup are not available on path in nova_compute container

Environment
===========
1. RDO Victoria

2. TripleO deployed Ceph Nautilus

3. Neutron OVN

Additional Info
==============

[root@overcloud-compute-0 ~]# podman exec -it nova_libvirt /bin/bash
[root@overcloud-compute-0 /]# swtpm --version
TPM emulator version 0.2.0, Copyright (c) 2014 IBM Corp.

[root@overcloud-compute-0 ~]# podman exec -it nova_compute /bin/bash
bash-4.4$ swtpm
bash: swtpm: command not found
bash-4.4$ Error: non zero exit code: 137: OCI runtime error

020-10-26 07:26:49.378 8 ERROR oslo_service.service [-] Error starting thread.: nova.exception.InvalidConfiguration: vTPM support is configured but the 'swtpm' and 'swtpm_setup' binaries could not be found on PATH.
2020-10-26 07:26:49.378 8 ERROR oslo_service.service Traceback (most recent call last):
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/oslo_service/service.py", line 807, in run_service
2020-10-26 07:26:49.378 8 ERROR oslo_service.service service.start()
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/service.py", line 159, in start
2020-10-26 07:26:49.378 8 ERROR oslo_service.service self.manager.init_host()
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 1414, in init_host
2020-10-26 07:26:49.378 8 ERROR oslo_service.service self.driver.init_host(host=self.host)
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 691, in init_host
2020-10-26 07:26:49.378 8 ERROR oslo_service.service self._check_vtpm_support()
2020-10-26 07:26:49.378 8 ERROR oslo_service.service File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 772, in _check_vtpm_support
2020-10-26 07:26:49.378 8 ERROR oslo_service.service raise exception.InvalidConfiguration(msg)
2020-10-26 07:26:49.378 8 ERROR oslo_service.service nova.exception.InvalidConfiguration: vTPM support is configured but the 'swtpm' and 'swtpm_setup' binaries could not be found on PATH.
2020-10-26 07:26:49.378 8 ERROR oslo_service.service

Revision history for this message
Kevin Jones (kevindjones) wrote :

Deployment failure error:

2020-10-26 03:11:39.242490 | ea78fe08-222d-1182-0241-000000000203 | TASK | Discovering nova hosts
2020-10-26 03:11:40.856732 | ea78fe08-222d-1182-0241-000000000203 | FATAL | Discovering nova hosts | undercloud -> 10.100.4.83 | error={"changed": false, "cmd": ["podman", "exec", "nova_compute", "nova-manage", "cell_v2", "discover_hosts
", "--by-service"], "delta": "0:00:01.324516", "end": "2020-10-25 23:11:40.812879", "msg": "non-zero return code", "rc": 137, "start": "2020-10-25 23:11:39.488363", "stderr": "Error: non zero exit code: 137: OCI runtime error", "stderr_lines"
: ["Error: non zero exit code: 137: OCI runtime error"], "stdout": "", "stdout_lines": []}

Revision history for this message
Kevin Jones (kevindjones) wrote :

I finally got through enabling vTPM in OpenStack Victoria. It involves modifying openstack-nova-compute and openstack-nova-libvirt, as well as disabling SELinux on the computes.

https://kdjlab.com/enabling-virtual-tpm-in-openstack-victoria/

Revision history for this message
Kevin Jones (kevindjones) wrote :

https://hub.docker.com/repository/docker/kjw3/openstack-nova-compute

Dockerfile for openstack-nova-compute:

[cloud-user@rhel8-image-builder ~]$ cat openstack-nova-compute/Dockerfile
# This will be replaced in the file Dockerfile.modified
FROM openstack-nova-compute

# switch to root to install packages
USER root

# install packages
RUN curl "https://content.kdjlab.com/public/advanced-virtualization.repo" -o "/etc/yum.repos.d/advanced-virtualization.repo"
RUN dnf install -y swtpm trousers swtpm-tools && dnf clean all && rm -rf /var/cache/dnf

#Add user and group for swtpm
USER tss

# switch the container back to the default user
USER nova

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (master)
Changed in tripleo:
status: New → In Progress
Changed in tripleo:
status: In Progress → Triaged
importance: Undecided → High
tags: added: victoria-backport-potential wallaby-backport-potential
Changed in tripleo:
status: Triaged → In Progress
Changed in tripleo:
assignee: nobody → Bogdan Dobrelya (bogdando)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/811609
Committed: https://opendev.org/openstack/tripleo-common/commit/8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a
Submitter: "Zuul (22348)"
Branch: master

commit 8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Sep 29 10:10:12 2021 +0200

    Align swtpm packages for nova-libvirt/compute

    Both containers require swtpm/tools and trousers packages installed.
    Also add tss user/group.

    Closes-bug: #1901530

    Change-Id: Id5b83ad4e5a4e223f43adac95d0079d5cdcc2749
    Signed-off-by: Bogdan Dobrelya <email address hidden>

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-common/+/812107

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-common/+/812019

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/812107
Committed: https://opendev.org/openstack/tripleo-common/commit/323cfa6db309bd33be746968b77ecc1c92433299
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 323cfa6db309bd33be746968b77ecc1c92433299
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Sep 29 10:10:12 2021 +0200

    Align swtpm packages for nova-libvirt/compute

    Both containers require swtpm/tools and trousers packages installed.
    Also add tss user/group.

    Closes-bug: #1901530

    Change-Id: Id5b83ad4e5a4e223f43adac95d0079d5cdcc2749
    Signed-off-by: Bogdan Dobrelya <email address hidden>
    (cherry picked from commit 8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-common/+/812019
Committed: https://opendev.org/openstack/tripleo-common/commit/cf231fb265a795c03218f3c889dea892dd1d1d3a
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit cf231fb265a795c03218f3c889dea892dd1d1d3a
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Sep 29 10:10:12 2021 +0200

    Align swtpm packages for nova-libvirt/compute

    Both containers require swtpm/tools and trousers packages installed.
    Also add tss user/group.

    Closes-bug: #1901530

    Change-Id: Id5b83ad4e5a4e223f43adac95d0079d5cdcc2749
    Signed-off-by: Bogdan Dobrelya <email address hidden>
    (cherry picked from commit 8cbdb71d7eac2a8dc13c65c2d42fa24fac31de4a)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 16.2.0

This issue was fixed in the openstack/tripleo-common 16.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 15.4.0

This issue was fixed in the openstack/tripleo-common 15.4.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers