MEM_ENCRYPTION_CONTEXT trait is missing from the compute RP even if AMD SEV is enabled on the compute node

Bug #1975686 reported by Balazs Gibizer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Undecided
Balazs Gibizer

Bug Description

Compute nodes with amd-sev enabled are reporting that support is available but MEM_ENCRYPTION_CONTEXT is not present in the placement traits for the compute nodes.

# Domain capabilites report support
[heat-admin@computeamdsev-1 log]$ sudo podman exec -it -u root nova_virtqemud virsh domcapabilities | grep -A 12 features
  <features>
    <gic supported='no'/>
    <vmcoreinfo supported='yes'/>
    <genid supported='yes'/>
    <backingStoreInput supported='yes'/>
    <backup supported='yes'/>
    <sev supported='yes'>
      <cbitpos>47</cbitpos>
      <reducedPhysBits>1</reducedPhysBits>
      <maxGuests>509</maxGuests>
      <maxESGuests>0</maxESGuests>
    </sev>
  </features>
</domainCapabilities>

# It is active as well in /sys/module/kvm_amd
[heat-admin@computeamdsev-1 log]$ cat /sys/module/kvm_amd/parameters/sev
Y
[heat-admin@computeamdsev-1 log]$

# I do not see any errors with sev during startup
[heat-admin@computeamdsev-1 log]$ sudo dmesg | grep -i sev
[ 0.000000] Command line: BOOT_IMAGE=(lvmid/nZkWaZ-f6bk-Bfto-h9OG-k1Sc-Y6RB-1Q3yZV/t77pr1-3H2Y-ml4l-MMJh-bp3H-zk2j-6z4W6w)/boot/vmlinuz-5.14.0-70.5.1.el9_0.x86_64 root=LABEL=img-rootfs ro console=ttyS0 console=ttyS0,115200n81 no_timer_check crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M hugepagesz=1GB hugepages=32 default_hugepagesz=1GB mem_encrypt=on kvm_amd.sev=1 console=tty0 console=ttyS0,115200 no_timer_check nofb nomodeset vga=normal console=tty0 console=ttyS0,115200 audit=1 nousb
[ 0.000000] Kernel command line: BOOT_IMAGE=(lvmid/nZkWaZ-f6bk-Bfto-h9OG-k1Sc-Y6RB-1Q3yZV/t77pr1-3H2Y-ml4l-MMJh-bp3H-zk2j-6z4W6w)/boot/vmlinuz-5.14.0-70.5.1.el9_0.x86_64 root=LABEL=img-rootfs ro console=ttyS0 console=ttyS0,115200n81 no_timer_check crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M hugepagesz=1GB hugepages=32 default_hugepagesz=1GB mem_encrypt=on kvm_amd.sev=1 console=tty0 console=ttyS0,115200 no_timer_check nofb nomodeset vga=normal console=tty0 console=ttyS0,115200 audit=1 nousb
[ 0.000000] Any video related functionality will be severely degraded, and you may not even be able to suspend the system properly
[ 101.753478] ccp 0000:24:00.1: sev enabled
[ 101.769894] ccp 0000:24:00.1: SEV firmware update successful
[ 102.058746] ccp 0000:24:00.1: SEV API:0.24 build:14
[ 120.398153] systemd[1]: Hostname set to <computeamdsev-1>.
[ 149.487548] SEV supported: 509 ASIDs

# MEM_ENCRYPTION_CONTEXT is not present
(overcloud) [stack@undercloud-0 ~]$ !21
openstack --os-placement-api-version 1.17 resource provider trait list ba3bccf9-c283-4cb5-a14d-35ae7ba88533
/usr/lib/python3.9/site-packages/ansible/_vendor/__init__.py:42: UserWarning: One or more Python packages bundled by this ansible-core distribution were already loaded (pyparsing). This may result in undefined behavior.
  warnings.warn('One or more Python packages bundled by this ansible-core distribution were already '
+---------------------------------------+
| name |
+---------------------------------------+
| COMPUTE_GRAPHICS_MODEL_NONE |
| COMPUTE_ACCELERATORS |
| COMPUTE_NET_VIF_MODEL_VMXNET3 |
| COMPUTE_STORAGE_BUS_VIRTIO |
| COMPUTE_NET_VIF_MODEL_E1000E |
| COMPUTE_VOLUME_ATTACH_WITH_TAG |
| COMPUTE_NET_ATTACH_INTERFACE |
| HW_CPU_X86_BMI2 |
| COMPUTE_VOLUME_EXTEND |
| HW_CPU_X86_SSE |
| COMPUTE_NET_VIF_MODEL_RTL8139 |
| COMPUTE_GRAPHICS_MODEL_VIRTIO |
| COMPUTE_IMAGE_TYPE_RAW |
| COMPUTE_TRUSTED_CERTS |
| HW_CPU_X86_SSE42 |
| HW_CPU_X86_SSSE3 |
| HW_CPU_X86_SSE2 |
| COMPUTE_STORAGE_BUS_IDE |
| COMPUTE_SECURITY_UEFI_SECURE_BOOT |
| COMPUTE_SOCKET_PCI_NUMA_AFFINITY |
| COMPUTE_IMAGE_TYPE_AMI |
| COMPUTE_GRAPHICS_MODEL_CIRRUS |
| COMPUTE_VOLUME_MULTI_ATTACH |
| HW_CPU_X86_SSE4A |
| HW_CPU_X86_SSE41 |
| COMPUTE_IMAGE_TYPE_QCOW2 |
| COMPUTE_IMAGE_TYPE_AKI |
| HW_CPU_X86_AVX2 |
| HW_CPU_X86_FMA3 |
| HW_CPU_X86_MMX |
| HW_CPU_HYPERTHREADING |
| COMPUTE_NET_VIF_MODEL_NE2K_PCI |
| HW_CPU_X86_SVM |
| HW_CPU_X86_AVX |
| COMPUTE_IMAGE_TYPE_ISO |
| HW_CPU_X86_CLMUL |
| HW_CPU_X86_ABM |
| COMPUTE_NET_VIF_MODEL_SPAPR_VLAN |
| COMPUTE_STORAGE_BUS_SCSI |
| HW_CPU_X86_AMD_SVM |
| COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG |
| COMPUTE_STORAGE_BUS_FDC |
| COMPUTE_NET_VIF_MODEL_VIRTIO |
| COMPUTE_NET_VIF_MODEL_PCNET |
| COMPUTE_STORAGE_BUS_SATA |
| HW_CPU_X86_F16C |
| COMPUTE_NET_VIF_MODEL_E1000 |
| COMPUTE_DEVICE_TAGGING |
| COMPUTE_NODE |
| COMPUTE_GRAPHICS_MODEL_VGA |
| COMPUTE_IMAGE_TYPE_ARI |
| HW_CPU_X86_SHA |
| HW_CPU_X86_AESNI |
| COMPUTE_RESCUE_BFV |
| COMPUTE_STORAGE_BUS_USB |
| HW_CPU_X86_BMI |
+---------------------------------------+

It is seen on stable/wallaby.

From the compute logs I see that:

2022-05-23 21:25:20.873 2 DEBUG nova.virt.libvirt.host [req-bc5c2030-5a68-4f5e-be8b-924f24962ef9 - - - - -] /sys/module/kvm_amd/parameters/sev contains [Y
] _kernel_supports_amd_sev /usr/lib/python3.9/site-packages/nova/virt/libvirt/host.py:1557
2022-05-23 21:25:20.873 2 INFO nova.virt.libvirt.host [req-bc5c2030-5a68-4f5e-be8b-924f24962ef9 - - - - -] kernel doesn't support AMD SEV

The nova code looks for the "1\n" [1] in the file but it contains "Y\n" instead

    def _kernel_supports_amd_sev(self) -> bool:
        if not os.path.exists(SEV_KERNEL_PARAM_FILE):
            LOG.debug("%s does not exist", SEV_KERNEL_PARAM_FILE)
            return False

        with open(SEV_KERNEL_PARAM_FILE) as f:
            contents = f.read()
            LOG.debug("%s contains [%s]", SEV_KERNEL_PARAM_FILE, contents)
            return contents == "1\n"

So it seems like a valid bug in nova.

[1] https://github.com/openstack/nova/blob/e44b1a940fdc45cc9dbb08e193a8c25052cf64e7/nova/virt/libvirt/host.py#L1696-L1704

Changed in nova:
assignee: nobody → Balazs Gibizer (balazs-gibizer)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/843254

Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.opendev.org/c/openstack/nova/+/843254
Committed: https://opendev.org/openstack/nova/commit/ab51a5dd25b8d4c66562148b43b1022eb5ceed7e
Submitter: "Zuul (22348)"
Branch: master

commit ab51a5dd25b8d4c66562148b43b1022eb5ceed7e
Author: Balazs Gibizer <email address hidden>
Date: Wed May 25 12:02:09 2022 +0200

    Accept both 1 and Y as AMD SEV KVM kernel param value

    The libvirt virt dirver checks the AMD KVM kernel module parameter SEV
    to see if that feature is enabled. However it seems that the
    /sys/module/kvm_amd/parameters/sev file can either contain "1\n" or
    "Y\n" to indicate that the feature is enabled. Nova only checked for
    "1\n" so far making the feature disabled on compute nodes with "Y\n"
    value. Now the logic is extended to accept both.

    Closes-Bug: #1975686
    Change-Id: I737e1d73242430b6756178eb0bf9bd6ec5c94160

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/nova/+/843819

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/843938

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/843939

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 26.0.0.0rc1

This issue was fixed in the openstack/nova 26.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/nova/+/843819
Committed: https://opendev.org/openstack/nova/commit/8a1b4975f71f9ce1446db689afb092d6e0a670a7
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 8a1b4975f71f9ce1446db689afb092d6e0a670a7
Author: Balazs Gibizer <email address hidden>
Date: Wed May 25 12:02:09 2022 +0200

    Accept both 1 and Y as AMD SEV KVM kernel param value

    The libvirt virt dirver checks the AMD KVM kernel module parameter SEV
    to see if that feature is enabled. However it seems that the
    /sys/module/kvm_amd/parameters/sev file can either contain "1\n" or
    "Y\n" to indicate that the feature is enabled. Nova only checked for
    "1\n" so far making the feature disabled on compute nodes with "Y\n"
    value. Now the logic is extended to accept both.

    Closes-Bug: #1975686
    Change-Id: I737e1d73242430b6756178eb0bf9bd6ec5c94160
    (cherry picked from commit ab51a5dd25b8d4c66562148b43b1022eb5ceed7e)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/nova/+/843938
Committed: https://opendev.org/openstack/nova/commit/1aa9e0de6fb7049f531908b20613c7d2ab18c2e3
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 1aa9e0de6fb7049f531908b20613c7d2ab18c2e3
Author: Balazs Gibizer <email address hidden>
Date: Wed May 25 12:02:09 2022 +0200

    Accept both 1 and Y as AMD SEV KVM kernel param value

    The libvirt virt dirver checks the AMD KVM kernel module parameter SEV
    to see if that feature is enabled. However it seems that the
    /sys/module/kvm_amd/parameters/sev file can either contain "1\n" or
    "Y\n" to indicate that the feature is enabled. Nova only checked for
    "1\n" so far making the feature disabled on compute nodes with "Y\n"
    value. Now the logic is extended to accept both.

    Closes-Bug: #1975686
    Change-Id: I737e1d73242430b6756178eb0bf9bd6ec5c94160
    (cherry picked from commit ab51a5dd25b8d4c66562148b43b1022eb5ceed7e)
    (cherry picked from commit 8a1b4975f71f9ce1446db689afb092d6e0a670a7)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 24.2.1

This issue was fixed in the openstack/nova 24.2.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 25.1.1

This issue was fixed in the openstack/nova 25.1.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.