Jammy Libvirt Support for Intel Sapphire Rapids

[Ammad Ali]

[Impact]

Libvirt in previous versions of Ubuntu is missing support for Intel Sapphire Rapids processors.

This should be fixed as a part of hardware enablement.

The issue is fixed through a series of patches that add necessary CPU features and definitions, and adding support for alias names in Jammy.

[Test Plan]

To test the fix, an Intel Sapphire Rapids system must be used.

Install libvirt with:

$ sudo apt install qemu-kvm libvirt-daemon-system

Then check the processor identification with virsh:

$ virsh capabilities

It should respond with a model of either SapphireRapids or SapphireRapids-noTSX in the format:

<capabilities>

  <host>
    <uuid>XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>SapphireRapids-noTSX</model>
      <vendor>Intel</vendor>
...

[Where problems could occur]

The series of patches provided for this fix all relate to one of two categories. The first is CPU map and feature additions, which includes patches with the name *add-missing-feature-*.patch and *add-new-CPU-model-*.patch. For these the main problem that could occur is a change of identification for processors that use these features. These processors are likely not yet supported anyway, but could be identified as Sapphire Rapids instead of something else.

The second category is patches that enable support for alias names in features in Jammy. These make some fundamental changes to CPU map interpretation, and could cause general issues checking features for CPU models.

[Other Info]

The fix for this is included alongside the fix for https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/2027838

[Original Description]

Hi,

Few months back, libvirt upstream now have support for Intel Sapphire Rapids CPU model support.

https://github.com/libvirt/libvirt/commit/96c8d39af007000daf3d5dfa845365f66379aaac

It would be great if that can be backported to jammy libvirt release.

[Lena Voytek (lvoytek)]

Hello Ammad,

I created a PPA for lunar and jammy that backports Sapphire Rapids support here: https://launchpad.net/~lvoytek/+archive/ubuntu/libvirt-sapphirerapids-uri

If you would like to test it you can run the following:

$ sudo add-apt-repository ppa:lvoytek/libvirt-sapphirerapids-uri
$ sudo apt update
$ sudo apt upgrade

Thanks!

[Ammad Ali (syedammad83)]

Hi,

Here is my lscpu info.

# lscpu
Architecture: x86_64
  CPU op-mode(s): 32-bit, 64-bit
  Address sizes: 46 bits physical, 57 bits virtual
  Byte Order: Little Endian
CPU(s): 128
  On-line CPU(s) list: 0-127
Vendor ID: GenuineIntel
  Model name: Intel(R) Xeon(R) Gold 6430
    CPU family: 6
    Model: 143
    Thread(s) per core: 2
    Core(s) per socket: 32
    Socket(s): 2
    Stepping: 8
    Frequency boost: enabled
    CPU max MHz: 2101.0000
    CPU min MHz: 800.0000
    BogoMIPS: 4200.00
    Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc
                          art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma
                          cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 c
                         dp_l3 invpcid_single cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invp
                         cid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_o
                         ccup_llc cqm_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni
                         vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm md_clear serialize tsxldtrk pcon
                         fig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities
Virtualization features:
  Virtualization: VT-x
Caches (sum of all):
  L1d: 3 MiB (64 instances)
  L1i: 2 MiB (64 instances)
  L2: 128 MiB (64 instances)
  L3: 120 MiB (2 instances)
NUMA:
  NUMA node(s): 2
  NUMA node0 CPU(s): 0-31,64-95
  NUMA node1 CPU(s): 32-63,96-127
Vulnerabilities:
  Gather data sampling: Not affected
  Itlb multihit: Not affected
  L1tf: Not affected
  Mds: Not affected
  Meltdown: Not affected
  Mmio stale data: Not affected
  Retbleed: Not affected
  Spec rstack overflow: Not affected
  Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
  Srbds: Not affected
  Tsx async abort: Not affected

I have installed the libvirt pa...

[Lena Voytek (lvoytek)]

Hi Ammad,

Thanks for testing and providing all the info, sorry the fix failed. I updated the ppa to include the noTSX model for Sapphire Rapids which should be picked up on your system if you would like to test again.

Thanks!

[Ammad Ali (syedammad83)]

Hi,

I have tested and its working fine now.

# dpkg -l | grep libvirt
ii libsys-virt-perl 8.0.0-1build1 amd64 Perl module providing an extension for the libvirt library
ii libvirt-clients 8.0.0-1ubuntu7.8~ppa2 amd64 Programs for the libvirt library
ii libvirt-daemon 8.0.0-1ubuntu7.8~ppa2 amd64 Virtualization daemon
ii libvirt-daemon-config-network 8.0.0-1ubuntu7.8~ppa2 all Libvirt daemon configuration files (default network)
ii libvirt-daemon-config-nwfilter 8.0.0-1ubuntu7.8~ppa2 all Libvirt daemon configuration files (default network filters)
ii libvirt-daemon-driver-qemu 8.0.0-1ubuntu7.8~ppa2 amd64 Virtualization daemon QEMU connection driver
ii libvirt-daemon-system 8.0.0-1ubuntu7.8~ppa2 amd64 Libvirt daemon configuration files
ii libvirt-daemon-system-systemd 8.0.0-1ubuntu7.8~ppa2 all Libvirt daemon configuration files (systemd)

# virsh capabilities
<capabilities>

  <host>
    <uuid>00000000-0000-0000-0000-3cecef36b5d9</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>SapphireRapids-noTSX</model>
      <vendor>Intel</vendor>
      <microcode version='721421520'/>
      <counter name='tsc' frequency='2100000000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='32' threads='2'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='tsc_adjust'/>
      <feature name='sgx'/>
      <feature name='cmt'/>
      <feature name='intel-pt'/>
      <feature name='ospke'/>
      <feature name='waitpkg'/>
      <feature name='cldemote'/>
      <feature name='movdiri'/>
      <feature name='movdir64b'/>
      <feature name='sgxlc'/>
      <feature name='pks'/>
      <feature name='md-clear'/>
      <feature name='pconfig'/>
      <feature name='arch-lbr'/>
      <feature name='stibp'/>
      <feature name='core-capability'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='invtsc'/>
      <feature name='tsx-ctrl'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>

[Lena Voytek (lvoytek)]

Thanks for the verification! I'll update the bug to prepare for the change and get the patches through review

[Timo Aaltonen (tjaalton)]

Hello Ammad, or anyone else affected,

Accepted libvirt into lunar-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/9.0.0-2ubuntu1.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-lunar to verification-done-lunar. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-lunar. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello Ammad, or anyone else affected,

Accepted libvirt into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/8.0.0-1ubuntu7.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ammad Ali (syedammad83)]

Hi,

I have tested the package on jammy-proposed its working fine.

# virsh capabilities
<capabilities>

  <host>
    <uuid>00000000-0000-0000-0000-3cecef36b5d9</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>SapphireRapids-noTSX</model>
      <vendor>Intel</vendor>
      <microcode version='721421520'/>
      <counter name='tsc' frequency='2100000000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='32' threads='2'/>

# dpkg -l | grep libvirt
ii libvirt-clients 8.0.0-1ubuntu7.8 amd64 Programs for the libvirt library
ii libvirt-daemon 8.0.0-1ubuntu7.8 amd64 Virtualization daemon
ii libvirt-daemon-config-network 8.0.0-1ubuntu7.8 all Libvirt daemon configuration files (default network)
ii libvirt-daemon-config-nwfilter 8.0.0-1ubuntu7.8 all Libvirt daemon configuration files (default network filters)
ii libvirt-daemon-driver-qemu 8.0.0-1ubuntu7.8 amd64 Virtualization daemon QEMU connection driver
ii libvirt-daemon-system 8.0.0-1ubuntu7.8 amd64 Libvirt daemon configuration files
ii libvirt-daemon-system-systemd 8.0.0-1ubuntu7.8 all Libvirt daemon configuration files (systemd)
ii libvirt-glib-1.0-data 4.0.0-2 all Common files for libvirt GLib library
ii libvirt0:amd64 8.0.0-1ubuntu7.8 amd64 library for interfacing with different virtualization systems

Ammad Ali

[Lena Voytek (lvoytek)]

Thanks for the verification!

[Ammad Ali (syedammad83)]

I have tested the package on Lunar, its not picking up correct cpu model.

# dpkg -l | grep libvirt
ii libvirt-clients 9.0.0-2ubuntu1.3 amd64 Programs for the libvirt library
ii libvirt-daemon 9.0.0-2ubuntu1.3 amd64 Virtualization daemon
ii libvirt-daemon-config-network 9.0.0-2ubuntu1.3 all Libvirt daemon configuration files (default network)
ii libvirt-daemon-config-nwfilter 9.0.0-2ubuntu1.3 all Libvirt daemon configuration files (default network filters)
ii libvirt-daemon-driver-qemu 9.0.0-2ubuntu1.3 amd64 Virtualization daemon QEMU connection driver
ii libvirt-daemon-system 9.0.0-2ubuntu1.3 amd64 Libvirt daemon configuration files
ii libvirt-daemon-system-systemd 9.0.0-2ubuntu1.3 all Libvirt daemon configuration files (systemd)
ii libvirt-l10n 9.0.0-2ubuntu1.3 all localization for the libvirt library
ii libvirt0:amd64 9.0.0-2ubuntu1.3 amd64 library for interfacing with different virtualization systems

  <host>
    <uuid>9792f200-b8b3-11ed-8000-3cecef36b5d9</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Broadwell-noTSX-IBRS</model>
      <vendor>Intel</vendor>
      <microcode version='721421520'/>
      <signature family='6' model='143' stepping='8'/>
      <counter name='tsc' frequency='2100000000' scaling='yes'/>
      <topology sockets='1' dies='1' cores='32' threads='2'/>
      <maxphysaddr mode='emulate' bits='46'/>
      <feature name='vme'/>

# lscpu
Architecture: x86_64
  CPU op-mode(s): 32-bit, 64-bit
  Address sizes: 46 bits physical, 57 bits virtual
  Byte Order: Little Endian
CPU(s): 128
  On-line CPU(s) list: 0-127
Vendor ID: GenuineIntel
  BIOS Vendor ID: Intel(R) Corporation
  Model name: Intel(R) Xeon(R) Gold 6430
    BIOS Model name: Intel(R) Xeon(R) Gold 6430 CPU @ 2.1GHz

# lsb_release -c
No LSB modules are available.
Codename: lunar

[Christian Ehrhardt  (paelzer)]

Thank you Ammad for the testing on the respective hardware, due to the downtime via Christmas time not much happened the last two weeks.

OTOH sadly yet another debug + spin a build + verification round will make the fix appear just when lunar goes out of support.

@Lena if this seems easy feel free to refresh it, but otherwise should we just not fix lunar given that the much longer living Jammy version is fine?

[Lena Voytek (lvoytek)]

Thanks for the help Ammad. Since the patches to get lunar are substantially different from Jammy and since lunar is going EOL this month it's probably best to skip it now. Marking as wont-fix

[Ammad Ali (syedammad83)]

Ack, when will this patch released to jammy updates ?

[Christian Ehrhardt  (paelzer)]

Now everything is ready it should happen almost any day.

There is a bit of a backlog that piled up over the Christmas shutdown, but it looks all good and now just need an SRU team member to release it.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 8.0.0-1ubuntu7.8

---------------
libvirt (8.0.0-1ubuntu7.8) jammy; urgency=medium

  * d/p/u/lp-2028057-*, d/libvirt0.install: Add named types and definitions,
    along with QEMU alias syncing for Intel SapphireRapids (LP: #2028057)

 -- Lena Voytek <email address hidden> Wed, 29 Nov 2023 14:52:52 -0700

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for libvirt has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.