Support for XSAVES intel instructions in QEMU

[Alexandre Bécholey]

Dear QEMU developers,

I am running Hyper-V on qemu+kvm. During it initialization, it checks for XSAVES support: first it executes CPUID with EAX = 0xd and ECX = 1 and looks at bit 3 in the returned value of EAX (Supports XSAVES/XRSTORS and IA32_XSS [1]), and then it reads the MSR IA32_VMX_PROCBASED_CTLS2 (index 0x48B) and looks at bit 20 (Enable XSAVES/XSTORS [2]). If CPUID shows that XSAVES is supported and the bit is not enabled in the MSR, Hyper-V decides to fail and stops its initialization. It used to work until last spring/summer where something might have changed in either KVM or QEMU.

It seems that KVM sets the correct flags (in CPUID and the MSR) when the host CPU supports XSAVES. In QEMU, based on comments in target/i386/cpu.c it seems that XSAVES is not added in
builtin_x86_defs[].features[FEAT_VMX_SECONDARY_CTLS] because it might break live migration. Therefore, when setting the MSR for the vcpu, QEMU is masking off the feature.

I have tested two possible solutions:
- adding the flag in .features[FEAT_VMX_SECONDARY_CTLS]
- removing the support of the instruction in feature_word_info[FEAT_XSAVE].feat_names

Both solutions work and Hyper-v is happily running. I can provide a patch for the solution you might consider applying. Otherwise, is there a better way to fix the issue?

Qemu version: 4.2.0
Kernel version: 5.5.4
Qemu command: https://gist.github.com/0xabe-io/b4d797538e2160252addc1d1d64738e2

Many thanks,
Alexandre

Ref:
[1] Intel SDM Volume 2A, chapter 3, page 196
[2] Intel SDM Volume 3C, chapter 24, page 11

[Paolo Bonzini (bonzini)]

Are you using Libvirt? If so, you can just remove xsaves in the Libvirt XML ("<feature policy='disable' name='xsaves'/>">).

It seems to me that this is a Hyper-V bug, but I understand that this is not a configuration that happens on real hardware.

Adding the flag to all Skylake and newer systems (including Denverton and Snowridge) is the best choice, but we cannot just add it; if you want to send a patch, see the "Intel Atom Processor (SnowRidge)" model for an example of how to do it.

[Alexandre Bécholey (alexandre-becholey)]

Yes, I am using Libvirt and disabling the feature that way works, thank you!

I can provide a patch. However I don't understand what you mean by: "but we cannot just add it". If you cannot add it, the patch will be rejected, right?

[Thomas Huth (th-huth)]

The QEMU project is currently moving its bug tracking to another system.
For this we need to know which bugs are still valid and which could be
closed already. Thus we are setting older bugs to "Incomplete" now.

If you still think this bug report here is valid, then please switch
the state back to "New" within the next 60 days, otherwise this report
will be marked as "Expired". Or please mark it as "Fix Released" if
the problem has been solved with a newer version of QEMU already.

Thank you and sorry for the inconvenience.

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]