default cpu (qemu64) no more capable of nesting
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Release Notes for Ubuntu |
Fix Released
|
Undecided
|
Christian Ehrhardt | ||
libvirt (Ubuntu) |
Invalid
|
Undecided
|
Unassigned | ||
qemu (Ubuntu) |
Fix Released
|
Undecided
|
Christian Ehrhardt |
Bug Description
TL;DR this is the time to decide to either drop debian/
Default nesting issue:
uvt-kvm create --memory 2048 --cpu 4 --disk 16 --password=ubuntu focal-kvm release=focal arch=amd64 label=daily
Default CPU used is:
<cpu mode='custom' match='exact' check='full'>
<model fallback=
<feature policy='require' name='vmx'/> <-- even has VMX enabled
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='lahf_lm'/>
<feature policy='disable' name='svm'/>
</cpu>
Guest:
uvt-kvm create --disk 5 --machine-type ubuntu --password=ubuntu focal-2nd-lvm release=focal arch=amd64 label=daily
It comes down to non-loadable module in the lvl1 guest:
$ sudo modprobe kvm_intel
modprobe: ERROR: could not insert 'kvm_intel': Input/output error
Try the same with host-passthrough to check if it is the (default) cpu type
<cpu mode='host-
$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used
<cpu mode='host-model' check='none'/>
Even adapting the qemu64 type to represent the features of Haswell didn't work.
<cpu mode='custom' match='exact' check='full'>
<model fallback=
<feature policy='require' name='vmx'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='lahf_lm'/>
<feature policy='disable' name='svm'/>
<feature policy='require' name='aes'/>
<feature policy='require' name='avx'/>
<feature policy='require' name='avx2'/>
<feature policy='require' name='bmi1'/>
<feature policy='require' name='bmi2'/>
<feature policy='require' name='erms'/>
<feature policy='require' name='fma'/>
<feature policy='require' name='fsgsbase'/>
<feature policy='require' name='invpcid'/>
<feature policy='require' name='movbe'/>
<feature policy='require' name='pcid'/>
<feature policy='require' name='pclmuldq'/>
<feature policy='require' name='popcnt'/>
<feature policy='require' name='rdtscp'/>
<feature policy='require' name='smep'/>
<feature policy='require' name='spec-ctrl'/>
<feature policy='require' name='sse4.1'/>
<feature policy='require' name='sse4.2'/>
<feature policy='require' name='ssse3'/>
<feature policy='require' name='tsc-
<feature policy='require' name='xsave'/>
<feature policy='require' name='ss'/>
<feature policy='require' name='vme'/>
<feature policy='require' name='pat'/>
<feature policy='require' name='rdrand'/>
<feature policy='require' name='f16c'/>
<feature policy='require' name='arat'/>
<feature policy='require' name='tsc_adjust'/>
<feature policy='require' name='umip'/>
<feature policy='require' name='md-clear'/>
<feature policy='require' name='stibp'/>
<feature policy='require' name='arch-
<feature policy='require' name='ssbd'/>
<feature policy='require' name='xsaveopt'/>
<feature policy='require' name='pdpe1gb'/>
<feature policy='require' name='abm'/>
<feature policy='require' name='ibpb'/>
<feature policy='require' name='amd-ssbd'/>
<feature policy='require' name='skip-
</cpu>
The reason is that VMX now is set in subfeatures and therefore even with the same "input" definition the guest looses features.
60a63,68
> tpr_shadow
> vnmi
> flexpriority
> ept
> vpid
> ept_ad
This is just dependent on the userspace stack (qemu upgrade) due to the change:
https:/
Even the same commandline will deliver different results:
Eoan vs Focal
E:
-cpu qemu64,
F:
-cpu qemu64,
Just remaining differences:
-lahf_lm=on
+lahf-lm=on
-pclmuldq=on
+pclmulqdq=on
-tsc_adjust=on
+tsc-adjust=on
=> args renamed
But CPU flags change a lot:
-tpr_shadow
-vnmi
-flexpriority
-ept
-vpid
-ept_ad
Due to the commit above our old Delta in debian/
We'll need to accept the degradation (to be closer to upstream) or - this also will be an upgrade regression for some users - fix the bug by changing it to what was added to the kvm64 type in the commit above.
+ /* VMX features from Cedar Mill/Prescott */
+ .features[
+ .features[
+ .features[
+ .features[
+ VMX_PIN_
+ .features[
+ VMX_CPU_
+ VMX_CPU_
+ VMX_CPU_
+ VMX_CPU_
+ VMX_CPU_
+ VMX_CPU_
+ VMX_CPU_
.xlevel = 0x80000008,
.model_id = "Common KVM processor"
Related branches
- Rafael David Tinoco (community): Approve
- Canonical Server: Pending requested
- git-ubuntu developers: Pending requested
-
Diff: 7167 lines (+6697/-18)68 files modifieddebian/changelog (+29/-0)
debian/patches/series (+64/-1)
debian/patches/stable/lp-1867519-arm-arm-powerctl-rebuild-hflags-after-setting-CP15-b.patch (+48/-0)
debian/patches/stable/lp-1867519-arm-arm-powerctl-set-NSACR.-CP11-CP10-bits-in-arm_se.patch (+49/-0)
debian/patches/stable/lp-1867519-backup-top-Begin-drain-earlier.patch (+46/-0)
debian/patches/stable/lp-1867519-block-Activate-recursively-even-for-already-active-n.patch (+108/-0)
debian/patches/stable/lp-1867519-block-backup-top-fix-failure-path.patch (+97/-0)
debian/patches/stable/lp-1867519-block-block-copy-fix-progress-calculation.patch (+201/-0)
debian/patches/stable/lp-1867519-block-fix-crash-on-zero-length-unaligned-write-and-r.patch (+107/-0)
debian/patches/stable/lp-1867519-block-io-fix-bdrv_co_do_copy_on_readv.patch (+44/-0)
debian/patches/stable/lp-1867519-block-nbd-extract-the-common-cleanup-code.patch (+78/-0)
debian/patches/stable/lp-1867519-block-nbd-fix-memory-leak-in-nbd_open.patch (+76/-0)
debian/patches/stable/lp-1867519-block-qcow2-threads-fix-qcow2_decompress.patch (+79/-0)
debian/patches/stable/lp-1867519-hw-i386-pc-fix-regression-in-parsing-vga-cmdline-par.patch (+58/-0)
debian/patches/stable/lp-1867519-intel_iommu-a-fix-to-vtd_find_as_from_bus_num.patch (+44/-0)
debian/patches/stable/lp-1867519-intel_iommu-add-present-bit-check-for-pasid-table-en.patch (+202/-0)
debian/patches/stable/lp-1867519-iotests-add-test-for-backup-top-failure-on-permissio.patch (+138/-0)
debian/patches/stable/lp-1867519-job-refactor-progress-to-separate-object.patch (+230/-0)
debian/patches/stable/lp-1867519-plugins-core-add-missing-break-in-cb_to_tcg_flags.patch (+41/-0)
debian/patches/stable/lp-1867519-qcow2-Fix-alloc_cluster_abort-for-pre-existing-clust.patch (+39/-0)
debian/patches/stable/lp-1867519-qcow2-Fix-qcow2_alloc_cluster_abort-for-external-dat.patch (+44/-0)
debian/patches/stable/lp-1867519-qcow2-bitmaps-fix-qcow2_can_store_new_dirty_bitmap.patch (+102/-0)
debian/patches/stable/lp-1867519-qemu-img-Fix-convert-n-B-for-backing-less-targets.patch (+54/-0)
debian/patches/stable/lp-1867519-s390-sclp-improve-special-wait-psw-logic.patch (+40/-0)
debian/patches/stable/lp-1867519-target-arm-Return-correct-IL-bit-in-merge_syn_data_a.patch (+46/-0)
debian/patches/stable/lp-1867519-target-arm-Set-ISSIs16Bit-in-make_issinfo.patch (+42/-0)
debian/patches/stable/lp-1867519-target-arm-arm-semi-fix-SYS_OPEN-to-return-nonzero-f.patch (+79/-0)
debian/patches/stable/lp-1867519-target-arm-ensure-we-use-current-exception-state-aft.patch (+127/-0)
debian/patches/stable/lp-1867519-target-i386-kvm-initialize-feature-MSRs-very-early.patch (+169/-0)
debian/patches/stable/lp-1867519-tcg-save-vaddr-temp-for-plugin-usage.patch (+98/-0)
debian/patches/stable/lp-1867519-tpm-ppi-page-align-PPI-RAM.patch (+47/-0)
debian/patches/stable/lp-1867519-vfio-pci-Don-t-remove-irqchip-notifier-if-not-regist.patch (+50/-0)
debian/patches/stable/lp-1867519-virtio-gracefully-handle-invalid-region-caches.patch (+331/-0)
debian/patches/stable/lp-1867519-virtio-mmio-update-queue-size-on-guest-write.patch (+40/-0)
debian/patches/stable/lp-1867519-virtio-net-delete-also-control-queue-when-TX-RX-dele.patch (+41/-0)
debian/patches/stable/lp-1867519-virtio-update-queue-size-on-guest-write.patch (+40/-0)
debian/patches/ubuntu/lp-1835546-Sync-pv.patch (+98/-0)
debian/patches/ubuntu/lp-1835546-pc-bios-s390x-Save-iplb-location-in-lowcore.patch (+138/-0)
debian/patches/ubuntu/lp-1835546-s390x-Add-SIDA-memory-ops.patch (+141/-0)
debian/patches/ubuntu/lp-1835546-s390x-Add-missing-vcpu-reset-functions.patch (+165/-0)
debian/patches/ubuntu/lp-1835546-s390x-Add-unpack-facility-feature-to-GA1.patch (+67/-0)
debian/patches/ubuntu/lp-1835546-s390x-Beautify-diag308-handling.patch (+119/-0)
debian/patches/ubuntu/lp-1835546-s390x-Don-t-do-a-normal-reset-on-the-initial-cpu.patch (+41/-0)
debian/patches/ubuntu/lp-1835546-s390x-Move-clear-reset.patch (+135/-0)
debian/patches/ubuntu/lp-1835546-s390x-Move-diagnose-308-subcodes-and-rcs-into-ipl.h.patch (+67/-0)
debian/patches/ubuntu/lp-1835546-s390x-Move-initial-reset.patch (+148/-0)
debian/patches/ubuntu/lp-1835546-s390x-Move-reset-normal-to-shared-reset-handler.patch (+134/-0)
debian/patches/ubuntu/lp-1835546-s390x-ipl-Consolidate-iplb-validity-check-into-one-f.patch (+70/-0)
debian/patches/ubuntu/lp-1835546-s390x-kvm-Make-kvm_sclp_service_call-void.patch (+72/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Add-migration-blocker.patch (+70/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Disable-address-checks-for-PV-guest-I.patch (+126/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Handle-SIGP-store-status-correctly.patch (+50/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Inhibit-balloon-when-switching-to-pro.patch (+91/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-KVM-intercept-changes.patch (+66/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Move-IO-control-structures-over-SIDA.patch (+162/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Move-STSI-data-over-SIDAD.patch (+61/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Move-diag-308-data-over-SIDA.patch (+84/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-SCLP-interpretation.patch (+162/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Set-guest-IPL-PSW.patch (+51/-0)
debian/patches/ubuntu/lp-1835546-s390x-protvirt-Support-unpack-facility.patch (+875/-0)
debian/patches/ubuntu/lp-1847361-modules-load-upgrade.patch (+125/-0)
debian/patches/ubuntu/lp-1847361-vhost-correctly-turn-on-VIRTIO_F_IOMMU_PLATFORM.patch (+61/-0)
debian/qemu-block-extra.postrm.in (+43/-0)
debian/qemu-block-extra.prerm.in (+45/-0)
debian/qemu-system-gui.postrm.in (+44/-0)
debian/qemu-system-gui.prerm.in (+46/-0)
debian/rules (+12/-0)
dev/null (+0/-17)
Changed in qemu (Ubuntu): | |
status: | New → Triaged |
assignee: | nobody → Christian Ehrhardt (paelzer) |
Changed in libvirt (Ubuntu): | |
status: | New → Triaged |
Changed in ubuntu-release-notes: | |
status: | New → In Progress |
Changed in ubuntu-release-notes: | |
status: | In Progress → Fix Released |
With the fix applied, the most reduced case is:
<cpu mode='custom' match='exact' check='full'> 'forbid' >qemu64< /model>
<model fallback=
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='lahf_lm'/>
<feature policy='disable' name='svm'/>
</cpu>
Which ends up as:
-cpu qemu64
That is without VMX (as expected)
If adding FMX as feature (didn't work to enable it before)
<cpu mode='custom' match='exact' check='full'> 'forbid' >qemu64< /model>
<model fallback=
<feature policy='require' name='vmx'/>
=> Nested KVM works
=> It got a smaller subset of cpuflags than in the past - just vmx + tpr_shadow
And if started with <cpu> tag at all it gets: 'forbid' >qemu64< /model>
<cpu mode='custom' match='exact' check='full'>
<model fallback=
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='lahf_lm'/>
<feature policy='disable' name='svm'/>
</cpu>
So we are missing to get VMX auto added here as we did in the past.
We can enable VMX now (ok for upgraders of old guests), but the same XML/commandline still is too different.
Maybe I need to re-add the main flag as well, lets do a rebuild over night.