KVM_EXIT_MMIO has increased in Qemu4.0.0 when compared to Qemu 2.11.0

[ANIMESH KUMAR SINHA]

I was able to generate trace dump in Qemu for kvm_run_exit event in both QEMU 2.11.0 and QEMU 4.0.0
From the trace i noticed that the number of KVM_KXIT_MMIO calls has increased alot and is causing delay in testcase execution.

I executed same testcase from Qemu 2.11 and Qemu4.
Inside Virtual machine when using qemu 2.11 testcase got completed in 11 seconds
but the same testcase when executed on Qemu 4.0.0 got executed in 26 seconds.

I did a bit of digging and extracted the kvm_run_exit to figure out whats going on.

Please find
Stats from Qemu2.11:

KVM_EXIT_UNKNOWN : 0
KVM_EXIT_EXCEPTION : 0
KVM_EXIT_IO : 182513
KVM_EXIT_HYPERCALL : 0
KVM_EXIT_DEBUG : 0
KVM_EXIT_HLT : 0
KVM_EXIT_MMIO : 216701
KVM_EXIT_IRQ_WINDOW_OPEN : 0
KVM_EXIT_SHUTDOWN : 0
KVM_EXIT_FAIL_ENTRY : 0
KVM_EXIT_INTR : 0
KVM_EXIT_SET_TPR : 0
KVM_EXIT_TPR_ACCESS : 0
KVM_EXIT_S390_SIEIC : 0
KVM_EXIT_S390_RESET : 0
KVM_EXIT_DCR : 0
KVM_EXIT_NMI : 0
KVM_EXIT_INTERNAL_ERROR : 0
KVM_EXIT_OSI : 0
KVM_EXIT_PAPR_HCALL : 0
KVM_EXIT_S390_UCONTROL : 0
KVM_EXIT_WATCHDOG : 0
KVM_EXIT_S390_TSCH : 0
KVM_EXIT_EPR : 0
KVM_EXIT_SYSTEM_EVENT : 0
KVM_EXIT_S390_STSI : 0
KVM_EXIT_IOAPIC_EOI : 0
KVM_EXIT_HYPERV : 0

KVM_RUN_EXIT : 399214 (Total in Qemu 2.11 for a testcase)

Stats For Qemu 4.0.0:

VM_EXIT_UNKNOWN : 0
KVM_EXIT_EXCEPTION : 0
KVM_EXIT_IO : 163729
KVM_EXIT_HYPERCALL : 0
KVM_EXIT_DEBUG : 0
KVM_EXIT_HLT : 0
KVM_EXIT_MMIO : 1094231
KVM_EXIT_IRQ_WINDOW_OPEN : 46
KVM_EXIT_SHUTDOWN : 0
KVM_EXIT_FAIL_ENTRY : 0
KVM_EXIT_INTR : 0
KVM_EXIT_SET_TPR : 0
KVM_EXIT_TPR_ACCESS : 0
KVM_EXIT_S390_SIEIC : 0
KVM_EXIT_S390_RESET : 0
KVM_EXIT_DCR : 0
KVM_EXIT_NMI : 0
KVM_EXIT_INTERNAL_ERROR : 0
KVM_EXIT_OSI : 0
KVM_EXIT_PAPR_HCALL : 0
KVM_EXIT_S390_UCONTROL : 0
KVM_EXIT_WATCHDOG : 0
KVM_EXIT_S390_TSCH : 0
KVM_EXIT_EPR : 0
KVM_EXIT_SYSTEM_EVENT : 0
KVM_EXIT_S390_STSI : 0
KVM_EXIT_IOAPIC_EOI : 464
KVM_EXIT_HYPERV : 0

KVM_RUN_EXIT : 1258470 (Total in qemu 4.0.0 for same testcase)

From above analysis i found that the number of KVM_EXIT_MMIO has increased by 4.x.

Could someone from qemu community help me understand as to why the MMIO exits have increased in qemu4 ?

The results i obtained are after running same testcase.
On Qemu2.11 testcase gets completed in : 11seconds
on Qemu4.11 testcase gets completed in : 26 seconds

VM Qcow2 used in Ubuntu 16.04
VM kernel OS is : 4.4 generic

Let me know incase more information is required .

[Thomas Huth (th-huth)]

Can you provide a link to the test case that you used? Otherwise, can you try to bisect the problem? Can you please also try with the latest version of QEMU? v4.0 is rather old already.

[ANIMESH KUMAR SINHA (animeshk)]

The testcase is a rather a very basic one.
Doing a mmap on a pcie device and triggering a read and write operation on the Virtual Address, which would in-turn trigger transaction requests on the host device.

I understand that Qemu v4.0 may be and old one, but the issue is that test case execution has become rather very slow with Qemu4.0.0 and the only thing i could notice from using tracebacks were related with increase in KVM_MMIO_EXIT when compared with Qemu2.11.0 version.

Can it be verified as to why the number of KVM_EXIT_MMIO has increased drastically in qemu4 when compared to Qemu2.11.0 ?

[ANIMESH KUMAR SINHA (animeshk)]

Could someone update on this?

[ANIMESH KUMAR SINHA (animeshk)]

One observation i made was that adding qemu arguments "-nic none" improved the testcase performance.

Does "-nic none" reduce KVM_EXIT_MMIO ?

Are there any other qemu arguments that can be used to improve qemu overall performance ?