qemu-system-x86_64 takes 100% CPU

[chenlidong]

I have rhel6 inside qemu VM. qemu process starts to take CPU cycles and OS inside VM is very slow and sluggish.

the qemu version is the latest upstream git.

the kernel version is 3.12.0.

linux-0rsg:/home/chenlidong # uname -a
Linux linux-0rsg 3.12.0-rc1-1.16-desktop+ #5 SMP PREEMPT Sun Sep 22 22:07:40 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux

linux-0rsg:/home/chenlidong # qemu-system-x86_64 --version
QEMU emulator version 1.6.50, Copyright (c) 2003-2008 Fabrice Bellard

the command line of qemu is below:

chenlidong@linux-0rsg:~/develop/qemu> ps -ef | grep qemu
root 19030 1 14 19:00 ? 00:04:24 /usr/local/bin/qemu-system-x86_64 -name rhel6 -S -M pc-i440fx-1.6 -m 2048 -smp 1,sockets=1,cores=1,threads=1 -uuid 1925a96a-54b9-3c4a-dda0-6b42fdd0af2c -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/rhel6.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/rhel6.img,if=none,id=drive-ide0-0-0,format=raw,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=21,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:b3:b8:53,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

i used perf top, the result is below:
Samples: 57K of event 'cycles', Event count (approx.): 26336443124
 15.38% perf-22465.map [.] 0x00007f143b99c9c6
  5.90% qemu-system-x86_64 [.] phys_page_find
  4.48% qemu-system-x86_64 [.] address_space_translate_internal
  3.30% qemu-system-x86_64 [.] compute_all_subw
  3.15% qemu-system-x86_64 [.] check_regs
  2.56% qemu-system-x86_64 [.] tb_find_fast
  2.34% qemu-system-x86_64 [.] tb_find_slow
  2.16% qemu-system-x86_64 [.] cpu_x86_handle_mmu_fault
  2.05% qemu-system-x86_64 [.] address_space_lookup_region
  1.66% qemu-system-x86_64 [.] cpu_x86_exec
  1.55% qemu-system-x86_64 [.] address_space_translate
  1.54% qemu-system-x86_64 [.] lshift
  1.29% qemu-system-x86_64 [.] int128_make64
  1.27% qemu-system-x86_64 [.] helper_cc_compute_all
  1.23% qemu-system-x86_64 [.] memory_region_is_ram
  1.23% qemu-system-x86_64 [.] int128_sub
  1.22% qemu-system-x86_64 [.] cpu_get_tb_cpu_state
  1.19% qemu-system-x86_64 [.] lduw_p
  1.17% qemu-system-x86_64 [.] tcg_constant_folding
  1.16% qemu-system-x86_64 [.] ldq_phys_internal
  1.09% qemu-system-x86_64 [.] int128_min
  1.08% qemu-system-x86_64 [.] tlb_set_page
  1.02% qemu-system-x86_64 [.] ldq_p
  0.90% qemu-system-x86_64 [.] tb_jmp_cache_hash_func
  0.79% qemu-system-x86_64 [.] cpu_tb_exec
  0.77% qemu-system-x86_64 [.] qemu_get_ram_ptr
  0.75% qemu-system-x86_64 [.] helper_ret_lduw_mmu
  0.75% qemu-system-x86_64 [.] qemu_get_ram_block
  0.67% qemu-system-x86_64 [.] tcg_liveness_analysis
  0.67% qemu-system-x86_64 [.] int128_get64
  0.66% qemu-system-x86_64 [.] tcg_reg_alloc_op
  0.66% qemu-system-x86_64 [.] reset_all_temps
  0.64% qemu-system-x86_64 [.] int128_ge
  0.61% qemu-system-x86_64 [.] tcg_out_opc
  0.59% qemu-system-x86_64 [.] qemu_loglevel_mask
  0.54% qemu-system-x86_64 [.] int128_le
  0.53% qemu-system-x86_64 [.] tcg_out8
  0.52% qemu-system-x86_64 [.] ldq_le_p
  0.48% qemu-system-x86_64 [.] xen_enabled
  0.47% qemu-system-x86_64 [.] ldq_phys
  0.46% qemu-system-x86_64 [.] memory_access_is_direct
  0.44% qemu-system-x86_64 [.] memory_region_section_get_iotlb
  0.44% qemu-system-x86_64 [.] memory_region_get_ram_addr
  0.44% qemu-system-x86_64 [.] tcg_gen_code_common
  0.43% qemu-system-x86_64 [.] cpu_physical_memory_get_dirty_flags

in most time, the funcation tb_find_slow used most cpu resource. and i used gdb print the stack.

(gdb) bt
#0 0x00007f54128cff5d in tb_find_slow (env=0x7f5413a1d0c0, pc=140192155492023, cs_base=0, flags=4246199) at /home/chenlidong/develop/qemu/cpu-exec.c:124
#1 0x00007f54128d0199 in tb_find_fast (env=0x7f5413a1d0c0) at /home/chenlidong/develop/qemu/cpu-exec.c:172
#2 0x00007f54128d0950 in cpu_x86_exec (env=0x7f5413a1d0c0) at /home/chenlidong/develop/qemu/cpu-exec.c:599
#3 0x00007f54128d39d8 in tcg_cpu_exec (env=0x7f5413a1d0c0) at /home/chenlidong/develop/qemu/cpus.c:1183
#4 0x00007f54128d3aef in tcg_exec_all () at /home/chenlidong/develop/qemu/cpus.c:1216
#5 0x00007f54128d2ebc in qemu_tcg_cpu_thread_fn (arg=0x7f5413a1cf80) at /home/chenlidong/develop/qemu/cpus.c:882
#6 0x00007f541047de0f in start_thread () from /lib64/libpthread.so.0
#7 0x00007f54101b17dd in clone () from /lib64/libc.so.6

[chenlidong (chen-lidong-kernel)]

i used the old version of qemu. when the vm is booting, the cpu is still 100%.
but the time of guest os boot is 2min50seconds. the latest upstream is7min21seconds.
so i think this is a performance problem.

the old version i used:
chenlidong@linux-0rsg:~> qemu-kvm --version
QEMU emulator version 1.3.1 (kvm-1.3.1-3.6.2), Copyright (c) 2003-2008 Fabrice Bellard

there are so many modification between the two version.
i will test the other version to find out which modification cause this problem.

[chenlidong (chen-lidong-kernel)]

i find out the reason.

because i used the --enable-debug option for the latest upstream.

so this is not a bug.

[Stefan Hajnoczi (stefanha)]

On Tue, Oct 08, 2013 at 11:51:19AM -0000, chenlidong wrote:
> chenlidong@linux-0rsg:~/develop/qemu> ps -ef | grep qemu
> root 19030 1 14 19:00 ? 00:04:24 /usr/local/bin/qemu-system-x86_64 -name rhel6 -S -M pc-i440fx-1.6 -m 2048 -smp 1,sockets=1,cores=1,threads=1 -uuid 1925a96a-54b9-3c4a-dda0-6b42fdd0af2c -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/rhel6.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/rhel6.img,if=none,id=drive-ide0-0-0,format=raw,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=21,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:b3:b8:53,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
>
> i used perf top, the result is below:
> Samples: 57K of event 'cycles', Event count (approx.): 26336443124
> 15.38% perf-22465.map [.] 0x00007f143b99c9c6
> 5.90% qemu-system-x86_64 [.] phys_page_find
> 4.48% qemu-system-x86_64 [.] address_space_translate_internal
> 3.30% qemu-system-x86_64 [.] compute_all_subw
> 3.15% qemu-system-x86_64 [.] check_regs
> 2.56% qemu-system-x86_64 [.] tb_find_fast
> 2.34% qemu-system-x86_64 [.] tb_find_slow

KVM is disabled - you are not using hardware virtualization extensions.
Add -enable-kvm to the command-line or make sure libvirt is using "kvm"
mode.

Stefan