kvm - BUG: Soft lookup - CPU#2 stuck for 225s [swapper:0]

[Martin]

Binary package hint: qemu-kvm

I'm very frustrated with ubuntu and kvm / qemu.
I use the newest version (kermic) and all updates are installed.

With the last kernel I got ext4 errors in my vms (kermic too).
This looks fixed with the new kernel update in the last days, but I get still CPU stuck error messages like:

[2690.373130] BUG: Soft lookup - CPU#2 stuck for 225s [swapper:0]

I get the same error messages every 4-8 hours and the webserver and mailserver get stuck. The machine is still online and running (login prompt, shell and ssh over bridge virtio). The process (swapper then apache then other services) changes each time. I want to use ubuntu with kvm in a production env.
I use a INTEL P55 Motherboard (newest BIOS) with 8GB DDR3 (checked with memtest) and 4 SATA2 with RAID1 and LVM.
The VMs are on a LVM too.

Complete kern.log:
---
Jan 20 17:20:10 kvm-1 kernel: [145236.212262] BUG: soft lockup - CPU#2 stuck for 219s! [kvm:5601]
Jan 20 17:20:10 kvm-1 kernel: [145236.212303] Modules linked in: tun ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp kvm_intel kvm bridge stp iptable_filter ip_tables x_tables lp parport usb_storage raid10 raid456 raid6_pq async_xor async_memcpy async_tx xor raid1 raid0 multipath linear e1000e
Jan 20 17:20:10 kvm-1 kernel: [145236.212319] CPU 2:
Jan 20 17:20:10 kvm-1 kernel: [145236.212320] Modules linked in: tun ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT xt_tcpudp kvm_intel kvm bridge stp iptable_filter ip_tables x_tables lp parport usb_storage raid10 raid456 raid6_pq async_xor async_memcpy async_tx xor raid1 raid0 multipath linear e1000e
Jan 20 17:20:10 kvm-1 kernel: [145236.212334] Pid: 5601, comm: kvm Not tainted 2.6.31-17-server #54-Ubuntu
Jan 20 17:20:10 kvm-1 kernel: [145236.212336] RIP: 0010:[<ffffffffa0105a6a>] [<ffffffffa0105a6a>] vcpu_enter_guest+0x26a/0x5c0 [kvm]
Jan 20 17:20:10 kvm-1 kernel: [145236.212348] RSP: 0018:ffff8801d610fd28 EFLAGS: 00000202
Jan 20 17:20:10 kvm-1 kernel: [145236.212350] RAX: 0000000000000001 RBX: ffff8801d610fd88 RCX: ffff880223c0a880
Jan 20 17:20:10 kvm-1 kernel: [145236.212351] RDX: 0000000000000000 RSI: 0000000000000400 RDI: 0000000000000007
Jan 20 17:20:10 kvm-1 kernel: [145236.212353] RBP: ffffffff810129ce R08: ffff880078231808 R09: 0000000000000000
Jan 20 17:20:10 kvm-1 kernel: [145236.212355] R10: 0000000000000000 R11: 00000000000e7ef0 R12: ffffffffa0130791
Jan 20 17:20:10 kvm-1 kernel: [145236.212356] R13: ffff8801d610fca8 R14: 0000000000000000 R15: 000000000000002c
Jan 20 17:20:10 kvm-1 kernel: [145236.212358] FS: 00007fd6b0478910(0000) GS:ffff880028070000(0000) knlGS:00000000f6f1b6c0
Jan 20 17:20:10 kvm-1 kernel: [145236.212360] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b
Jan 20 17:20:10 kvm-1 kernel: [145236.212361] CR2: 0000000009f96468 CR3: 000000014a7f9000 CR4: 00000000000026f0
Jan 20 17:20:10 kvm-1 kernel: [145236.212363] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jan 20 17:20:10 kvm-1 kernel: [145236.212364] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jan 20 17:20:10 kvm-1 kernel: [145236.212366] Call Trace:
Jan 20 17:20:10 kvm-1 kernel: [145236.212374] [<ffffffffa0105a5c>] ? vcpu_enter_guest+0x25c/0x5c0 [kvm]
Jan 20 17:20:10 kvm-1 kernel: [145236.212381] [<ffffffffa0118397>] ? apic_has_pending_timer+0x37/0x50 [kvm]
Jan 20 17:20:10 kvm-1 kernel: [145236.212389] [<ffffffffa0105e23>] ? __vcpu_run+0x63/0x320 [kvm]
Jan 20 17:20:10 kvm-1 kernel: [145236.212396] [<ffffffffa010bdb2>] ? kvm_arch_vcpu_ioctl_run+0x82/0x1c0 [kvm]
Jan 20 17:20:10 kvm-1 kernel: [145236.212403] [<ffffffffa01015e3>] ? kvm_vcpu_ioctl+0x473/0x5c0 [kvm]
Jan 20 17:20:10 kvm-1 kernel: [145236.212408] [<ffffffff812768d6>] ? rb_erase+0xd6/0x160
Jan 20 17:20:10 kvm-1 kernel: [145236.212411] [<ffffffff8112da1d>] ? vfs_ioctl+0x1d/0xa0
Jan 20 17:20:10 kvm-1 kernel: [145236.212415] [<ffffffff81077441>] ? common_timer_set+0x111/0x1a0
Jan 20 17:20:10 kvm-1 kernel: [145236.212417] [<ffffffff8112dba9>] ? do_vfs_ioctl+0x79/0x400
Jan 20 17:20:10 kvm-1 kernel: [145236.212419] [<ffffffff8112dfb1>] ? sys_ioctl+0x81/0xa0
Jan 20 17:20:10 kvm-1 kernel: [145236.212423] [<ffffffff81011fc2>] ? system_call_fastpath+0x16/0x1b
Jan 20 17:21:32 kvm-1 kernel: [145317.476878] br0: port 3(vnet1) entering disabled state
Jan 20 17:21:32 kvm-1 kernel: [145317.516399] device vnet1 left promiscuous mode
Jan 20 17:21:32 kvm-1 kernel: [145317.516402] br0: port 3(vnet1) entering disabled state
Jan 20 17:21:32 kvm-1 kernel: [145317.963533] type=1505 audit(1264004492.698:24): operation="profile_remove" pid=15738 name=libvirt-09bb7ec1-e97e-49f0-bcdf-4eef76b1ccc6 namespace=default
Jan 20 17:22:11 kvm-1 kernel: [145356.951674] type=1505 audit(1264004531.688:25): operation="profile_load" pid=15751 name=libvirt-09bb7ec1-e97e-49f0-bcdf-4eef76b1ccc6
Jan 20 17:22:11 kvm-1 kernel: [145356.977138] device vnet1 entered promiscuous mode
Jan 20 17:22:11 kvm-1 kernel: [145356.978395] br0: port 3(vnet1) entering forwarding state
Jan 20 17:22:21 kvm-1 kernel: [145367.009748] vnet1: no IPv6 routers present
---

I'm thinking about to switch to the newest beta release (11.04 LTS) which has the newest kernel and the newest kvm packages.

[Jeremy Foshee (jeremyfoshee)]

Hi Martin,

This bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? Can you try with the latest development release of Ubuntu? ISO CD images are available from http://cdimage.ubuntu.com/releases/ .

If it remains an issue, could you run the following command from a Terminal (Applications->Accessories->Terminal). It will automatically gather and attach updated debug information to this report.

apport-collect -p linux 510207

Also, if you could test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

    [This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]

[Jeremy Foshee (jeremyfoshee)]

This bug report was marked as Incomplete and has not had any updated comments for quite some time. As a result this bug is being closed. Please reopen if this is still an issue in the current Ubuntu release http://www.ubuntu.com/getubuntu/download . Also, please be sure to provide any requested information that may have been missing. To reopen the bug, click on the current status under the Status column and change the status back to "New". Thanks.

[This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.]