Confirm on Ubuntu 9.10 under KVM virtualisation.
silenkov@v_mysql03:~$ uname -a Linux v_mysql06 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 GNU/Linux
Kern.log output:
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] BUG: soft lockup - CPU#2 stuck for 61s! [mysqld:17040] Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Modules linked in: ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs iptable_filter ip_tables x_tables i2c_piix4 psmouse lp parport serio_raw virtio_balloon virtio_net virtio_blk floppy virtio_pci virtio_ring virtio Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] CPU 2: Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Modules linked in: ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs iptable_filter ip_tables x_tables i2c_piix4 psmouse lp parport serio_raw virtio_balloon virtio_net virtio_blk floppy virtio_pci virtio_ring virtio Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Pid: 17040, comm: mysqld Not tainted 2.6.31-14-generic #48-Ubuntu Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RIP: 0010:[<ffffffff81035257>] [<ffffffff81035257>] kvm_mmu_op+0x47/0x70 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RSP: 0018:ffff88007c10fc18 EFLAGS: 00000297 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RAX: 0000000000000000 RBX: ffff88007c10fc48 RCX: 0000000001a3e5b0 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RDX: 0000000000000000 RSI: 0000000000000020 RDI: ffff880001a3e5b0 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RBP: ffffffff81012bae R08: 00000000ffffff77 R09: 0000000000000000 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88007c10fc48 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] R13: ffffffff81012bae R14: 0000000000000000 R15: 0000000000000002 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] FS: 00007fc401254910(0000) GS:ffff880001a30000(0000) knlGS:0000000000000000 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] CR2: 00007fc3e04351d8 CR3: 000000005acd1000 CR4: 00000000000006e0 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Call Trace: Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff8103524b>] ? kvm_mmu_op+0x3b/0x70 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810352b4>] ? kvm_leave_lazy_mmu+0x34/0x50 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f5c07>] ? zap_pte_range+0x187/0x400 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810e5344>] ? release_pages+0x234/0x260 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810dba70>] ? T.768+0x3a0/0x440 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f6040>] ? unmap_page_range+0x1c0/0x290 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f6277>] ? unmap_vmas+0x167/0x2f0 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f75e1>] ? zap_page_range+0xb1/0x120 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f35e7>] ? madvise_vma+0x1a7/0x1f0 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f375d>] ? sys_madvise+0x12d/0x270 Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff81012002>] ? system_call_fastpath+0x16/0x1b
root@v_mysql03:~# free total used free shared buffers cached Mem: 3041464 645500 2395964 0 59464 163000 -/+ buffers/cache: 423036 2618428 Swap: 4194296 0 4194296
Couldn't reproduce a bug. It appears randomly without interaction of server load. Memory and CPU's are ok.
Confirm on Ubuntu 9.10 under KVM virtualisation.
silenkov@ v_mysql03: ~$ uname -a
Linux v_mysql06 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 GNU/Linux
Kern.log output:
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] BUG: soft lockup - CPU#2 stuck for 61s! [mysqld:17040] ffffffff8103525 7>] [<ffffffff81035 257>] kvm_mmu_ op+0x47/ 0x70 10fc18 EFLAGS: 00000297 0(0000) GS:ffff880001a3 0000(0000) knlGS:000000000 0000000 24b>] ? kvm_mmu_ op+0x3b/ 0x70 2b4>] ? kvm_leave_ lazy_mmu+ 0x34/0x50 c07>] ? zap_pte_ range+0x187/ 0x400 344>] ? release_ pages+0x234/ 0x260 a70>] ? T.768+0x3a0/0x440 040>] ? unmap_page_ range+0x1c0/ 0x290 277>] ? unmap_vmas+ 0x167/0x2f0 5e1>] ? zap_page_ range+0xb1/ 0x120 5e7>] ? madvise_ vma+0x1a7/ 0x1f0 75d>] ? sys_madvise+ 0x12d/0x270 002>] ? system_ call_fastpath+ 0x16/0x1b
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Modules linked in: ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs iptable_filter ip_tables x_tables i2c_piix4 psmouse lp parport serio_raw virtio_balloon virtio_net virtio_blk floppy virtio_pci virtio_ring virtio
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] CPU 2:
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Modules linked in: ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs iptable_filter ip_tables x_tables i2c_piix4 psmouse lp parport serio_raw virtio_balloon virtio_net virtio_blk floppy virtio_pci virtio_ring virtio
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Pid: 17040, comm: mysqld Not tainted 2.6.31-14-generic #48-Ubuntu
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RIP: 0010:[<
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RSP: 0018:ffff88007c
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RAX: 0000000000000000 RBX: ffff88007c10fc48 RCX: 0000000001a3e5b0
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RDX: 0000000000000000 RSI: 0000000000000020 RDI: ffff880001a3e5b0
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] RBP: ffffffff81012bae R08: 00000000ffffff77 R09: 0000000000000000
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88007c10fc48
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] R13: ffffffff81012bae R14: 0000000000000000 R15: 0000000000000002
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] FS: 00007fc40125491
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] CR2: 00007fc3e04351d8 CR3: 000000005acd1000 CR4: 00000000000006e0
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] Call Trace:
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff81035
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff81035
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f5
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810e5
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810db
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f6
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f6
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f7
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f3
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff810f3
Mar 3 05:32:51 v_mysql03 kernel: [217826.790638] [<ffffffff81012
root@v_mysql03:~# free
total used free shared buffers cached
Mem: 3041464 645500 2395964 0 59464 163000
-/+ buffers/cache: 423036 2618428
Swap: 4194296 0 4194296
Couldn't reproduce a bug. It appears randomly without interaction of server load. Memory and CPU's are ok.