ubuntu14.04(3.13.0-37.64),kernel crash (mapcount 0 page_mapcount 1)

[gaoliang]

 Bug Description:

this issue（mapcount 0 page_mapcount 1） has already been resolved by 3.13.0-36, but this bug arised.
memory usage is close to 100 percent.

Here is I believe the first record in syslog when it starts:

Jul 3 09:50:08 GMQ kernel: [2764507.350057] sd 1:0:0:22: [sdd] Test WP failed, assume Write Enabled
Jul 3 09:50:08 GMQ kernel: [2764507.350210] sd 1:0:0:22: [sdd] Asking for cache data failed
Jul 3 09:50:08 GMQ kernel: [2764507.350220] sd 1:0:0:22: [sdd] Assuming drive cache: write through
Jul 3 09:50:08 GMQ kernel: [2764507.654448] mapcount 0 page_mapcount 1
Jul 3 09:50:08 GMQ kernel: [2764507.654556] ------------[ cut here ]------------
Jul 3 09:50:08 GMQ kernel: [2764507.654561] Kernel BUG at ffffffff811b23bc [verbose debug info unavailable]
Jul 3 09:50:08 GMQ kernel: [2764507.654566] invalid opcode: 0000 [#1] SMP
Jul 3 09:50:08 GMQ kernel: [2764507.654570] Modules linked in: ip6table_filter(F) ip6_tables(F) iptable_filter(F) i]
Jul 3 09:50:08 GMQ kernel: [2764507.654663] CPU: 25 PID: 613 Comm: kswapd3 Tainted: GF O 3.13. #1
Jul 3 09:50:08 GMQ kernel: [2764507.654667] Hardware name: H3C FlexServer R590, BIOS P77 12/20/2013
Jul 3 09:50:08 GMQ kernel: [2764507.654670] task: ffff880fe7c8c830 ti: ffff880fe6c7e000 task.ti: fff880fe6c7e000
Jul 3 09:50:08 GMQ kernel: [2764507.654674] RIP: 0010:[<ffffffff811b23bc>] [<ffffffff811b23bc>] split_huge_page_to_list+0x2dc/0x7b0
Jul 3 09:50:08 GMQ kernel: [2764507.654689] RSP: 0018:ffff880fe6c7f858 EFLAGS: 00010297
Jul 3 09:50:08 GMQ kernel: [2764507.654692] RAX: 0000000000000001 RBX: ffff883fde776f18 RCX: 0000000000000000
Jul 3 09:50:08 GMQ kernel: [2764507.654695] RDX: ffff88403f82fff0 RSI: ffff88403f82e3d8 RDI: 0000000000000246
Jul 3 09:50:08 GMQ kernel: [2764507.654699] RBP: ffff880fe6c7f918 R08: 0000000000000000 R09: 0000000000000000
Jul 3 09:50:08 GMQ kernel: [2764507.654702] R10: 00000000001eb0af R11: 0e200e200e200e20 R12: 0000000000000000
Jul 3 09:50:08 GMQ kernel: [2764507.654706] R13: ffff881fe681aa00 R14: ffffea00f4ae8000 R15: ffffea00f4ae8000
Jul 3 09:50:08 GMQ kernel: [2764507.654710] FS: 0000000000000000(0000) GS:ffff88403f820000(0000) knlGS:0000000000000000
Jul 3 09:50:08 GMQ kernel: [2764507.654713] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 3 09:50:08 GMQ kernel: [2764507.654716] CR2: 000000003bfc2000 CR3: 0000000001c0d000 CR4: 00000000000427e0
Jul 3 09:50:08 GMQ kernel: [2764507.654720] DR0: 0000000000000003 DR1: 00000000000000b0 DR2: 0000000000000001
Jul 3 09:50:08 GMQ kernel: [2764507.654724] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jul 3 09:50:08 GMQ kernel: [2764507.654726] Stack:
Jul 3 09:50:08 GMQ kernel: [2764507.654729] ffffea00f4ae8000 ffff880fe5bc6980 ffff880f00000000 ffffffff81193bba
Jul 3 09:50:08 GMQ kernel: [2764507.654741] ffff883fe75c6300 00000007f4474000 0000000000042541 0000000000042541
Jul 3 09:50:08 GMQ kernel: [2764507.654748] 00000000e6c7f8f8 0000000000000000 00000007f4534200 ffff883fe75c6330
Jul 3 09:50:08 GMQ kernel: [2764507.654756] Call Trace:
Jul 3 09:50:08 GMQ kernel: [2764507.654778] [<ffffffff81193bba>] ? scan_swap_map+0x37a/0x510
Jul 3 09:50:08 GMQ kernel: [2764507.654783] [<ffffffff81193faa>] ? get_swap_page+0x1aa/0x210
Jul 3 09:50:08 GMQ kernel: [2764507.654792] [<ffffffff81189df0>] ? page_mkclean_one+0xd0/0xd0
Jul 3 09:50:08 GMQ kernel: [2764507.654798] [<ffffffff8119190a>] add_to_swap+0x8a/0xa0
Jul 3 09:50:08 GMQ kernel: [2764507.654811] [<ffffffff811674fb>] shrink_page_list+0x5db/0xa30
Jul 3 09:50:08 GMQ kernel: [2764507.654826] [<ffffffff8116804a>] shrink_inactive_list+0x1ea/0x5b0
Jul 3 09:50:08 GMQ kernel: [2764507.654832] [<ffffffff81168b5c>] shrink_lruvec+0x35c/0x6e0
Jul 3 09:50:08 GMQ kernel: [2764507.654838] [<ffffffff81168f46>] shrink_zone+0x66/0x1a0
Jul 3 09:50:08 GMQ kernel: [2764507.654843] [<ffffffff8116a11e>] balance_pgdat+0x3be/0x620
Jul 3 09:50:08 GMQ kernel: [2764507.654854] [<ffffffff810760d2>] ? del_timer_sync+0x52/0x60
Jul 3 09:50:08 GMQ kernel: [2764507.654859] [<ffffffff8116a4f7>] kswapd+0x177/0x4a0
Jul 3 09:50:08 GMQ kernel: [2764507.654870] [<ffffffff810adb20>] ? __wake_up_sync+0x20/0x20
Jul 3 09:50:08 GMQ kernel: [2764507.654875] [<ffffffff8116a380>] ? balance_pgdat+0x620/0x620
Jul 3 09:50:08 GMQ kernel: [2764507.654883] [<ffffffff8108d079>] kthread+0xc9/0xe0
Jul 3 09:50:08 GMQ kernel: [2764507.654889] [<ffffffff8108cfb0>] ? flush_kthread_worker+0xb0/0xb0
Jul 3 09:50:08 GMQ kernel: [2764507.654900] [<ffffffff81760ffc>] ret_from_fork+0x7c/0xb0
Jul 3 09:50:08 GMQ kernel: [2764507.654905] [<ffffffff8108cfb0>] ? flush_kthread_worker+0xb0/0xb0
Jul 3 09:50:08 GMQ kernel: [2764507.654908] Code: df 48 89 43 08 4c 89 e8 49 03 46 10 48 89 43 10 41 0f b7 76 05 e8 f5 f3 fb ff f6 43 08 01 74 0b 48 8b 03 a8 08 0f 85 34 ff ff ff <0f> 0b f3 90 49 8b 06 a9 00 00 00 01 75 f4 e9 f8 fe ff ff 90 f0
Jul 3 09:50:08 GMQ kernel: [2764507.654946] RIP [<ffffffff811b23bc>] split_huge_page_to_list+0x2dc/0x7b0
Jul 3 09:50:08 GMQ kernel: [2764507.654950] RSP <ffff880fe6c7f858>
Jul 3 09:50:08 GMQ kernel: [2764507.725793] ---[ end trace 0c60ca2f2aae4c6c ]---
Jul 3 09:50:09 GMQ kernel: [2764507.742363] ------------[ cut here ]------------
Jul 3 09:50:09 GMQ kernel: [2764507.742380] WARNING: CPU: 25 PID: 613 at kernel/exit.c:704 do_exit+0x55/0xa70()
Jul 3 09:50:09 GMQ kernel: [2764507.742384] Modules linked in: ip6table_filter(F) ip6_tables(F) iptable_filter(F) ip_tables(F) ebtable_nat(F) nbd(F) [last unloaded: ipmi_si]
Jul 3 09:50:09 GMQ kernel: [2764507.742495] CPU: 25 PID: 613 Comm: kswapd3 Tainted: GF D O 3.13.6 #1
Jul 3 09:50:09 GMQ kernel: [2764507.742499] Hardware name: H3C FlexServer R590, BIOS P77 12/20/2013
Jul 3 09:50:09 GMQ kernel: [2764507.742503] 00000000000002c0 ffff880fe6c7f548 ffffffff81750690 0000000000000007
Jul 3 09:50:09 GMQ kernel: [2764507.742513] 0000000000000000 ffff880fe6c7f588 ffffffff81068c6c 000000000000000b
Jul 3 09:50:09 GMQ kernel: [2764507.742521] ffff880fe7c8c830 000000000000000b ffff880fe6c7f7a8 0000000000000000
Jul 3 09:50:09 GMQ kernel: [2764507.742531] Call Trace:
Jul 3 09:50:09 GMQ kernel: [2764507.742547] [<ffffffff81750690>] dump_stack+0x46/0x58
Jul 3 09:50:09 GMQ kernel: [2764507.742553] [<ffffffff81068c6c>] warn_slowpath_common+0x8c/0xc0
Jul 3 09:50:09 GMQ kernel: [2764507.742558] [<ffffffff81068cba>] warn_slowpath_null+0x1a/0x20
Jul 3 09:50:09 GMQ kernel: [2764507.742563] [<ffffffff8106aff5>] do_exit+0x55/0xa70
Jul 3 09:50:09 GMQ kernel: [2764507.742584] [<ffffffff810c13dc>] ? kmsg_dump+0x9c/0xc0
Jul 3 09:50:09 GMQ kernel: [2764507.742595] [<ffffffff817599b8>] oops_end+0xb8/0x160
Jul 3 09:50:09 GMQ kernel: [2764507.742605] [<ffffffff810183e8>] die+0x58/0x90
Jul 3 09:50:09 GMQ kernel: [2764507.742610] [<ffffffff8175914b>] do_trap+0xcb/0x170
Jul 3 09:50:09 GMQ kernel: [2764507.742616] [<ffffffff8101557c>] do_invalid_op+0xac/0x110
Jul 3 09:50:09 GMQ kernel: [2764507.742625] [<ffffffff811b23bc>] ? split_huge_page_to_list+0x2dc/0x7b0
Jul 3 09:50:09 GMQ kernel: [2764507.742631] [<ffffffff810bfa04>] ? wake_up_klogd+0x34/0x40
Jul 3 09:50:09 GMQ kernel: [2764507.742636] [<ffffffff810bfca5>] ? console_unlock+0x295/0x3e0
Jul 3 09:50:09 GMQ kernel: [2764507.742643] [<ffffffff817628de>] invalid_op+0x1e/0x30
Jul 3 09:50:09 GMQ kernel: [2764507.742648] [<ffffffff811b23bc>] ? split_huge_page_to_list+0x2dc/0x7b0
Jul 3 09:50:09 GMQ kernel: [2764507.742652] [<ffffffff811b2862>] ? split_huge_page_to_list+0x782/0x7b0
Jul 3 09:50:09 GMQ kernel: [2764507.742662] [<ffffffff81193bba>] ? scan_swap_map+0x37a/0x510
Jul 3 09:50:09 GMQ kernel: [2764507.742667] [<ffffffff81193faa>] ? get_swap_page+0x1aa/0x210
Jul 3 09:50:09 GMQ kernel: [2764507.742676] [<ffffffff81189df0>] ? page_mkclean_one+0xd0/0xd0
Jul 3 09:50:09 GMQ kernel: [2764507.742681] [<ffffffff8119190a>] add_to_swap+0x8a/0xa0
Jul 3 09:50:09 GMQ kernel: [2764507.742703] [<ffffffff811674fb>] shrink_page_list+0x5db/0xa30
Jul 3 09:50:09 GMQ kernel: [2764507.742710] [<ffffffff8116804a>] shrink_inactive_list+0x1ea/0x5b0
Jul 3 09:50:09 GMQ kernel: [2764507.742715] [<ffffffff81168b5c>] shrink_lruvec+0x35c/0x6e0
Jul 3 09:50:09 GMQ kernel: [2764507.742722] [<ffffffff81168f46>] shrink_zone+0x66/0x1a0
Jul 3 09:50:09 GMQ kernel: [2764507.742728] [<ffffffff8116a11e>] balance_pgdat+0x3be/0x620
Jul 3 09:50:09 GMQ kernel: [2764507.742735] [<ffffffff810760d2>] ? del_timer_sync+0x52/0x60
Jul 3 09:50:09 GMQ kernel: [2764507.742741] [<ffffffff8116a4f7>] kswapd+0x177/0x4a0
Jul 3 09:50:09 GMQ kernel: [2764507.742751] [<ffffffff810adb20>] ? __wake_up_sync+0x20/0x20
Jul 3 09:50:09 GMQ kernel: [2764507.742756] [<ffffffff8116a380>] ? balance_pgdat+0x620/0x620
Jul 3 09:50:09 GMQ kernel: [2764507.742767] [<ffffffff8108d079>] kthread+0xc9/0xe0
Jul 3 09:50:09 GMQ kernel: [2764507.742771] [<ffffffff8108cfb0>] ? flush_kthread_worker+0xb0/0xb0
Jul 3 09:50:09 GMQ kernel: [2764507.742777] [<ffffffff81760ffc>] ret_from_fork+0x7c/0xb0
Jul 3 09:50:09 GMQ kernel: [2764507.742781] [<ffffffff8108cfb0>] ? flush_kthread_worker+0xb0/0xb0
Jul 3 09:50:09 GMQ kernel: [2764507.742784] ---[ end trace 0c60ca2f2aae4c6d ]---

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1488049

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[gaoliang (gaoliang0926)]

hey

sorry,not able to connect to Internet.

memory usage is close to 100 percent.There are many running kvm virtual machines then.

mem：256G
cpu： 24

[gaoliang (gaoliang0926)]

hey

sorry,not able to connect to Internet.

memory usage is close to 100 percent.There are many running kvm virtual machines then.

mem：256G
cpu： 24

[gaoliang (gaoliang0926)]

sorry,not able to connect to Internet. not able to run 'apport-collect 1488049' command.

[penalvch (penalvch)]

gaoliang, thank you for reporting this and helping make Ubuntu better. Are you able to connect to the internet when booted back into the working 3.13.0-36 kernel, or are you saying you are presently not able to connect the server to the internet?

Also, could you please test the latest upstream kernel available from the very top line at the top of the page from http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=N;O=D (the release names are irrelevant for testing, and please do not test the daily folder)? Install instructions are available at https://wiki.ubuntu.com/Kernel/MainlineBuilds . This will allow additional upstream developers to examine the issue.

If the latest kernel did not allow you to test to the issue (ex. you couldn't boot into the OS) please make a comment in your report about this, and continue to test the next most recent kernel version until you can test to the issue. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested. If this issue is fixed in the mainline kernel, please add the following tags by clicking on the yellow circle with a black pencil icon, next to the word Tags, located at the bottom of the report description:
kernel-fixed-upstream
kernel-fixed-upstream-X.Y-rcZ

Where X, Y, and Z are numbers corresponding to the kernel version.

If the mainline kernel does not fix the issue, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-X.Y-rcZ

Please note, an error to install the kernel does not fit the criteria of kernel-bug-exists-upstream.

Once testing of the latest upstream kernel is complete, please mark this report's Status as Confirmed. Please let us know your results.

Thank you for your understanding.

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]