Mar 20 14:13:10 i-deadbeef kernel: [501130.316238] kthreadd invoked oom-killer: gfp_mask=0x26000c0, order=2, oom_score_adj=0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316242] kthreadd cpuset=/ mems_allowed=0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316247] CPU: 4 PID: 2 Comm: kthreadd Not tainted 4.4.0-66-generic #87-Ubuntu Mar 20 14:13:10 i-deadbeef kernel: [501130.316249] Hardware name: Xen HVM domU, BIOS 4.2.amazon 12/12/2016 Mar 20 14:13:10 i-deadbeef kernel: [501130.316250] 0000000000000286 0000000052170f2b ffff880f7b447ab8 ffffffff813f8083 Mar 20 14:13:10 i-deadbeef kernel: [501130.316253] ffff880f7b447c70 ffff880149997000 ffff880f7b447b28 ffffffff8120b0fe Mar 20 14:13:10 i-deadbeef kernel: [501130.316255] ffffffff81cd63bf 0000000000000000 ffffffff81e677e0 0000000000000206 Mar 20 14:13:10 i-deadbeef kernel: [501130.316257] Call Trace: Mar 20 14:13:10 i-deadbeef kernel: [501130.316264] [] dump_stack+0x63/0x90 Mar 20 14:13:10 i-deadbeef kernel: [501130.316268] [] dump_header+0x5a/0x1c5 Mar 20 14:13:10 i-deadbeef kernel: [501130.316272] [] oom_kill_process+0x202/0x3c0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316273] [] out_of_memory+0x219/0x460 Mar 20 14:13:10 i-deadbeef kernel: [501130.316276] [] __alloc_pages_slowpath.constprop.88+0x938/0xad0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316279] [] __alloc_pages_nodemask+0x286/0x2a0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316281] [] alloc_kmem_pages_node+0x4b/0xc0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316285] [] copy_process+0x1be/0x1b70 Mar 20 14:13:10 i-deadbeef kernel: [501130.316288] [] ? enqueue_task_fair+0xaa/0x8b0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316291] [] ? kthread_create_on_node+0x1e0/0x1e0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316293] [] _do_fork+0x80/0x360 Mar 20 14:13:10 i-deadbeef kernel: [501130.316297] [] ? __schedule+0x3b6/0xa30 Mar 20 14:13:10 i-deadbeef kernel: [501130.316299] [] kernel_thread+0x29/0x30 Mar 20 14:13:10 i-deadbeef kernel: [501130.316301] [] kthreadd+0x158/0x1a0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316303] [] ? kthread_create_on_cpu+0x60/0x60 Mar 20 14:13:10 i-deadbeef kernel: [501130.316305] [] ret_from_fork+0x3f/0x70 Mar 20 14:13:10 i-deadbeef kernel: [501130.316307] [] ? kthread_create_on_cpu+0x60/0x60 Mar 20 14:13:10 i-deadbeef kernel: [501130.316308] Mem-Info: Mar 20 14:13:10 i-deadbeef kernel: [501130.316312] active_anon:244727 inactive_anon:63643 isolated_anon:0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316312] active_file:3503535 inactive_file:3454203 isolated_file:0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316312] unevictable:7912588 dirty:91461 writeback:946 unstable:0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316312] slab_reclaimable:344660 slab_unreclaimable:11722 Mar 20 14:13:10 i-deadbeef kernel: [501130.316312] mapped:2769810 shmem:164018 pagetables:59669 bounce:0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316312] free:84252 free_pcp:13 free_cma:0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316315] Node 0 DMA free:15904kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0k B unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unrecla imable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Mar 20 14:13:10 i-deadbeef kernel: [501130.316319] lowmem_reserve[]: 0 1935 61373 61373 61373 Mar 20 14:13:10 i-deadbeef kernel: [501130.316322] Node 0 DMA32 free:253676kB min:2128kB low:2660kB high:3192kB active_anon:10932kB inactive_anon:4kB active_file:8kB inactive_file:8kB unevictable:1671168kB isolated(anon):0kB isolated(file):0kB present:2080768kB managed:2000104kB mlocked:1671168kB dirty:0kB writeback:0kB mapped:1412kB shmem:12kB slab_reclaimable:57284kB slab_unreclaimable:820kB kernel_stack:320kB pagetables:4460kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:16728 all_unreclaimable? yes Mar 20 14:13:10 i-deadbeef kernel: [501130.316326] lowmem_reserve[]: 0 0 59437 59437 59437 Mar 20 14:13:10 i-deadbeef kernel: [501130.316329] Node 0 Normal free:67428kB min:65432kB low:81788kB high:98148kB active_anon:967976kB inactive_anon:254568kB active_file:14014132kB inactive_file:13816804kB unevictable:29979184kB isolated(anon):0kB isolated(file):0kB present:61865984kB managed:60863808kB mlocked:29979184kB dirty:365844kB writeback:3784kB mapped:11077828kB shmem:656060kB slab_reclaimable:1321356kB slab_unreclaimable:46068kB kernel_stack:7616kB pagetables:234216kB unstable:0kB bounce:0kB free_pcp:52kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:4864 all_unreclaimable? no Mar 20 14:13:10 i-deadbeef kernel: [501130.316333] lowmem_reserve[]: 0 0 0 0 0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316335] Node 0 DMA: 0*4kB 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15904kB Mar 20 14:13:10 i-deadbeef kernel: [501130.316344] Node 0 DMA32: 517*4kB (UME) 423*8kB (UME) 394*16kB (UME) 260*32kB (ME) 150*64kB (ME) 118*128kB (ME) 64*256kB (ME) 42*512kB (ME) 21*1024kB (M) 7*2048kB (ME) 33*4096kB (UM) = 253676kB Mar 20 14:13:10 i-deadbeef kernel: [501130.316354] Node 0 Normal: 16436*4kB (UM) 0*8kB 31*16kB (H) 19*32kB (H) 3*64kB (H) 3*128kB (H) 1*256kB (H) 0*512kB 1*1024kB (H) 0*2048kB 0*4096kB = 68704kB Mar 20 14:13:10 i-deadbeef kernel: [501130.316363] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB Mar 20 14:13:10 i-deadbeef kernel: [501130.316365] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Mar 20 14:13:10 i-deadbeef kernel: [501130.316365] 7127678 total pagecache pages Mar 20 14:13:10 i-deadbeef kernel: [501130.316367] 0 pages in swap cache Mar 20 14:13:10 i-deadbeef kernel: [501130.316368] Swap cache stats: add 0, delete 0, find 0/0 Mar 20 14:13:10 i-deadbeef kernel: [501130.316369] Free swap = 0kB Mar 20 14:13:10 i-deadbeef kernel: [501130.316370] Total swap = 0kB Mar 20 14:13:10 i-deadbeef kernel: [501130.316371] 15990685 pages RAM Mar 20 14:13:10 i-deadbeef kernel: [501130.316372] 0 pages HighMem/MovableOnly Mar 20 14:13:10 i-deadbeef kernel: [501130.316372] 270731 pages reserved Mar 20 14:13:10 i-deadbeef kernel: [501130.316373] 0 pages cma reserved Mar 20 14:13:10 i-deadbeef kernel: [501130.316374] 0 pages hwpoisoned Mar 20 14:13:10 i-deadbeef kernel: [501130.316375] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name Mar 20 14:13:10 i-deadbeef kernel: [501130.316381] [ 389] 0 389 37585 26848 77 4 0 0 systemd-journal Mar 20 14:13:10 i-deadbeef kernel: [501130.316383] [ 455] 0 455 23693 222 16 3 0 0 lvmetad Mar 20 14:13:10 i-deadbeef kernel: [501130.316385] [ 480] 0 480 9859 786 20 3 0 -1000 systemd-udevd Mar 20 14:13:10 i-deadbeef kernel: [501130.316387] [ 1018] 0 1018 4031 599 11 3 0 0 dhclient Mar 20 14:13:10 i-deadbeef kernel: [501130.316389] [ 1176] 104 1176 64100 790 27 3 0 0 rsyslogd Mar 20 14:13:10 i-deadbeef kernel: [501130.316391] [ 1179] 0 1179 1100 318 8 3 0 0 acpid Mar 20 14:13:10 i-deadbeef kernel: [501130.316392] [ 1182] 0 1182 1099 298 7 3 0 0 runsvdir Mar 20 14:13:10 i-deadbeef kernel: [501130.316394] [ 1184] 0 1184 67585 227 34 3 0 0 accounts-daemon Mar 20 14:13:10 i-deadbeef kernel: [501130.316396] [ 1187] 0 1187 71211 2610 32 5 0 0 snapd Mar 20 14:13:10 i-deadbeef kernel: [501130.316398] [ 1190] 0 1190 5187 720 16 3 0 0 systemd-logind Mar 20 14:13:10 i-deadbeef kernel: [501130.316400] [ 1196] 107 1196 9670 405 25 3 0 -900 dbus-daemon Mar 20 14:13:10 i-deadbeef kernel: [501130.316401] [ 1204] 0 1204 1306 31 8 3 0 0 iscsid Mar 20 14:13:10 i-deadbeef kernel: [501130.316403] [ 1205] 0 1205 1431 879 8 3 0 -17 iscsid Mar 20 14:13:10 i-deadbeef kernel: [501130.316404] [ 1207] 0 1207 1061 149 7 3 0 0 runsv Mar 20 14:13:10 i-deadbeef kernel: [501130.316406] [ 1208] 0 1208 1097 101 8 3 0 0 svlogd Mar 20 14:13:10 i-deadbeef kernel: [501130.316408] [ 1240] 0 1240 58784 1011 17 3 0 0 lxcfs Mar 20 14:13:10 i-deadbeef kernel: [501130.316409] [ 1242] 0 1242 68215 603 37 3 0 0 polkitd Mar 20 14:13:10 i-deadbeef kernel: [501130.316411] [ 1314] 0 1314 1606 126 8 3 0 0 agetty Mar 20 14:13:10 i-deadbeef kernel: [501130.316413] [ 1315] 0 1315 1560 163 8 3 0 0 agetty Mar 20 14:13:10 i-deadbeef kernel: [501130.316415] [ 1318] 65534 1318 192324 625 43 4 0 0 nscd Mar 20 14:13:10 i-deadbeef kernel: [501130.316416] [ 1328] 0 1328 4901 441 14 3 0 0 irqbalance Mar 20 14:13:10 i-deadbeef kernel: [501130.316418] [ 1456] 113 1456 113060 1948 53 3 0 -1000 nslcd Mar 20 14:13:10 i-deadbeef kernel: [501130.316420] [ 1606] 0 1606 4873 597 14 3 0 0 cron Mar 20 14:13:10 i-deadbeef kernel: [501130.316421] [ 1608] 0 1608 4505 190 14 3 0 0 atd Mar 20 14:13:10 i-deadbeef kernel: [501130.316423] [ 5387] 0 5387 1061 184 7 3 0 0 runsv Mar 20 14:13:10 i-deadbeef kernel: [501130.316424] [ 5408] 0 5408 1097 310 8 3 0 0 svlogd Mar 20 14:13:10 i-deadbeef kernel: [501130.316426] [ 6648] 0 6648 172321 4005 43 6 0 0 graylog-collect Mar 20 14:13:10 i-deadbeef kernel: [501130.316428] [ 6671] 0 6671 4517 1648 15 5 0 0 remote_syslog Mar 20 14:13:10 i-deadbeef kernel: [501130.316429] [ 7379] 2001 7379 9253 788 23 3 0 0 systemd Mar 20 14:13:10 i-deadbeef kernel: [501130.316431] [ 7382] 2001 7382 37412 487 38 3 0 0 (sd-pam) Mar 20 14:13:10 i-deadbeef kernel: [501130.316434] [13306] 0 13306 14911 731 33 3 0 -1000 sshd Mar 20 14:13:10 i-deadbeef kernel: [501130.316436] [13373] 112 13373 24552 750 18 3 0 0 ntpd Mar 20 14:13:10 i-deadbeef kernel: [501130.316437] [31523] 997 31523 61748042 10732817 58374 256 0 0 java Mar 20 14:13:10 i-deadbeef kernel: [501130.316441] [20872] 0 20872 220188 3358 59 4 0 0 collectd Mar 20 14:13:10 i-deadbeef kernel: [501130.316444] [28319] 0 28319 115955 3937 38 6 0 0 filebeat Mar 20 14:13:10 i-deadbeef kernel: [501130.316446] [24134] 2044 24134 9253 1107 22 3 0 0 systemd Mar 20 14:13:10 i-deadbeef kernel: [501130.316448] [24139] 2044 24139 54123 819 40 3 0 0 (sd-pam) Mar 20 14:13:10 i-deadbeef kernel: [501130.316450] [24370] 0 24370 56291 1654 51 5 0 0 tsauditd Mar 20 14:13:10 i-deadbeef kernel: [501130.316453] [24386] 0 24386 225853 7290 47 16 0 0 node Mar 20 14:13:10 i-deadbeef kernel: [501130.316455] [24392] 0 24392 330305 17077 91 30 0 0 node Mar 20 14:13:10 i-deadbeef kernel: [501130.316457] [24405] 0 24405 34984 1170 42 3 0 0 tsfim Mar 20 14:13:10 i-deadbeef kernel: [501130.316459] Out of memory: Kill process 31523 (java) score 686 or sacrifice child Mar 20 14:13:10 i-deadbeef kernel: [501130.321697] Killed process 31523 (java) total-vm:246992168kB, anon-rss:31969456kB, file-rss:10960940kB