[453556.407716] check_smart_sta invoked oom-killer: gfp_mask=0x27000c0(GFP_KERNEL_ACCOUNT|__GFP_NOTRACK), order=2, oom_score_adj=0 [453556.407718] check_smart_sta cpuset=/ mems_allowed=0 [453556.407723] CPU: 3 PID: 16886 Comm: check_smart_sta Not tainted 4.8.0-36-generic #36~16.04.1-Ubuntu [453556.407724] Hardware name: MSI MS-7816/H87-G43 (MS-7816), BIOS V2.14B11 06/30/2014 [453556.407726] 0000000000000286 00000000d13a58b7 ffff90e46cf47b38 ffffffffa562d7b3 [453556.407729] ffff90e46cf47cc8 ffff90ec31098e80 ffff90e46cf47ba0 ffffffffa542e9bb [453556.407731] ffff90e46cf47b40 0000000000000000 0000000000000000 0000000000000000 [453556.407733] Call Trace: [453556.407739] [] dump_stack+0x63/0x90 [453556.407743] [] dump_header+0x5c/0x1dc [453556.407748] [] oom_kill_process+0x226/0x3f0 [453556.407750] [] out_of_memory+0x35a/0x3f0 [453556.407752] [] __alloc_pages_slowpath+0x9fb/0xa20 [453556.407754] [] __alloc_pages_nodemask+0x2ba/0x300 [453556.407759] [] copy_process.part.30+0x146/0x1b50 [453556.407761] [] _do_fork+0xe7/0x3f0 [453556.407764] [] ? __set_task_blocked+0x41/0xa0 [453556.407766] [] ? __set_current_blocked+0x36/0x60 [453556.407767] [] SyS_clone+0x19/0x20 [453556.407771] [] do_syscall_64+0x5e/0xc0 [453556.407775] [] entry_SYSCALL64_slow_path+0x25/0x25 [453556.407776] Mem-Info: [453556.407781] active_anon:352362 inactive_anon:133457 isolated_anon:0 active_file:3146209 inactive_file:350557 isolated_file:0 unevictable:8 dirty:1741 writeback:1 unstable:0 slab_reclaimable:4131034 slab_unreclaimable:10509 mapped:8525 shmem:1465 pagetables:2597 bounce:0 free:49897 free_pcp:128 free_cma:0 [453556.407784] Node 0 active_anon:1409448kB inactive_anon:533828kB active_file:12584836kB inactive_file:1402228kB unevictable:32kB isolated(anon):0kB isolated(file):0kB mapped:34100kB dirty:6964kB writeback:4kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 806912kB anon_thp: 5860kB writeback_tmp:0kB unstable:0kB pages_scanned:96 all_unreclaimable? no [453556.407785] Node 0 DMA free:15896kB min:32kB low:44kB high:56kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15984kB managed:15900kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:4kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [453556.407788] lowmem_reserve[]: 0 3348 31968 31968 31968 [453556.407791] Node 0 DMA32 free:121544kB min:7076kB low:10504kB high:13932kB active_anon:156196kB inactive_anon:0kB active_file:563576kB inactive_file:3804kB unevictable:0kB writepending:4kB present:3531156kB managed:3465588kB mlocked:0kB slab_reclaimable:2616200kB slab_unreclaimable:2424kB kernel_stack:144kB pagetables:44kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [453556.407793] lowmem_reserve[]: 0 0 28619 28619 28619 [453556.407795] Node 0 Normal free:62148kB min:60472kB low:89776kB high:119080kB active_anon:1252968kB inactive_anon:533828kB active_file:12021260kB inactive_file:1398424kB unevictable:32kB writepending:6964kB present:29851648kB managed:29310000kB mlocked:32kB slab_reclaimable:13907936kB slab_unreclaimable:39608kB kernel_stack:4592kB pagetables:10344kB bounce:0kB free_pcp:512kB local_pcp:0kB free_cma:0kB [453556.407798] lowmem_reserve[]: 0 0 0 0 0 [453556.407800] Node 0 DMA: 0*4kB 1*8kB (U) 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15896kB [453556.407808] Node 0 DMA32: 12197*4kB (UME) 9159*8kB (UME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 122060kB [453556.407813] Node 0 Normal: 15396*4kB (UMEH) 25*8kB (H) 13*16kB (H) 6*32kB (H) 3*64kB (H) 2*128kB (H) 1*256kB (H) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 62888kB [453556.407821] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [453556.407822] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [453556.407822] 3498277 total pagecache pages [453556.407823] 50 pages in swap cache [453556.407824] Swap cache stats: add 2044, delete 1994, find 38/54 [453556.407825] Free swap = 16760896kB [453556.407825] Total swap = 16768892kB [453556.407826] 8349697 pages RAM [453556.407826] 0 pages HighMem/MovableOnly [453556.407827] 151825 pages reserved [453556.407827] 0 pages cma reserved [453556.407827] 0 pages hwpoisoned [453556.407828] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name [453556.407847] [ 434] 0 434 9582 599 24 3 62 0 cgmanager [453556.407848] [ 502] 0 502 7487 179 20 3 48 0 upstart-udev-br [453556.407851] [ 517] 0 517 11442 549 23 3 467 -1000 systemd-udevd [453556.407853] [ 695] 0 695 7471 271 19 3 45 0 upstart-socket- [453556.407855] [ 1197] 0 1197 7475 254 18 3 44 0 upstart-file-br [453556.407857] [ 1248] 0 1248 6502 912 17 3 133 0 screen [453556.407859] [ 1250] 0 1250 4948 929 15 3 223 0 bash [453556.407860] [ 1251] 0 1251 6234 602 17 3 92 0 screen [453556.407861] [ 1253] 0 1253 4948 931 15 3 226 0 bash [453556.407863] [ 1265] 0 1265 4948 966 15 3 211 0 bash [453556.407865] [ 1292] 101 1292 64099 494 27 3 101 0 rsyslogd [453556.407866] [ 1334] 0 1334 4948 1178 14 3 0 0 bash [453556.407868] [ 1347] 102 1347 10724 290 24 3 59 0 dbus-daemon [453556.407869] [ 1395] 0 1395 43186 947 54 4 0 0 nginx [453556.407871] [ 1396] 33 1396 44326 2927 61 4 0 0 nginx [453556.407873] [ 1397] 33 1397 44358 2832 62 4 0 0 nginx [453556.407874] [ 1398] 33 1398 44249 2713 60 4 0 0 nginx [453556.407876] [ 1399] 33 1399 44259 2860 61 4 0 0 nginx [453556.407878] [ 1400] 33 1400 44262 2738 60 4 0 0 nginx [453556.407880] [ 1401] 33 1401 44262 2724 60 4 0 0 nginx [453556.407881] [ 1402] 33 1402 44262 2858 61 4 0 0 nginx [453556.407883] [ 1403] 33 1403 44270 2746 61 4 0 0 nginx [453556.407884] [ 1427] 0 1427 2396 617 11 3 0 0 wave.sh [453556.407886] [ 1431] 0 1431 4948 1167 15 3 0 0 bash [453556.407888] [ 1444] 0 1444 489968 396463 822 5 0 0 wave [453556.407890] [ 1446] 0 1446 33853 25058 65 3 0 0 waveclean [453556.407891] [ 1548] 0 1548 3985 403 13 3 0 0 getty [453556.407893] [ 1550] 0 1550 3985 401 13 3 0 0 getty [453556.407895] [ 1551] 0 1551 40878 838 36 3 0 0 thermald [453556.407896] [ 1555] 0 1555 3985 401 12 3 0 0 getty [453556.407898] [ 1556] 0 1556 3985 416 13 3 0 0 getty [453556.407900] [ 1558] 0 1558 3985 403 12 3 0 0 getty [453556.407901] [ 1580] 0 1580 6517 609 19 3 0 0 cron [453556.407903] [ 1582] 0 1582 1100 29 8 3 0 0 acpid [453556.407904] [ 1613] 0 1613 16380 1159 36 4 0 -1000 sshd [453556.407906] [ 1621] 0 1621 25742 85 17 3 0 0 lvmetad [453556.407908] [ 1633] 0 1633 4884 65 13 3 0 0 irqbalance [453556.407909] [ 1669] 0 1669 1103 159 8 3 0 0 tail [453556.407911] [ 1673] 0 1673 4948 1162 16 3 0 0 bash [453556.407913] [ 1691] 0 1691 1103 177 8 3 0 0 tail [453556.407914] [ 1695] 0 1695 4948 1176 15 3 0 0 bash [453556.407916] [ 1769] 0 1769 70249 3047 39 3 0 0 fail2ban-server [453556.407918] [ 1783] 0 1783 1103 167 8 3 0 0 tail [453556.407920] [ 1787] 0 1787 4948 1153 15 3 0 0 bash [453556.407921] [ 1966] 107 1966 125766 1065 44 4 0 0 icinga2 [453556.407923] [ 1975] 107 1975 589094 3496 85 5 0 0 icinga2 [453556.407925] [ 2085] 0 2085 3353 43 11 3 0 0 mcelog [453556.407926] [ 2098] 0 2098 3422 472 11 3 0 0 mdadm [453556.407928] [ 2271] 0 2271 16352 695 24 3 0 0 master [453556.407929] [ 2294] 106 2294 16910 314 25 3 0 0 qmgr [453556.407931] [ 2350] 0 2350 49452 15546 92 3 0 0 puppet [453556.407933] [ 2434] 104 2434 27509 911 25 3 0 0 ntpd [453556.407935] [ 2705] 0 2705 7137 596 18 3 0 0 systemd-logind [453556.407936] [ 2721] 0 2721 6367 438 18 3 0 0 smartd [453556.407938] [ 2942] 0 2942 3985 401 14 3 0 0 getty [453556.407939] [ 9235] 106 9235 20176 614 32 3 0 0 tlsmgr [453556.407941] [20585] 1000 20585 302705 8289 123 4 0 0 collectd [453556.407943] [20601] 1000 20601 7691 1627 21 3 0 0 nginx_stats.pl [453556.407945] [ 7220] 0 7220 140361 20235 116 4 0 0 mcollectived [453556.407947] [ 4564] 106 4564 16869 1096 24 4 0 0 pickup [453556.407949] [16863] 0 16863 43748 1701 56 3 0 0 curl [453556.407951] [16865] 107 16865 4521 735 15 3 0 0 check_smart_sta [453556.407952] [16885] 107 16885 4521 576 13 3 0 0 check_smart_sta [453556.407954] [16886] 107 16886 4521 80 13 3 0 0 check_smart_sta [453556.407955] Out of memory: Kill process 1444 (wave) score 31 or sacrifice child [453556.408643] Killed process 16863 (curl) total-vm:174992kB, anon-rss:932kB, file-rss:5872kB, shmem-rss:0kB