Sep 16 12:10:28 hydra01 kernel: [2090136.881120] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881123] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881126] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881128] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881131] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881134] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881136] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881138] Node 0 Normal per-cpu: Sep 16 12:10:28 hydra01 kernel: [2090136.881142] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881144] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881147] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881149] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881152] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881155] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881157] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881160] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881162] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881165] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881168] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881170] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881178] active_anon:1293148 inactive_anon:831660 isolated_anon:0 Sep 16 12:10:28 hydra01 kernel: [2090136.881178] active_file:1869813 inactive_file:3481473 isolated_file:0 Sep 16 12:10:28 hydra01 kernel: [2090136.881178] unevictable:0 dirty:24 writeback:0 unstable:0 Sep 16 12:10:28 hydra01 kernel: [2090136.881178] free:84628 slab_reclaimable:382242 slab_unreclaimable:107386 Sep 16 12:10:28 hydra01 kernel: [2090136.881178] mapped:26109 shmem:151 pagetables:9549 bounce:0 Sep 16 12:10:28 hydra01 kernel: [2090136.881178] free_cma:0 Sep 16 12:10:28 hydra01 kernel: [2090136.881184] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:10:28 hydra01 kernel: [2090136.881193] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:10:28 hydra01 kernel: [2090136.881199] Node 0 DMA32 free:146604kB min:6224kB low:7780kB high:9336kB active_anon:439564kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171216kB slab_unreclaimable:46916kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:28 hydra01 kernel: [2090136.881209] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:10:28 hydra01 kernel: [2090136.881214] Node 0 Normal free:176064kB min:61324kB low:76652kB high:91984kB active_anon:4733028kB inactive_anon:2533372kB active_file:7384056kB inactive_file:12556124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:96kB writeback:0kB mapped:100512kB shmem:604kB slab_reclaimable:1357752kB slab_unreclaimable:382628kB kernel_stack:8016kB pagetables:36668kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:28 hydra01 kernel: [2090136.881223] lowmem_reserve[]: 0 0 0 0 Sep 16 12:10:28 hydra01 kernel: [2090136.881228] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:10:28 hydra01 kernel: [2090136.881250] Node 0 DMA32: 3003*4kB (UEM) 3176*8kB (UEM) 6032*16kB (UEM) 382*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146668kB Sep 16 12:10:28 hydra01 kernel: [2090136.881268] Node 0 Normal: 18458*4kB (UEMR) 11750*8kB (UEMR) 468*16kB (UEMR) 8*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 177240kB Sep 16 12:10:28 hydra01 kernel: [2090136.881288] 5783692 total pagecache pages Sep 16 12:10:28 hydra01 kernel: [2090136.881291] 432252 pages in swap cache Sep 16 12:10:28 hydra01 kernel: [2090136.881294] Swap cache stats: add 20773462, delete 20341210, find 12661739/14316126 Sep 16 12:10:28 hydra01 kernel: [2090136.881297] Free swap = 185330256kB Sep 16 12:10:28 hydra01 kernel: [2090136.881299] Total swap = 187617276kB Sep 16 12:10:28 hydra01 kernel: [2090137.003126] 8388592 pages RAM Sep 16 12:10:28 hydra01 kernel: [2090137.003135] 151696 pages reserved Sep 16 12:10:28 hydra01 kernel: [2090137.003141] 8715664 pages shared Sep 16 12:10:28 hydra01 kernel: [2090137.003146] 3412418 pages non-shared Sep 16 12:10:34 hydra01 kernel: [2090142.417114] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:10:34 hydra01 kernel: [2090142.417123] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:10:34 hydra01 kernel: [2090142.417125] Call Trace: Sep 16 12:10:34 hydra01 kernel: [2090142.417139] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:10:34 hydra01 kernel: [2090142.417147] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:10:34 hydra01 kernel: [2090142.417153] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:10:34 hydra01 kernel: [2090142.417161] [] alloc_pages_current+0xb8/0x180 Sep 16 12:10:34 hydra01 kernel: [2090142.417166] [] __get_free_pages+0xe/0x50 Sep 16 12:10:34 hydra01 kernel: [2090142.417173] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:10:34 hydra01 kernel: [2090142.417179] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:10:34 hydra01 kernel: [2090142.417185] [] __kmalloc+0x161/0x170 Sep 16 12:10:34 hydra01 kernel: [2090142.417191] [] ? get_iowait_time+0x60/0x60 Sep 16 12:10:34 hydra01 kernel: [2090142.417196] [] stat_open+0x56/0xc0 Sep 16 12:10:34 hydra01 kernel: [2090142.417202] [] proc_reg_open+0xa0/0x180 Sep 16 12:10:34 hydra01 kernel: [2090142.417209] [] ? seq_release_private+0x60/0x60 Sep 16 12:10:34 hydra01 kernel: [2090142.417215] [] do_dentry_open+0x203/0x290 Sep 16 12:10:34 hydra01 kernel: [2090142.417220] [] ? init_once+0x20/0x20 Sep 16 12:10:34 hydra01 kernel: [2090142.417224] [] vfs_open+0x49/0x50 Sep 16 12:10:34 hydra01 kernel: [2090142.417229] [] do_last+0x296/0xea0 Sep 16 12:10:34 hydra01 kernel: [2090142.417234] [] ? inode_permission+0x18/0x50 Sep 16 12:10:34 hydra01 kernel: [2090142.417238] [] ? link_path_walk+0x6b/0x880 Sep 16 12:10:34 hydra01 kernel: [2090142.417243] [] path_openat+0xb3/0x4b0 Sep 16 12:10:34 hydra01 kernel: [2090142.417249] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:10:34 hydra01 kernel: [2090142.417255] [] ? unmap_region+0xdb/0x120 Sep 16 12:10:34 hydra01 kernel: [2090142.417260] [] do_filp_open+0x41/0xa0 Sep 16 12:10:34 hydra01 kernel: [2090142.417266] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:10:34 hydra01 kernel: [2090142.417270] [] do_sys_open+0xf3/0x230 Sep 16 12:10:34 hydra01 kernel: [2090142.417275] [] sys_open+0x21/0x30 Sep 16 12:10:34 hydra01 kernel: [2090142.417280] [] system_call_fastpath+0x1a/0x1f Sep 16 12:10:34 hydra01 kernel: [2090142.417283] Mem-Info: Sep 16 12:10:34 hydra01 kernel: [2090142.417286] Node 0 DMA per-cpu: Sep 16 12:10:34 hydra01 kernel: [2090142.417291] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417293] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417296] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417299] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417301] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417304] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417306] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417309] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417312] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417314] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417317] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417320] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417322] Node 0 DMA32 per-cpu: Sep 16 12:10:34 hydra01 kernel: [2090142.417326] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417328] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417331] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417333] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417336] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417339] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417341] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417344] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417346] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417349] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417352] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417354] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417356] Node 0 Normal per-cpu: Sep 16 12:10:34 hydra01 kernel: [2090142.417360] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417362] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417365] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417368] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417370] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417373] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417375] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417378] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417381] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417383] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417386] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417389] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417396] active_anon:1293257 inactive_anon:831692 isolated_anon:0 Sep 16 12:10:34 hydra01 kernel: [2090142.417396] active_file:1869815 inactive_file:3481317 isolated_file:0 Sep 16 12:10:34 hydra01 kernel: [2090142.417396] unevictable:0 dirty:31 writeback:0 unstable:0 Sep 16 12:10:34 hydra01 kernel: [2090142.417396] free:84629 slab_reclaimable:382211 slab_unreclaimable:107374 Sep 16 12:10:34 hydra01 kernel: [2090142.417396] mapped:25933 shmem:151 pagetables:9557 bounce:0 Sep 16 12:10:34 hydra01 kernel: [2090142.417396] free_cma:0 Sep 16 12:10:34 hydra01 kernel: [2090142.417402] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:10:34 hydra01 kernel: [2090142.417411] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:10:34 hydra01 kernel: [2090142.417417] Node 0 DMA32 free:146572kB min:6224kB low:7780kB high:9336kB active_anon:439564kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171216kB slab_unreclaimable:46948kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:34 hydra01 kernel: [2090142.417427] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:10:34 hydra01 kernel: [2090142.417432] Node 0 Normal free:176100kB min:61324kB low:76652kB high:91984kB active_anon:4733464kB inactive_anon:2533500kB active_file:7384064kB inactive_file:12555500kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:124kB writeback:0kB mapped:99808kB shmem:604kB slab_reclaimable:1357628kB slab_unreclaimable:382548kB kernel_stack:8008kB pagetables:36700kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:34 hydra01 kernel: [2090142.417441] lowmem_reserve[]: 0 0 0 0 Sep 16 12:10:34 hydra01 kernel: [2090142.417446] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:10:34 hydra01 kernel: [2090142.417467] Node 0 DMA32: 3003*4kB (UEM) 3176*8kB (UEM) 6032*16kB (UEM) 381*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146636kB Sep 16 12:10:34 hydra01 kernel: [2090142.417486] Node 0 Normal: 18468*4kB (UEMR) 11720*8kB (UEMR) 509*16kB (UEMR) 11*32kB (UER) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 177792kB Sep 16 12:10:34 hydra01 kernel: [2090142.417506] 5783540 total pagecache pages Sep 16 12:10:34 hydra01 kernel: [2090142.417508] 432252 pages in swap cache Sep 16 12:10:34 hydra01 kernel: [2090142.417511] Swap cache stats: add 20773462, delete 20341210, find 12661739/14316126 Sep 16 12:10:34 hydra01 kernel: [2090142.417514] Free swap = 185330256kB Sep 16 12:10:34 hydra01 kernel: [2090142.417516] Total swap = 187617276kB Sep 16 12:10:34 hydra01 kernel: [2090142.610878] 8388592 pages RAM Sep 16 12:10:34 hydra01 kernel: [2090142.610892] 151696 pages reserved Sep 16 12:10:34 hydra01 kernel: [2090142.610895] 8715014 pages shared Sep 16 12:10:34 hydra01 kernel: [2090142.610897] 3412363 pages non-shared Sep 16 12:10:48 hydra01 kernel: [2090156.877149] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:10:48 hydra01 kernel: [2090156.877158] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:10:48 hydra01 kernel: [2090156.877161] Call Trace: Sep 16 12:10:48 hydra01 kernel: [2090156.877174] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:10:48 hydra01 kernel: [2090156.877180] [] ? drain_local_pages+0x16/0x20 Sep 16 12:10:48 hydra01 kernel: [2090156.877188] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:10:48 hydra01 kernel: [2090156.877194] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:10:48 hydra01 kernel: [2090156.877202] [] alloc_pages_current+0xb8/0x180 Sep 16 12:10:48 hydra01 kernel: [2090156.877207] [] __get_free_pages+0xe/0x50 Sep 16 12:10:48 hydra01 kernel: [2090156.877214] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:10:48 hydra01 kernel: [2090156.877220] [] __kmalloc+0x161/0x170 Sep 16 12:10:48 hydra01 kernel: [2090156.877227] [] ? get_iowait_time+0x60/0x60 Sep 16 12:10:48 hydra01 kernel: [2090156.877231] [] stat_open+0x56/0xc0 Sep 16 12:10:48 hydra01 kernel: [2090156.877238] [] proc_reg_open+0xa0/0x180 Sep 16 12:10:48 hydra01 kernel: [2090156.877244] [] ? seq_release_private+0x60/0x60 Sep 16 12:10:48 hydra01 kernel: [2090156.877250] [] do_dentry_open+0x203/0x290 Sep 16 12:10:48 hydra01 kernel: [2090156.877255] [] ? init_once+0x20/0x20 Sep 16 12:10:48 hydra01 kernel: [2090156.877259] [] vfs_open+0x49/0x50 Sep 16 12:10:48 hydra01 kernel: [2090156.877264] [] do_last+0x296/0xea0 Sep 16 12:10:48 hydra01 kernel: [2090156.877269] [] ? inode_permission+0x18/0x50 Sep 16 12:10:48 hydra01 kernel: [2090156.877274] [] ? link_path_walk+0x6b/0x880 Sep 16 12:10:48 hydra01 kernel: [2090156.877279] [] path_openat+0xb3/0x4b0 Sep 16 12:10:48 hydra01 kernel: [2090156.877284] [] do_filp_open+0x41/0xa0 Sep 16 12:10:48 hydra01 kernel: [2090156.877290] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:10:48 hydra01 kernel: [2090156.877295] [] do_sys_open+0xf3/0x230 Sep 16 12:10:48 hydra01 kernel: [2090156.877303] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:10:48 hydra01 kernel: [2090156.877307] [] sys_open+0x21/0x30 Sep 16 12:10:48 hydra01 kernel: [2090156.877313] [] system_call_fastpath+0x1a/0x1f Sep 16 12:10:48 hydra01 kernel: [2090156.877316] Mem-Info: Sep 16 12:10:48 hydra01 kernel: [2090156.877318] Node 0 DMA per-cpu: Sep 16 12:10:48 hydra01 kernel: [2090156.877323] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877326] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877328] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877331] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877333] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877336] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877338] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877341] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877344] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877346] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877349] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877352] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877354] Node 0 DMA32 per-cpu: Sep 16 12:10:48 hydra01 kernel: [2090156.877358] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877360] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877363] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877366] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877368] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877371] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877373] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877376] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877379] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877381] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877384] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877387] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877389] Node 0 Normal per-cpu: Sep 16 12:10:48 hydra01 kernel: [2090156.877392] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877395] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877397] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877400] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877403] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877405] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877408] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877410] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877413] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877416] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877418] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877421] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877428] active_anon:1293344 inactive_anon:831787 isolated_anon:0 Sep 16 12:10:48 hydra01 kernel: [2090156.877428] active_file:1869857 inactive_file:3482209 isolated_file:0 Sep 16 12:10:48 hydra01 kernel: [2090156.877428] unevictable:0 dirty:108 writeback:0 unstable:0 Sep 16 12:10:48 hydra01 kernel: [2090156.877428] free:83817 slab_reclaimable:382210 slab_unreclaimable:107318 Sep 16 12:10:48 hydra01 kernel: [2090156.877428] mapped:26059 shmem:151 pagetables:9654 bounce:0 Sep 16 12:10:48 hydra01 kernel: [2090156.877428] free_cma:0 Sep 16 12:10:48 hydra01 kernel: [2090156.877435] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:10:48 hydra01 kernel: [2090156.877443] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:10:48 hydra01 kernel: [2090156.877450] Node 0 DMA32 free:146592kB min:6224kB low:7780kB high:9336kB active_anon:439544kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:36kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171216kB slab_unreclaimable:46868kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:48 hydra01 kernel: [2090156.877459] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:10:48 hydra01 kernel: [2090156.877464] Node 0 Normal free:172832kB min:61324kB low:76652kB high:91984kB active_anon:4733832kB inactive_anon:2533880kB active_file:7384232kB inactive_file:12559068kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:396kB writeback:0kB mapped:100312kB shmem:604kB slab_reclaimable:1357624kB slab_unreclaimable:382404kB kernel_stack:8024kB pagetables:37088kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:48 hydra01 kernel: [2090156.877474] lowmem_reserve[]: 0 0 0 0 Sep 16 12:10:48 hydra01 kernel: [2090156.877478] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:10:48 hydra01 kernel: [2090156.877500] Node 0 DMA32: 3004*4kB (UEM) 3178*8kB (UEM) 6033*16kB (UEM) 381*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146672kB Sep 16 12:10:48 hydra01 kernel: [2090156.877518] Node 0 Normal: 17917*4kB (UEMR) 11742*8kB (UEMR) 389*16kB (UEMR) 7*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 173716kB Sep 16 12:10:48 hydra01 kernel: [2090156.877538] 5784492 total pagecache pages Sep 16 12:10:48 hydra01 kernel: [2090156.877540] 432252 pages in swap cache Sep 16 12:10:48 hydra01 kernel: [2090156.877543] Swap cache stats: add 20773462, delete 20341210, find 12661739/14316126 Sep 16 12:10:48 hydra01 kernel: [2090156.877546] Free swap = 185330256kB Sep 16 12:10:48 hydra01 kernel: [2090156.877548] Total swap = 187617276kB Sep 16 12:10:48 hydra01 kernel: [2090157.070886] 8388592 pages RAM Sep 16 12:10:48 hydra01 kernel: [2090157.070898] 151696 pages reserved Sep 16 12:10:48 hydra01 kernel: [2090157.070910] 8710989 pages shared Sep 16 12:10:48 hydra01 kernel: [2090157.070920] 3417737 pages non-shared Sep 16 12:10:53 hydra01 kernel: [2090161.874944] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:10:53 hydra01 kernel: [2090161.874953] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:10:53 hydra01 kernel: [2090161.874955] Call Trace: Sep 16 12:10:53 hydra01 kernel: [2090161.874969] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:10:53 hydra01 kernel: [2090161.874976] [] ? drain_local_pages+0x16/0x20 Sep 16 12:10:53 hydra01 kernel: [2090161.874983] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:10:53 hydra01 kernel: [2090161.874989] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:10:53 hydra01 kernel: [2090161.874997] [] alloc_pages_current+0xb8/0x180 Sep 16 12:10:53 hydra01 kernel: [2090161.875002] [] __get_free_pages+0xe/0x50 Sep 16 12:10:53 hydra01 kernel: [2090161.875009] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:10:53 hydra01 kernel: [2090161.875015] [] __kmalloc+0x161/0x170 Sep 16 12:10:53 hydra01 kernel: [2090161.875022] [] ? get_iowait_time+0x60/0x60 Sep 16 12:10:53 hydra01 kernel: [2090161.875026] [] stat_open+0x56/0xc0 Sep 16 12:10:53 hydra01 kernel: [2090161.875033] [] proc_reg_open+0xa0/0x180 Sep 16 12:10:53 hydra01 kernel: [2090161.875039] [] ? seq_release_private+0x60/0x60 Sep 16 12:10:53 hydra01 kernel: [2090161.875045] [] do_dentry_open+0x203/0x290 Sep 16 12:10:53 hydra01 kernel: [2090161.875050] [] ? init_once+0x20/0x20 Sep 16 12:10:53 hydra01 kernel: [2090161.875054] [] vfs_open+0x49/0x50 Sep 16 12:10:53 hydra01 kernel: [2090161.875059] [] do_last+0x296/0xea0 Sep 16 12:10:53 hydra01 kernel: [2090161.875064] [] ? inode_permission+0x18/0x50 Sep 16 12:10:53 hydra01 kernel: [2090161.875068] [] ? link_path_walk+0x6b/0x880 Sep 16 12:10:53 hydra01 kernel: [2090161.875073] [] path_openat+0xb3/0x4b0 Sep 16 12:10:53 hydra01 kernel: [2090161.875078] [] do_filp_open+0x41/0xa0 Sep 16 12:10:53 hydra01 kernel: [2090161.875084] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:10:53 hydra01 kernel: [2090161.875088] [] do_sys_open+0xf3/0x230 Sep 16 12:10:53 hydra01 kernel: [2090161.875094] [] ? sys_select+0xcb/0x110 Sep 16 12:10:53 hydra01 kernel: [2090161.875098] [] sys_open+0x21/0x30 Sep 16 12:10:53 hydra01 kernel: [2090161.875104] [] system_call_fastpath+0x1a/0x1f Sep 16 12:10:53 hydra01 kernel: [2090161.875107] Mem-Info: Sep 16 12:10:53 hydra01 kernel: [2090161.875109] Node 0 DMA per-cpu: Sep 16 12:10:53 hydra01 kernel: [2090161.875114] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875117] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875119] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875122] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875124] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875127] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875130] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875132] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875135] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875137] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875140] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875143] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875145] Node 0 DMA32 per-cpu: Sep 16 12:10:53 hydra01 kernel: [2090161.875149] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875151] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875154] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875156] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875159] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875162] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875164] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875167] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875170] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875172] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875175] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875178] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875179] Node 0 Normal per-cpu: Sep 16 12:10:53 hydra01 kernel: [2090161.875183] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875186] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875188] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875191] CPU 3: hi: 186, btch: 31 usd: 169 Sep 16 12:10:53 hydra01 kernel: [2090161.875194] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875196] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875199] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875202] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875204] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875207] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875210] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875212] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875220] active_anon:1293299 inactive_anon:831783 isolated_anon:0 Sep 16 12:10:53 hydra01 kernel: [2090161.875220] active_file:1869977 inactive_file:3483159 isolated_file:0 Sep 16 12:10:53 hydra01 kernel: [2090161.875220] unevictable:0 dirty:303 writeback:0 unstable:0 Sep 16 12:10:53 hydra01 kernel: [2090161.875220] free:82342 slab_reclaimable:382226 slab_unreclaimable:107337 Sep 16 12:10:53 hydra01 kernel: [2090161.875220] mapped:26105 shmem:151 pagetables:9677 bounce:0 Sep 16 12:10:53 hydra01 kernel: [2090161.875220] free_cma:0 Sep 16 12:10:53 hydra01 kernel: [2090161.875226] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:10:53 hydra01 kernel: [2090161.875235] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:10:53 hydra01 kernel: [2090161.875242] Node 0 DMA32 free:146480kB min:6224kB low:7780kB high:9336kB active_anon:439544kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171280kB slab_unreclaimable:46932kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:53 hydra01 kernel: [2090161.875251] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:10:53 hydra01 kernel: [2090161.875256] Node 0 Normal free:167044kB min:61324kB low:76652kB high:91984kB active_anon:4733652kB inactive_anon:2533864kB active_file:7384712kB inactive_file:12562868kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:1204kB writeback:0kB mapped:100496kB shmem:604kB slab_reclaimable:1357624kB slab_unreclaimable:382416kB kernel_stack:8008kB pagetables:37180kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:10:53 hydra01 kernel: [2090161.875265] lowmem_reserve[]: 0 0 0 0 Sep 16 12:10:53 hydra01 kernel: [2090161.875270] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:10:53 hydra01 kernel: [2090161.875292] Node 0 DMA32: 3004*4kB (UEM) 3178*8kB (UEM) 6033*16kB (UEM) 377*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146544kB Sep 16 12:10:53 hydra01 kernel: [2090161.875310] Node 0 Normal: 17715*4kB (UEMR) 11340*8kB (UEMR) 350*16kB (UER) 7*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 169068kB Sep 16 12:10:53 hydra01 kernel: [2090161.875331] 5785391 total pagecache pages Sep 16 12:10:53 hydra01 kernel: [2090161.875333] 432252 pages in swap cache Sep 16 12:10:53 hydra01 kernel: [2090161.875336] Swap cache stats: add 20773462, delete 20341210, find 12661739/14316126 Sep 16 12:10:53 hydra01 kernel: [2090161.875338] Free swap = 185330256kB Sep 16 12:10:53 hydra01 kernel: [2090161.875340] Total swap = 187617276kB Sep 16 12:10:53 hydra01 kernel: [2090162.068325] 8388592 pages RAM Sep 16 12:10:53 hydra01 kernel: [2090162.068340] 151696 pages reserved Sep 16 12:10:53 hydra01 kernel: [2090162.068349] 8709628 pages shared Sep 16 12:10:53 hydra01 kernel: [2090162.068358] 3419558 pages non-shared Sep 16 12:11:04 hydra01 kernel: [2090172.396068] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:11:04 hydra01 kernel: [2090172.396080] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:11:04 hydra01 kernel: [2090172.396084] Call Trace: Sep 16 12:11:04 hydra01 kernel: [2090172.396101] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:11:04 hydra01 kernel: [2090172.396110] [] ? drain_local_pages+0x16/0x20 Sep 16 12:11:04 hydra01 kernel: [2090172.396120] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:11:04 hydra01 kernel: [2090172.396129] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:11:04 hydra01 kernel: [2090172.396140] [] alloc_pages_current+0xb8/0x180 Sep 16 12:11:04 hydra01 kernel: [2090172.396148] [] __get_free_pages+0xe/0x50 Sep 16 12:11:04 hydra01 kernel: [2090172.396159] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:11:04 hydra01 kernel: [2090172.396167] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:11:04 hydra01 kernel: [2090172.396174] [] __kmalloc+0x161/0x170 Sep 16 12:11:04 hydra01 kernel: [2090172.396183] [] ? get_iowait_time+0x60/0x60 Sep 16 12:11:04 hydra01 kernel: [2090172.396190] [] stat_open+0x56/0xc0 Sep 16 12:11:04 hydra01 kernel: [2090172.396198] [] proc_reg_open+0xa0/0x180 Sep 16 12:11:04 hydra01 kernel: [2090172.396208] [] ? seq_release_private+0x60/0x60 Sep 16 12:11:04 hydra01 kernel: [2090172.396216] [] do_dentry_open+0x203/0x290 Sep 16 12:11:04 hydra01 kernel: [2090172.396223] [] ? init_once+0x20/0x20 Sep 16 12:11:04 hydra01 kernel: [2090172.396229] [] vfs_open+0x49/0x50 Sep 16 12:11:04 hydra01 kernel: [2090172.396237] [] do_last+0x296/0xea0 Sep 16 12:11:04 hydra01 kernel: [2090172.396244] [] ? inode_permission+0x18/0x50 Sep 16 12:11:04 hydra01 kernel: [2090172.396251] [] ? link_path_walk+0x6b/0x880 Sep 16 12:11:04 hydra01 kernel: [2090172.396258] [] path_openat+0xb3/0x4b0 Sep 16 12:11:04 hydra01 kernel: [2090172.396268] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:11:04 hydra01 kernel: [2090172.396277] [] ? unmap_region+0xdb/0x120 Sep 16 12:11:04 hydra01 kernel: [2090172.396283] [] do_filp_open+0x41/0xa0 Sep 16 12:11:04 hydra01 kernel: [2090172.396291] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:11:04 hydra01 kernel: [2090172.396299] [] do_sys_open+0xf3/0x230 Sep 16 12:11:04 hydra01 kernel: [2090172.396306] [] sys_open+0x21/0x30 Sep 16 12:11:04 hydra01 kernel: [2090172.396315] [] system_call_fastpath+0x1a/0x1f Sep 16 12:11:04 hydra01 kernel: [2090172.396319] Mem-Info: Sep 16 12:11:04 hydra01 kernel: [2090172.396323] Node 0 DMA per-cpu: Sep 16 12:11:04 hydra01 kernel: [2090172.396329] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396334] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396337] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396341] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396345] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396350] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396354] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396359] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396364] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396368] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396373] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396377] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396381] Node 0 DMA32 per-cpu: Sep 16 12:11:04 hydra01 kernel: [2090172.396387] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396391] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396394] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396398] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396402] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396407] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396412] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396416] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396421] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396426] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396430] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396435] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396438] Node 0 Normal per-cpu: Sep 16 12:11:04 hydra01 kernel: [2090172.396444] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396447] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396452] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396456] CPU 3: hi: 186, btch: 31 usd: 159 Sep 16 12:11:04 hydra01 kernel: [2090172.396460] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396464] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396469] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396474] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396478] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396483] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396488] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396492] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396503] active_anon:1293893 inactive_anon:831818 isolated_anon:0 Sep 16 12:11:04 hydra01 kernel: [2090172.396503] active_file:1869982 inactive_file:3479497 isolated_file:0 Sep 16 12:11:04 hydra01 kernel: [2090172.396503] unevictable:0 dirty:235 writeback:0 unstable:0 Sep 16 12:11:04 hydra01 kernel: [2090172.396503] free:85766 slab_reclaimable:382170 slab_unreclaimable:107381 Sep 16 12:11:04 hydra01 kernel: [2090172.396503] mapped:26062 shmem:151 pagetables:9617 bounce:0 Sep 16 12:11:04 hydra01 kernel: [2090172.396503] free_cma:0 Sep 16 12:11:04 hydra01 kernel: [2090172.396514] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:11:04 hydra01 kernel: [2090172.396527] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:11:04 hydra01 kernel: [2090172.396536] Node 0 DMA32 free:146312kB min:6224kB low:7780kB high:9336kB active_anon:439544kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171272kB slab_unreclaimable:47076kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:04 hydra01 kernel: [2090172.396550] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:11:04 hydra01 kernel: [2090172.396557] Node 0 Normal free:181200kB min:61324kB low:76652kB high:91984kB active_anon:4736028kB inactive_anon:2534004kB active_file:7384732kB inactive_file:12547836kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:936kB writeback:0kB mapped:100324kB shmem:604kB slab_reclaimable:1357408kB slab_unreclaimable:382448kB kernel_stack:8032kB pagetables:36940kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:04 hydra01 kernel: [2090172.396571] lowmem_reserve[]: 0 0 0 0 Sep 16 12:11:04 hydra01 kernel: [2090172.396579] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:11:04 hydra01 kernel: [2090172.396609] Node 0 DMA32: 3004*4kB (UEM) 3179*8kB (UEM) 6032*16kB (UEM) 372*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146376kB Sep 16 12:11:04 hydra01 kernel: [2090172.396637] Node 0 Normal: 24076*4kB (UEMR) 10339*8kB (UEMR) 103*16kB (UEMR) 15*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 182808kB Sep 16 12:11:04 hydra01 kernel: [2090172.396666] 5781746 total pagecache pages Sep 16 12:11:04 hydra01 kernel: [2090172.396670] 432252 pages in swap cache Sep 16 12:11:04 hydra01 kernel: [2090172.396675] Swap cache stats: add 20773462, delete 20341210, find 12661739/14316126 Sep 16 12:11:04 hydra01 kernel: [2090172.396679] Free swap = 185330256kB Sep 16 12:11:04 hydra01 kernel: [2090172.396682] Total swap = 187617276kB Sep 16 12:11:04 hydra01 kernel: [2090172.591342] 8388592 pages RAM Sep 16 12:11:04 hydra01 kernel: [2090172.591353] 151696 pages reserved Sep 16 12:11:04 hydra01 kernel: [2090172.591363] 8703525 pages shared Sep 16 12:11:04 hydra01 kernel: [2090172.591373] 3422919 pages non-shared Sep 16 12:11:48 hydra01 kernel: [2090216.191561] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:11:48 hydra01 kernel: [2090216.191569] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:11:48 hydra01 kernel: [2090216.191572] Call Trace: Sep 16 12:11:48 hydra01 kernel: [2090216.191585] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:11:48 hydra01 kernel: [2090216.191592] [] ? drain_local_pages+0x16/0x20 Sep 16 12:11:48 hydra01 kernel: [2090216.191600] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:11:48 hydra01 kernel: [2090216.191606] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:11:48 hydra01 kernel: [2090216.191614] [] alloc_pages_current+0xb8/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.191620] [] __get_free_pages+0xe/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.191627] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:11:48 hydra01 kernel: [2090216.191633] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:11:48 hydra01 kernel: [2090216.191639] [] __kmalloc+0x161/0x170 Sep 16 12:11:48 hydra01 kernel: [2090216.191647] [] ? setkey+0x80/0x100 Sep 16 12:11:48 hydra01 kernel: [2090216.191654] [] ? get_iowait_time+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.191658] [] stat_open+0x56/0xc0 Sep 16 12:11:48 hydra01 kernel: [2090216.191665] [] proc_reg_open+0xa0/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.191672] [] ? seq_release_private+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.191677] [] do_dentry_open+0x203/0x290 Sep 16 12:11:48 hydra01 kernel: [2090216.191683] [] ? init_once+0x20/0x20 Sep 16 12:11:48 hydra01 kernel: [2090216.191687] [] vfs_open+0x49/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.191692] [] do_last+0x296/0xea0 Sep 16 12:11:48 hydra01 kernel: [2090216.191697] [] ? inode_permission+0x18/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.191701] [] ? link_path_walk+0x6b/0x880 Sep 16 12:11:48 hydra01 kernel: [2090216.191706] [] path_openat+0xb3/0x4b0 Sep 16 12:11:48 hydra01 kernel: [2090216.191713] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:11:48 hydra01 kernel: [2090216.191720] [] ? unmap_region+0xdb/0x120 Sep 16 12:11:48 hydra01 kernel: [2090216.191725] [] do_filp_open+0x41/0xa0 Sep 16 12:11:48 hydra01 kernel: [2090216.191730] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:11:48 hydra01 kernel: [2090216.191735] [] do_sys_open+0xf3/0x230 Sep 16 12:11:48 hydra01 kernel: [2090216.191740] [] sys_open+0x21/0x30 Sep 16 12:11:48 hydra01 kernel: [2090216.191745] [] system_call_fastpath+0x1a/0x1f Sep 16 12:11:48 hydra01 kernel: [2090216.191748] Mem-Info: Sep 16 12:11:48 hydra01 kernel: [2090216.191751] Node 0 DMA per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.191755] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191758] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191761] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191763] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191766] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191768] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191771] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191773] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191776] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191779] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191781] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191784] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191786] Node 0 DMA32 per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.191790] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191793] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191795] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191798] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191801] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191803] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191806] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191808] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191811] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191814] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191817] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191819] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191821] Node 0 Normal per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.191825] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191827] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191830] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191832] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191835] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191838] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191840] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191843] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191846] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191848] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191851] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191853] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191861] active_anon:1294036 inactive_anon:832044 isolated_anon:0 Sep 16 12:11:48 hydra01 kernel: [2090216.191861] active_file:1870016 inactive_file:3475947 isolated_file:0 Sep 16 12:11:48 hydra01 kernel: [2090216.191861] unevictable:0 dirty:223 writeback:0 unstable:0 Sep 16 12:11:48 hydra01 kernel: [2090216.191861] free:89341 slab_reclaimable:382072 slab_unreclaimable:107292 Sep 16 12:11:48 hydra01 kernel: [2090216.191861] mapped:26092 shmem:151 pagetables:9582 bounce:0 Sep 16 12:11:48 hydra01 kernel: [2090216.191861] free_cma:0 Sep 16 12:11:48 hydra01 kernel: [2090216.191868] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:11:48 hydra01 kernel: [2090216.191877] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:11:48 hydra01 kernel: [2090216.191883] Node 0 DMA32 free:146424kB min:6224kB low:7780kB high:9336kB active_anon:439544kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:16kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171172kB slab_unreclaimable:47028kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.191892] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:11:48 hydra01 kernel: [2090216.191898] Node 0 Normal free:195096kB min:61324kB low:76652kB high:91984kB active_anon:4736600kB inactive_anon:2534908kB active_file:7384868kB inactive_file:12534020kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:876kB writeback:0kB mapped:100444kB shmem:604kB slab_reclaimable:1357116kB slab_unreclaimable:382140kB kernel_stack:8016kB pagetables:36800kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.191907] lowmem_reserve[]: 0 0 0 0 Sep 16 12:11:48 hydra01 kernel: [2090216.191912] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:11:48 hydra01 kernel: [2090216.191933] Node 0 DMA32: 3005*4kB (UEM) 3179*8kB (UEM) 6033*16kB (UEM) 376*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146524kB Sep 16 12:11:48 hydra01 kernel: [2090216.191952] Node 0 Normal: 23742*4kB (UEMR) 11423*8kB (UEMR) 519*16kB (UER) 13*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 196736kB Sep 16 12:11:48 hydra01 kernel: [2090216.191972] 5778366 total pagecache pages Sep 16 12:11:48 hydra01 kernel: [2090216.191975] 432246 pages in swap cache Sep 16 12:11:48 hydra01 kernel: [2090216.191978] Swap cache stats: add 20773464, delete 20341218, find 12661748/14316137 Sep 16 12:11:48 hydra01 kernel: [2090216.191980] Free swap = 185330288kB Sep 16 12:11:48 hydra01 kernel: [2090216.191982] Total swap = 187617276kB Sep 16 12:11:48 hydra01 kernel: [2090216.384882] 8388592 pages RAM Sep 16 12:11:48 hydra01 kernel: [2090216.384892] 151696 pages reserved Sep 16 12:11:48 hydra01 kernel: [2090216.384897] 8687901 pages shared Sep 16 12:11:48 hydra01 kernel: [2090216.384898] 3434902 pages non-shared Sep 16 12:11:48 hydra01 kernel: [2090216.385798] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:11:48 hydra01 kernel: [2090216.385802] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:11:48 hydra01 kernel: [2090216.385803] Call Trace: Sep 16 12:11:48 hydra01 kernel: [2090216.385811] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:11:48 hydra01 kernel: [2090216.385815] [] ? drain_local_pages+0x16/0x20 Sep 16 12:11:48 hydra01 kernel: [2090216.385820] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:11:48 hydra01 kernel: [2090216.385823] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:11:48 hydra01 kernel: [2090216.385827] [] alloc_pages_current+0xb8/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.385830] [] __get_free_pages+0xe/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.385835] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:11:48 hydra01 kernel: [2090216.385838] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:11:48 hydra01 kernel: [2090216.385841] [] __kmalloc+0x161/0x170 Sep 16 12:11:48 hydra01 kernel: [2090216.385845] [] ? amiga_partition+0x2c3/0x4d0 Sep 16 12:11:48 hydra01 kernel: [2090216.385849] [] ? get_iowait_time+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.385851] [] stat_open+0x56/0xc0 Sep 16 12:11:48 hydra01 kernel: [2090216.385855] [] proc_reg_open+0xa0/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.385859] [] ? seq_release_private+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.385862] [] do_dentry_open+0x203/0x290 Sep 16 12:11:48 hydra01 kernel: [2090216.385865] [] ? init_once+0x20/0x20 Sep 16 12:11:48 hydra01 kernel: [2090216.385867] [] vfs_open+0x49/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.385870] [] do_last+0x296/0xea0 Sep 16 12:11:48 hydra01 kernel: [2090216.385872] [] ? inode_permission+0x18/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.385875] [] ? link_path_walk+0x6b/0x880 Sep 16 12:11:48 hydra01 kernel: [2090216.385877] [] path_openat+0xb3/0x4b0 Sep 16 12:11:48 hydra01 kernel: [2090216.385881] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:11:48 hydra01 kernel: [2090216.385885] [] ? unmap_region+0xdb/0x120 Sep 16 12:11:48 hydra01 kernel: [2090216.385887] [] do_filp_open+0x41/0xa0 Sep 16 12:11:48 hydra01 kernel: [2090216.385890] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:11:48 hydra01 kernel: [2090216.385893] [] do_sys_open+0xf3/0x230 Sep 16 12:11:48 hydra01 kernel: [2090216.385895] [] sys_open+0x21/0x30 Sep 16 12:11:48 hydra01 kernel: [2090216.385898] [] system_call_fastpath+0x1a/0x1f Sep 16 12:11:48 hydra01 kernel: [2090216.385900] Mem-Info: Sep 16 12:11:48 hydra01 kernel: [2090216.385901] Node 0 DMA per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.385904] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385905] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385906] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385908] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385909] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385911] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385912] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385913] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385915] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385916] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385917] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385919] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385920] Node 0 DMA32 per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.385922] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385923] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385925] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385926] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385927] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385929] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385930] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385932] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385933] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385934] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385936] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385937] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385938] Node 0 Normal per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.385940] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385941] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385943] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385944] CPU 3: hi: 186, btch: 31 usd: 23 Sep 16 12:11:48 hydra01 kernel: [2090216.385946] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385947] CPU 5: hi: 186, btch: 31 usd: 26 Sep 16 12:11:48 hydra01 kernel: [2090216.385948] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385950] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385951] CPU 8: hi: 186, btch: 31 usd: 112 Sep 16 12:11:48 hydra01 kernel: [2090216.385952] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385954] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385955] CPU 11: hi: 186, btch: 31 usd: 65 Sep 16 12:11:48 hydra01 kernel: [2090216.385959] active_anon:1294231 inactive_anon:832044 isolated_anon:0 Sep 16 12:11:48 hydra01 kernel: [2090216.385959] active_file:1870017 inactive_file:3475948 isolated_file:0 Sep 16 12:11:48 hydra01 kernel: [2090216.385959] unevictable:0 dirty:245 writeback:0 unstable:0 Sep 16 12:11:48 hydra01 kernel: [2090216.385959] free:88867 slab_reclaimable:382072 slab_unreclaimable:107290 Sep 16 12:11:48 hydra01 kernel: [2090216.385959] mapped:26095 shmem:151 pagetables:9676 bounce:0 Sep 16 12:11:48 hydra01 kernel: [2090216.385959] free_cma:0 Sep 16 12:11:48 hydra01 kernel: [2090216.385962] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:11:48 hydra01 kernel: [2090216.385967] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:11:48 hydra01 kernel: [2090216.385970] Node 0 DMA32 free:146456kB min:6224kB low:7780kB high:9336kB active_anon:439544kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171172kB slab_unreclaimable:47028kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.385975] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:11:48 hydra01 kernel: [2090216.385978] Node 0 Normal free:193168kB min:61324kB low:76652kB high:91984kB active_anon:4737380kB inactive_anon:2534908kB active_file:7384872kB inactive_file:12534024kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:960kB writeback:0kB mapped:100456kB shmem:604kB slab_reclaimable:1357116kB slab_unreclaimable:382132kB kernel_stack:8008kB pagetables:37176kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.385983] lowmem_reserve[]: 0 0 0 0 Sep 16 12:11:48 hydra01 kernel: [2090216.385985] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:11:48 hydra01 kernel: [2090216.385997] Node 0 DMA32: 3005*4kB (UEM) 3179*8kB (UEM) 6033*16kB (UEM) 376*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146524kB Sep 16 12:11:48 hydra01 kernel: [2090216.386006] Node 0 Normal: 22970*4kB (UEMR) 11420*8kB (UEMR) 519*16kB (UER) 13*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 193624kB Sep 16 12:11:48 hydra01 kernel: [2090216.386017] 5778366 total pagecache pages Sep 16 12:11:48 hydra01 kernel: [2090216.386018] 432246 pages in swap cache Sep 16 12:11:48 hydra01 kernel: [2090216.386020] Swap cache stats: add 20773464, delete 20341218, find 12661748/14316137 Sep 16 12:11:48 hydra01 kernel: [2090216.386021] Free swap = 185330288kB Sep 16 12:11:48 hydra01 kernel: [2090216.386022] Total swap = 187617276kB Sep 16 12:11:48 hydra01 kernel: [2090216.492050] 8388592 pages RAM Sep 16 12:11:48 hydra01 kernel: [2090216.492056] 151696 pages reserved Sep 16 12:11:48 hydra01 kernel: [2090216.492061] 8687809 pages shared Sep 16 12:11:48 hydra01 kernel: [2090216.492066] 3434971 pages non-shared Sep 16 12:11:48 hydra01 kernel: [2090216.493975] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:11:48 hydra01 kernel: [2090216.493979] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:11:48 hydra01 kernel: [2090216.493980] Call Trace: Sep 16 12:11:48 hydra01 kernel: [2090216.493988] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:11:48 hydra01 kernel: [2090216.493993] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:11:48 hydra01 kernel: [2090216.493996] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:11:48 hydra01 kernel: [2090216.494000] [] alloc_pages_current+0xb8/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.494003] [] __get_free_pages+0xe/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.494008] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:11:48 hydra01 kernel: [2090216.494011] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:11:48 hydra01 kernel: [2090216.494014] [] __kmalloc+0x161/0x170 Sep 16 12:11:48 hydra01 kernel: [2090216.494018] [] ? amiga_partition+0x2c3/0x4d0 Sep 16 12:11:48 hydra01 kernel: [2090216.494022] [] ? get_iowait_time+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.494024] [] stat_open+0x56/0xc0 Sep 16 12:11:48 hydra01 kernel: [2090216.494028] [] proc_reg_open+0xa0/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.494032] [] ? seq_release_private+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.494035] [] do_dentry_open+0x203/0x290 Sep 16 12:11:48 hydra01 kernel: [2090216.494038] [] ? init_once+0x20/0x20 Sep 16 12:11:48 hydra01 kernel: [2090216.494040] [] vfs_open+0x49/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.494043] [] do_last+0x296/0xea0 Sep 16 12:11:48 hydra01 kernel: [2090216.494045] [] ? inode_permission+0x18/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.494048] [] ? link_path_walk+0x6b/0x880 Sep 16 12:11:48 hydra01 kernel: [2090216.494050] [] path_openat+0xb3/0x4b0 Sep 16 12:11:48 hydra01 kernel: [2090216.494054] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:11:48 hydra01 kernel: [2090216.494058] [] ? unmap_region+0xdb/0x120 Sep 16 12:11:48 hydra01 kernel: [2090216.494060] [] do_filp_open+0x41/0xa0 Sep 16 12:11:48 hydra01 kernel: [2090216.494063] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:11:48 hydra01 kernel: [2090216.494066] [] do_sys_open+0xf3/0x230 Sep 16 12:11:48 hydra01 kernel: [2090216.494068] [] sys_open+0x21/0x30 Sep 16 12:11:48 hydra01 kernel: [2090216.494072] [] system_call_fastpath+0x1a/0x1f Sep 16 12:11:48 hydra01 kernel: [2090216.494073] Mem-Info: Sep 16 12:11:48 hydra01 kernel: [2090216.494074] Node 0 DMA per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.494077] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494078] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494080] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494081] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494082] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494084] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494085] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494086] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494088] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494089] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494090] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494092] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494093] Node 0 DMA32 per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.494095] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494096] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494098] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494099] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494100] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494102] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494103] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494104] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494106] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494107] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494109] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494110] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494111] Node 0 Normal per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.494113] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494114] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494115] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494117] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494118] CPU 4: hi: 186, btch: 31 usd: 30 Sep 16 12:11:48 hydra01 kernel: [2090216.494120] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494121] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494122] CPU 7: hi: 186, btch: 31 usd: 30 Sep 16 12:11:48 hydra01 kernel: [2090216.494124] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494125] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494127] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494128] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494132] active_anon:1293939 inactive_anon:832044 isolated_anon:0 Sep 16 12:11:48 hydra01 kernel: [2090216.494132] active_file:1870017 inactive_file:3475948 isolated_file:0 Sep 16 12:11:48 hydra01 kernel: [2090216.494132] unevictable:0 dirty:245 writeback:0 unstable:0 Sep 16 12:11:48 hydra01 kernel: [2090216.494132] free:89483 slab_reclaimable:382072 slab_unreclaimable:107290 Sep 16 12:11:48 hydra01 kernel: [2090216.494132] mapped:26095 shmem:151 pagetables:9567 bounce:0 Sep 16 12:11:48 hydra01 kernel: [2090216.494132] free_cma:0 Sep 16 12:11:48 hydra01 kernel: [2090216.494135] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:11:48 hydra01 kernel: [2090216.494140] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:11:48 hydra01 kernel: [2090216.494143] Node 0 DMA32 free:146456kB min:6224kB low:7780kB high:9336kB active_anon:439544kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171172kB slab_unreclaimable:47028kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.494148] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:11:48 hydra01 kernel: [2090216.494151] Node 0 Normal free:195632kB min:61324kB low:76652kB high:91984kB active_anon:4736212kB inactive_anon:2534908kB active_file:7384872kB inactive_file:12534024kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:960kB writeback:0kB mapped:100456kB shmem:604kB slab_reclaimable:1357116kB slab_unreclaimable:382132kB kernel_stack:8008kB pagetables:36740kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.494155] lowmem_reserve[]: 0 0 0 0 Sep 16 12:11:48 hydra01 kernel: [2090216.494158] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:11:48 hydra01 kernel: [2090216.494169] Node 0 DMA32: 3005*4kB (UEM) 3179*8kB (UEM) 6033*16kB (UEM) 376*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146524kB Sep 16 12:11:48 hydra01 kernel: [2090216.494179] Node 0 Normal: 23669*4kB (UEMR) 11407*8kB (UEMR) 520*16kB (UER) 13*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 196332kB Sep 16 12:11:48 hydra01 kernel: [2090216.494189] 5778366 total pagecache pages Sep 16 12:11:48 hydra01 kernel: [2090216.494190] 432246 pages in swap cache Sep 16 12:11:48 hydra01 kernel: [2090216.494192] Swap cache stats: add 20773464, delete 20341218, find 12661748/14316137 Sep 16 12:11:48 hydra01 kernel: [2090216.494193] Free swap = 185330288kB Sep 16 12:11:48 hydra01 kernel: [2090216.494194] Total swap = 187617276kB Sep 16 12:11:48 hydra01 kernel: [2090216.600190] 8388592 pages RAM Sep 16 12:11:48 hydra01 kernel: [2090216.600196] 151696 pages reserved Sep 16 12:11:48 hydra01 kernel: [2090216.600201] 8687802 pages shared Sep 16 12:11:48 hydra01 kernel: [2090216.600206] 3434961 pages non-shared Sep 16 12:11:48 hydra01 kernel: [2090216.863356] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:11:48 hydra01 kernel: [2090216.863365] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:11:48 hydra01 kernel: [2090216.863368] Call Trace: Sep 16 12:11:48 hydra01 kernel: [2090216.863381] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:11:48 hydra01 kernel: [2090216.863390] [] ? on_each_cpu_mask+0x37/0x70 Sep 16 12:11:48 hydra01 kernel: [2090216.863396] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:11:48 hydra01 kernel: [2090216.863403] [] alloc_pages_current+0xb8/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.863409] [] __get_free_pages+0xe/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.863416] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:11:48 hydra01 kernel: [2090216.863422] [] __kmalloc+0x161/0x170 Sep 16 12:11:48 hydra01 kernel: [2090216.863428] [] ? get_iowait_time+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.863433] [] stat_open+0x56/0xc0 Sep 16 12:11:48 hydra01 kernel: [2090216.863439] [] proc_reg_open+0xa0/0x180 Sep 16 12:11:48 hydra01 kernel: [2090216.863446] [] ? seq_release_private+0x60/0x60 Sep 16 12:11:48 hydra01 kernel: [2090216.863452] [] do_dentry_open+0x203/0x290 Sep 16 12:11:48 hydra01 kernel: [2090216.863457] [] ? init_once+0x20/0x20 Sep 16 12:11:48 hydra01 kernel: [2090216.863461] [] vfs_open+0x49/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.863466] [] do_last+0x296/0xea0 Sep 16 12:11:48 hydra01 kernel: [2090216.863471] [] ? inode_permission+0x18/0x50 Sep 16 12:11:48 hydra01 kernel: [2090216.863476] [] ? link_path_walk+0x6b/0x880 Sep 16 12:11:48 hydra01 kernel: [2090216.863481] [] path_openat+0xb3/0x4b0 Sep 16 12:11:48 hydra01 kernel: [2090216.863486] [] do_filp_open+0x41/0xa0 Sep 16 12:11:48 hydra01 kernel: [2090216.863491] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:11:48 hydra01 kernel: [2090216.863496] [] do_sys_open+0xf3/0x230 Sep 16 12:11:48 hydra01 kernel: [2090216.863501] [] ? sys_select+0xcb/0x110 Sep 16 12:11:48 hydra01 kernel: [2090216.863506] [] sys_open+0x21/0x30 Sep 16 12:11:48 hydra01 kernel: [2090216.863512] [] system_call_fastpath+0x1a/0x1f Sep 16 12:11:48 hydra01 kernel: [2090216.863514] Mem-Info: Sep 16 12:11:48 hydra01 kernel: [2090216.863517] Node 0 DMA per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.863522] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863525] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863527] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863530] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863532] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863535] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863537] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863540] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863542] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863545] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863548] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863551] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863553] Node 0 DMA32 per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.863557] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863559] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863562] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863565] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863567] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863570] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863572] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863575] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863578] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863580] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863583] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863585] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863587] Node 0 Normal per-cpu: Sep 16 12:11:48 hydra01 kernel: [2090216.863591] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863593] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863596] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863599] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863601] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863604] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863606] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863609] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863612] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863614] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863617] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863620] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863627] active_anon:1293939 inactive_anon:832044 isolated_anon:0 Sep 16 12:11:48 hydra01 kernel: [2090216.863627] active_file:1870017 inactive_file:3475948 isolated_file:0 Sep 16 12:11:48 hydra01 kernel: [2090216.863627] unevictable:0 dirty:136 writeback:0 unstable:0 Sep 16 12:11:48 hydra01 kernel: [2090216.863627] free:89483 slab_reclaimable:382072 slab_unreclaimable:107290 Sep 16 12:11:48 hydra01 kernel: [2090216.863627] mapped:26095 shmem:151 pagetables:9567 bounce:0 Sep 16 12:11:48 hydra01 kernel: [2090216.863627] free_cma:0 Sep 16 12:11:48 hydra01 kernel: [2090216.863633] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:11:48 hydra01 kernel: [2090216.863642] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:11:48 hydra01 kernel: [2090216.863648] Node 0 DMA32 free:146456kB min:6224kB low:7780kB high:9336kB active_anon:439544kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171172kB slab_unreclaimable:47028kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.863658] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:11:48 hydra01 kernel: [2090216.863663] Node 0 Normal free:195632kB min:61324kB low:76652kB high:91984kB active_anon:4736212kB inactive_anon:2534908kB active_file:7384872kB inactive_file:12534024kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:524kB writeback:0kB mapped:100456kB shmem:604kB slab_reclaimable:1357116kB slab_unreclaimable:382132kB kernel_stack:8008kB pagetables:36740kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:11:48 hydra01 kernel: [2090216.863672] lowmem_reserve[]: 0 0 0 0 Sep 16 12:11:48 hydra01 kernel: [2090216.863677] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:11:48 hydra01 kernel: [2090216.863699] Node 0 DMA32: 3005*4kB (UEM) 3179*8kB (UEM) 6033*16kB (UEM) 376*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146524kB Sep 16 12:11:48 hydra01 kernel: [2090216.863717] Node 0 Normal: 23730*4kB (UEMR) 11423*8kB (UEMR) 520*16kB (UER) 13*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 196704kB Sep 16 12:11:48 hydra01 kernel: [2090216.863737] 5778366 total pagecache pages Sep 16 12:11:48 hydra01 kernel: [2090216.863740] 432246 pages in swap cache Sep 16 12:11:48 hydra01 kernel: [2090216.863743] Swap cache stats: add 20773464, delete 20341218, find 12661748/14316137 Sep 16 12:11:48 hydra01 kernel: [2090216.863745] Free swap = 185330288kB Sep 16 12:11:48 hydra01 kernel: [2090216.863747] Total swap = 187617276kB Sep 16 12:11:48 hydra01 kernel: [2090217.057336] 8388592 pages RAM Sep 16 12:11:48 hydra01 kernel: [2090217.057348] 151696 pages reserved Sep 16 12:11:48 hydra01 kernel: [2090217.057358] 8687555 pages shared Sep 16 12:11:48 hydra01 kernel: [2090217.057368] 3434943 pages non-shared Sep 16 12:12:23 hydra01 kernel: [2090251.850155] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:12:23 hydra01 kernel: [2090251.850164] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:12:23 hydra01 kernel: [2090251.850167] Call Trace: Sep 16 12:12:23 hydra01 kernel: [2090251.850180] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:12:23 hydra01 kernel: [2090251.850187] [] ? drain_local_pages+0x16/0x20 Sep 16 12:12:23 hydra01 kernel: [2090251.850195] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:12:23 hydra01 kernel: [2090251.850201] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:12:23 hydra01 kernel: [2090251.850208] [] alloc_pages_current+0xb8/0x180 Sep 16 12:12:23 hydra01 kernel: [2090251.850214] [] __get_free_pages+0xe/0x50 Sep 16 12:12:23 hydra01 kernel: [2090251.850221] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:12:23 hydra01 kernel: [2090251.850227] [] __kmalloc+0x161/0x170 Sep 16 12:12:23 hydra01 kernel: [2090251.850233] [] ? get_iowait_time+0x60/0x60 Sep 16 12:12:23 hydra01 kernel: [2090251.850238] [] stat_open+0x56/0xc0 Sep 16 12:12:23 hydra01 kernel: [2090251.850244] [] proc_reg_open+0xa0/0x180 Sep 16 12:12:23 hydra01 kernel: [2090251.850252] [] ? seq_release_private+0x60/0x60 Sep 16 12:12:23 hydra01 kernel: [2090251.850257] [] do_dentry_open+0x203/0x290 Sep 16 12:12:23 hydra01 kernel: [2090251.850262] [] ? init_once+0x20/0x20 Sep 16 12:12:23 hydra01 kernel: [2090251.850299] [] vfs_open+0x49/0x50 Sep 16 12:12:23 hydra01 kernel: [2090251.850305] [] do_last+0x296/0xea0 Sep 16 12:12:23 hydra01 kernel: [2090251.850310] [] ? inode_permission+0x18/0x50 Sep 16 12:12:23 hydra01 kernel: [2090251.850314] [] ? link_path_walk+0x6b/0x880 Sep 16 12:12:23 hydra01 kernel: [2090251.850335] [] path_openat+0xb3/0x4b0 Sep 16 12:12:23 hydra01 kernel: [2090251.850353] [] do_filp_open+0x41/0xa0 Sep 16 12:12:23 hydra01 kernel: [2090251.850367] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:12:23 hydra01 kernel: [2090251.850372] [] do_sys_open+0xf3/0x230 Sep 16 12:12:23 hydra01 kernel: [2090251.850378] [] ? sys_select+0xcb/0x110 Sep 16 12:12:23 hydra01 kernel: [2090251.850382] [] sys_open+0x21/0x30 Sep 16 12:12:23 hydra01 kernel: [2090251.850388] [] system_call_fastpath+0x1a/0x1f Sep 16 12:12:23 hydra01 kernel: [2090251.850391] Mem-Info: Sep 16 12:12:23 hydra01 kernel: [2090251.850394] Node 0 DMA per-cpu: Sep 16 12:12:23 hydra01 kernel: [2090251.850399] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850401] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850404] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850407] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850409] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850412] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850414] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850417] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850420] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850422] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850425] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850428] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850430] Node 0 DMA32 per-cpu: Sep 16 12:12:23 hydra01 kernel: [2090251.850434] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850436] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850439] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850442] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850444] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850447] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850449] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850452] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850455] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850457] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850460] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850463] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850465] Node 0 Normal per-cpu: Sep 16 12:12:23 hydra01 kernel: [2090251.850468] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850471] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850474] CPU 2: hi: 186, btch: 31 usd: 171 Sep 16 12:12:23 hydra01 kernel: [2090251.850476] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850479] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850482] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850484] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850487] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850490] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850492] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850495] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850498] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850514] active_anon:1294165 inactive_anon:832169 isolated_anon:0 Sep 16 12:12:23 hydra01 kernel: [2090251.850514] active_file:1870046 inactive_file:3482397 isolated_file:0 Sep 16 12:12:23 hydra01 kernel: [2090251.850514] unevictable:0 dirty:117 writeback:0 unstable:0 Sep 16 12:12:23 hydra01 kernel: [2090251.850514] free:82475 slab_reclaimable:382018 slab_unreclaimable:107246 Sep 16 12:12:23 hydra01 kernel: [2090251.850514] mapped:25949 shmem:151 pagetables:9563 bounce:0 Sep 16 12:12:23 hydra01 kernel: [2090251.850514] free_cma:0 Sep 16 12:12:23 hydra01 kernel: [2090251.850527] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:12:23 hydra01 kernel: [2090251.850549] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:12:23 hydra01 kernel: [2090251.850591] Node 0 DMA32 free:146252kB min:6224kB low:7780kB high:9336kB active_anon:439532kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171156kB slab_unreclaimable:47092kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:23 hydra01 kernel: [2090251.850619] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:12:23 hydra01 kernel: [2090251.850625] Node 0 Normal free:168388kB min:61324kB low:76652kB high:91984kB active_anon:4737128kB inactive_anon:2535408kB active_file:7384988kB inactive_file:12559436kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:468kB writeback:0kB mapped:99872kB shmem:604kB slab_reclaimable:1356916kB slab_unreclaimable:381892kB kernel_stack:8008kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:23 hydra01 kernel: [2090251.850634] lowmem_reserve[]: 0 0 0 0 Sep 16 12:12:23 hydra01 kernel: [2090251.850639] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:12:23 hydra01 kernel: [2090251.850661] Node 0 DMA32: 3006*4kB (UEM) 3182*8kB (UEM) 6027*16kB (UEM) 377*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146488kB Sep 16 12:12:23 hydra01 kernel: [2090251.850679] Node 0 Normal: 16947*4kB (UEMR) 11623*8kB (UEMR) 506*16kB (UER) 8*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170788kB Sep 16 12:12:23 hydra01 kernel: [2090251.850700] 5784614 total pagecache pages Sep 16 12:12:23 hydra01 kernel: [2090251.850703] 432235 pages in swap cache Sep 16 12:12:23 hydra01 kernel: [2090251.850706] Swap cache stats: add 20773464, delete 20341229, find 12661761/14316150 Sep 16 12:12:23 hydra01 kernel: [2090251.850708] Free swap = 185330332kB Sep 16 12:12:23 hydra01 kernel: [2090251.850710] Total swap = 187617276kB Sep 16 12:12:23 hydra01 kernel: [2090252.035494] 8388592 pages RAM Sep 16 12:12:23 hydra01 kernel: [2090252.035499] 151696 pages reserved Sep 16 12:12:23 hydra01 kernel: [2090252.035502] 8684491 pages shared Sep 16 12:12:23 hydra01 kernel: [2090252.035504] 3444536 pages non-shared Sep 16 12:12:28 hydra01 kernel: [2090256.850812] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:12:28 hydra01 kernel: [2090256.850822] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:12:28 hydra01 kernel: [2090256.850826] Call Trace: Sep 16 12:12:28 hydra01 kernel: [2090256.850842] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:12:28 hydra01 kernel: [2090256.850852] [] ? drain_local_pages+0x16/0x20 Sep 16 12:12:28 hydra01 kernel: [2090256.850862] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:12:28 hydra01 kernel: [2090256.850870] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:12:28 hydra01 kernel: [2090256.850880] [] alloc_pages_current+0xb8/0x180 Sep 16 12:12:28 hydra01 kernel: [2090256.850888] [] __get_free_pages+0xe/0x50 Sep 16 12:12:28 hydra01 kernel: [2090256.850899] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:12:28 hydra01 kernel: [2090256.850907] [] __kmalloc+0x161/0x170 Sep 16 12:12:28 hydra01 kernel: [2090256.850916] [] ? get_iowait_time+0x60/0x60 Sep 16 12:12:28 hydra01 kernel: [2090256.850922] [] stat_open+0x56/0xc0 Sep 16 12:12:28 hydra01 kernel: [2090256.850931] [] proc_reg_open+0xa0/0x180 Sep 16 12:12:28 hydra01 kernel: [2090256.850941] [] ? seq_release_private+0x60/0x60 Sep 16 12:12:28 hydra01 kernel: [2090256.850949] [] do_dentry_open+0x203/0x290 Sep 16 12:12:28 hydra01 kernel: [2090256.850957] [] ? init_once+0x20/0x20 Sep 16 12:12:28 hydra01 kernel: [2090256.850963] [] vfs_open+0x49/0x50 Sep 16 12:12:28 hydra01 kernel: [2090256.850970] [] do_last+0x296/0xea0 Sep 16 12:12:28 hydra01 kernel: [2090256.850977] [] ? inode_permission+0x18/0x50 Sep 16 12:12:28 hydra01 kernel: [2090256.851018] [] ? link_path_walk+0x6b/0x880 Sep 16 12:12:28 hydra01 kernel: [2090256.851042] [] ? touch_atime+0x81/0x160 Sep 16 12:12:28 hydra01 kernel: [2090256.851055] [] path_openat+0xb3/0x4b0 Sep 16 12:12:28 hydra01 kernel: [2090256.851063] [] do_filp_open+0x41/0xa0 Sep 16 12:12:28 hydra01 kernel: [2090256.851079] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:12:28 hydra01 kernel: [2090256.851086] [] do_sys_open+0xf3/0x230 Sep 16 12:12:28 hydra01 kernel: [2090256.851093] [] ? sys_select+0xcb/0x110 Sep 16 12:12:28 hydra01 kernel: [2090256.851100] [] sys_open+0x21/0x30 Sep 16 12:12:28 hydra01 kernel: [2090256.851109] [] system_call_fastpath+0x1a/0x1f Sep 16 12:12:28 hydra01 kernel: [2090256.851113] Mem-Info: Sep 16 12:12:28 hydra01 kernel: [2090256.851117] Node 0 DMA per-cpu: Sep 16 12:12:28 hydra01 kernel: [2090256.851124] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851128] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851133] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851137] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851140] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851144] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851148] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851153] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851157] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851162] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851167] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851171] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851175] Node 0 DMA32 per-cpu: Sep 16 12:12:28 hydra01 kernel: [2090256.851181] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851186] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851190] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851193] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851197] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851202] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851206] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851211] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851215] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851220] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851230] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851242] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851253] Node 0 Normal per-cpu: Sep 16 12:12:28 hydra01 kernel: [2090256.851272] CPU 0: hi: 186, btch: 31 usd: 170 Sep 16 12:12:28 hydra01 kernel: [2090256.851284] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851296] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851309] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851315] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851320] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851325] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851330] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851334] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851339] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851343] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851347] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851358] active_anon:1294172 inactive_anon:832167 isolated_anon:0 Sep 16 12:12:28 hydra01 kernel: [2090256.851358] active_file:1870050 inactive_file:3481264 isolated_file:0 Sep 16 12:12:28 hydra01 kernel: [2090256.851358] unevictable:0 dirty:76 writeback:0 unstable:0 Sep 16 12:12:28 hydra01 kernel: [2090256.851358] free:83921 slab_reclaimable:382012 slab_unreclaimable:107230 Sep 16 12:12:28 hydra01 kernel: [2090256.851358] mapped:25999 shmem:151 pagetables:9563 bounce:0 Sep 16 12:12:28 hydra01 kernel: [2090256.851358] free_cma:0 Sep 16 12:12:28 hydra01 kernel: [2090256.851368] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:12:28 hydra01 kernel: [2090256.851382] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:12:28 hydra01 kernel: [2090256.851414] Node 0 DMA32 free:146424kB min:6224kB low:7780kB high:9336kB active_anon:439532kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171156kB slab_unreclaimable:47028kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:28 hydra01 kernel: [2090256.851429] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:12:28 hydra01 kernel: [2090256.851437] Node 0 Normal free:173416kB min:61324kB low:76652kB high:91984kB active_anon:4737156kB inactive_anon:2535400kB active_file:7385004kB inactive_file:12555288kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:304kB writeback:0kB mapped:100072kB shmem:604kB slab_reclaimable:1356892kB slab_unreclaimable:381892kB kernel_stack:8008kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:12:28 hydra01 kernel: [2090256.851459] lowmem_reserve[]: 0 0 0 0 Sep 16 12:12:28 hydra01 kernel: [2090256.851468] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:12:28 hydra01 kernel: [2090256.851502] Node 0 DMA32: 3006*4kB (UEM) 3182*8kB (UEM) 6027*16kB (UEM) 379*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146552kB Sep 16 12:12:28 hydra01 kernel: [2090256.851529] Node 0 Normal: 17698*4kB (UEMR) 11637*8kB (UEMR) 539*16kB (UEMR) 18*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 174752kB Sep 16 12:12:28 hydra01 kernel: [2090256.851559] 5783638 total pagecache pages Sep 16 12:12:28 hydra01 kernel: [2090256.851562] 432228 pages in swap cache Sep 16 12:12:28 hydra01 kernel: [2090256.851568] Swap cache stats: add 20773464, delete 20341236, find 12661768/14316157 Sep 16 12:12:28 hydra01 kernel: [2090256.851572] Free swap = 185330360kB Sep 16 12:12:28 hydra01 kernel: [2090256.851575] Total swap = 187617276kB Sep 16 12:12:28 hydra01 kernel: [2090257.043072] 8388592 pages RAM Sep 16 12:12:28 hydra01 kernel: [2090257.043082] 151696 pages reserved Sep 16 12:12:28 hydra01 kernel: [2090257.043085] 8683438 pages shared Sep 16 12:12:28 hydra01 kernel: [2090257.043088] 3443719 pages non-shared Sep 16 12:12:33 hydra01 kernel: [2090261.852454] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:12:33 hydra01 kernel: [2090261.852464] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:12:33 hydra01 kernel: [2090261.852466] Call Trace: Sep 16 12:12:33 hydra01 kernel: [2090261.852480] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:12:33 hydra01 kernel: [2090261.852487] [] ? drain_local_pages+0x16/0x20 Sep 16 12:12:33 hydra01 kernel: [2090261.852495] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:12:33 hydra01 kernel: [2090261.852501] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:12:33 hydra01 kernel: [2090261.852508] [] alloc_pages_current+0xb8/0x180 Sep 16 12:12:33 hydra01 kernel: [2090261.852514] [] __get_free_pages+0xe/0x50 Sep 16 12:12:33 hydra01 kernel: [2090261.852521] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:12:33 hydra01 kernel: [2090261.852527] [] __kmalloc+0x161/0x170 Sep 16 12:12:33 hydra01 kernel: [2090261.852533] [] ? get_iowait_time+0x60/0x60 Sep 16 12:12:33 hydra01 kernel: [2090261.852538] [] stat_open+0x56/0xc0 Sep 16 12:12:33 hydra01 kernel: [2090261.852544] [] proc_reg_open+0xa0/0x180 Sep 16 12:12:33 hydra01 kernel: [2090261.852551] [] ? seq_release_private+0x60/0x60 Sep 16 12:12:33 hydra01 kernel: [2090261.852557] [] do_dentry_open+0x203/0x290 Sep 16 12:12:33 hydra01 kernel: [2090261.852562] [] ? init_once+0x20/0x20 Sep 16 12:12:33 hydra01 kernel: [2090261.852566] [] vfs_open+0x49/0x50 Sep 16 12:12:33 hydra01 kernel: [2090261.852572] [] do_last+0x296/0xea0 Sep 16 12:12:33 hydra01 kernel: [2090261.852576] [] ? inode_permission+0x18/0x50 Sep 16 12:12:33 hydra01 kernel: [2090261.852581] [] ? link_path_walk+0x6b/0x880 Sep 16 12:12:33 hydra01 kernel: [2090261.852586] [] path_openat+0xb3/0x4b0 Sep 16 12:12:33 hydra01 kernel: [2090261.852591] [] do_filp_open+0x41/0xa0 Sep 16 12:12:33 hydra01 kernel: [2090261.852597] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:12:33 hydra01 kernel: [2090261.852602] [] do_sys_open+0xf3/0x230 Sep 16 12:12:33 hydra01 kernel: [2090261.852607] [] ? sys_select+0xcb/0x110 Sep 16 12:12:33 hydra01 kernel: [2090261.852611] [] sys_open+0x21/0x30 Sep 16 12:12:33 hydra01 kernel: [2090261.852617] [] system_call_fastpath+0x1a/0x1f Sep 16 12:12:33 hydra01 kernel: [2090261.852620] Mem-Info: Sep 16 12:12:33 hydra01 kernel: [2090261.852622] Node 0 DMA per-cpu: Sep 16 12:12:33 hydra01 kernel: [2090261.852627] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852630] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852633] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852635] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852638] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852640] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852643] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852646] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852648] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852651] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852653] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852656] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852658] Node 0 DMA32 per-cpu: Sep 16 12:12:33 hydra01 kernel: [2090261.852662] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852665] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852667] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852670] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852673] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852675] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852678] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852681] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852683] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852686] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852688] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852691] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852693] Node 0 Normal per-cpu: Sep 16 12:12:33 hydra01 kernel: [2090261.852697] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852699] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852702] CPU 2: hi: 186, btch: 31 usd: 186 Sep 16 12:12:33 hydra01 kernel: [2090261.852705] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852708] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852710] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852713] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852715] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852718] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852721] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852723] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852726] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852733] active_anon:1294262 inactive_anon:832170 isolated_anon:0 Sep 16 12:12:33 hydra01 kernel: [2090261.852733] active_file:1870058 inactive_file:3482781 isolated_file:0 Sep 16 12:12:33 hydra01 kernel: [2090261.852733] unevictable:0 dirty:118 writeback:0 unstable:0 Sep 16 12:12:33 hydra01 kernel: [2090261.852733] free:82260 slab_reclaimable:381980 slab_unreclaimable:107227 Sep 16 12:12:33 hydra01 kernel: [2090261.852733] mapped:26106 shmem:151 pagetables:9705 bounce:0 Sep 16 12:12:33 hydra01 kernel: [2090261.852733] free_cma:0 Sep 16 12:12:33 hydra01 kernel: [2090261.852740] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:12:33 hydra01 kernel: [2090261.852748] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:12:33 hydra01 kernel: [2090261.852755] Node 0 DMA32 free:146520kB min:6224kB low:7780kB high:9336kB active_anon:439528kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171156kB slab_unreclaimable:47028kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:33 hydra01 kernel: [2090261.852764] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:12:33 hydra01 kernel: [2090261.852769] Node 0 Normal free:166968kB min:61324kB low:76652kB high:91984kB active_anon:4737520kB inactive_anon:2535412kB active_file:7385036kB inactive_file:12560972kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:464kB writeback:0kB mapped:100500kB shmem:604kB slab_reclaimable:1356764kB slab_unreclaimable:381880kB kernel_stack:8008kB pagetables:37292kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:33 hydra01 kernel: [2090261.852779] lowmem_reserve[]: 0 0 0 0 Sep 16 12:12:33 hydra01 kernel: [2090261.852784] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:12:33 hydra01 kernel: [2090261.852805] Node 0 DMA32: 3007*4kB (UEM) 3182*8kB (UEM) 6027*16kB (UEM) 383*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146684kB Sep 16 12:12:33 hydra01 kernel: [2090261.852823] Node 0 Normal: 16209*4kB (UEMR) 11610*8kB (UEMR) 548*16kB (UEMR) 19*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 168756kB Sep 16 12:12:33 hydra01 kernel: [2090261.852843] 5785071 total pagecache pages Sep 16 12:12:33 hydra01 kernel: [2090261.852845] 432226 pages in swap cache Sep 16 12:12:33 hydra01 kernel: [2090261.852849] Swap cache stats: add 20773464, delete 20341238, find 12661771/14316160 Sep 16 12:12:33 hydra01 kernel: [2090261.852851] Free swap = 185330368kB Sep 16 12:12:33 hydra01 kernel: [2090261.852853] Total swap = 187617276kB Sep 16 12:12:33 hydra01 kernel: [2090262.046743] 8388592 pages RAM Sep 16 12:12:33 hydra01 kernel: [2090262.046759] 151696 pages reserved Sep 16 12:12:33 hydra01 kernel: [2090262.046772] 8682923 pages shared Sep 16 12:12:33 hydra01 kernel: [2090262.046776] 3445571 pages non-shared Sep 16 12:12:34 hydra01 kernel: [2090262.335687] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:12:34 hydra01 kernel: [2090262.335695] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:12:34 hydra01 kernel: [2090262.335698] Call Trace: Sep 16 12:12:34 hydra01 kernel: [2090262.335712] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:12:34 hydra01 kernel: [2090262.335718] [] ? drain_local_pages+0x16/0x20 Sep 16 12:12:34 hydra01 kernel: [2090262.335726] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:12:34 hydra01 kernel: [2090262.335732] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:12:34 hydra01 kernel: [2090262.335739] [] alloc_pages_current+0xb8/0x180 Sep 16 12:12:34 hydra01 kernel: [2090262.335745] [] __get_free_pages+0xe/0x50 Sep 16 12:12:34 hydra01 kernel: [2090262.335752] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:12:34 hydra01 kernel: [2090262.335758] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:12:34 hydra01 kernel: [2090262.335763] [] __kmalloc+0x161/0x170 Sep 16 12:12:34 hydra01 kernel: [2090262.335770] [] ? get_iowait_time+0x60/0x60 Sep 16 12:12:34 hydra01 kernel: [2090262.335775] [] stat_open+0x56/0xc0 Sep 16 12:12:34 hydra01 kernel: [2090262.335781] [] proc_reg_open+0xa0/0x180 Sep 16 12:12:34 hydra01 kernel: [2090262.335789] [] ? seq_release_private+0x60/0x60 Sep 16 12:12:34 hydra01 kernel: [2090262.335794] [] do_dentry_open+0x203/0x290 Sep 16 12:12:34 hydra01 kernel: [2090262.335799] [] ? init_once+0x20/0x20 Sep 16 12:12:34 hydra01 kernel: [2090262.335803] [] vfs_open+0x49/0x50 Sep 16 12:12:34 hydra01 kernel: [2090262.335809] [] do_last+0x296/0xea0 Sep 16 12:12:34 hydra01 kernel: [2090262.335813] [] ? inode_permission+0x18/0x50 Sep 16 12:12:34 hydra01 kernel: [2090262.335818] [] ? link_path_walk+0x6b/0x880 Sep 16 12:12:34 hydra01 kernel: [2090262.335823] [] path_openat+0xb3/0x4b0 Sep 16 12:12:34 hydra01 kernel: [2090262.335829] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:12:34 hydra01 kernel: [2090262.335835] [] ? unmap_region+0xdb/0x120 Sep 16 12:12:34 hydra01 kernel: [2090262.335840] [] do_filp_open+0x41/0xa0 Sep 16 12:12:34 hydra01 kernel: [2090262.335846] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:12:34 hydra01 kernel: [2090262.335851] [] do_sys_open+0xf3/0x230 Sep 16 12:12:34 hydra01 kernel: [2090262.335855] [] sys_open+0x21/0x30 Sep 16 12:12:34 hydra01 kernel: [2090262.335861] [] system_call_fastpath+0x1a/0x1f Sep 16 12:12:34 hydra01 kernel: [2090262.335864] Mem-Info: Sep 16 12:12:34 hydra01 kernel: [2090262.335867] Node 0 DMA per-cpu: Sep 16 12:12:34 hydra01 kernel: [2090262.335872] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335874] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335877] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335880] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335882] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335885] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335887] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335890] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335892] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335895] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335898] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335900] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335902] Node 0 DMA32 per-cpu: Sep 16 12:12:34 hydra01 kernel: [2090262.335906] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335909] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335912] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335914] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335917] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335920] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335922] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335925] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335927] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335930] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335933] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335935] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335937] Node 0 Normal per-cpu: Sep 16 12:12:34 hydra01 kernel: [2090262.335941] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335943] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335946] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335948] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335951] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335954] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335956] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335959] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335962] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335964] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335967] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335969] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:34 hydra01 kernel: [2090262.335977] active_anon:1294105 inactive_anon:832198 isolated_anon:0 Sep 16 12:12:34 hydra01 kernel: [2090262.335977] active_file:1870060 inactive_file:3482366 isolated_file:0 Sep 16 12:12:34 hydra01 kernel: [2090262.335977] unevictable:0 dirty:68 writeback:0 unstable:0 Sep 16 12:12:34 hydra01 kernel: [2090262.335977] free:83343 slab_reclaimable:381980 slab_unreclaimable:107195 Sep 16 12:12:34 hydra01 kernel: [2090262.335977] mapped:26119 shmem:151 pagetables:9617 bounce:0 Sep 16 12:12:34 hydra01 kernel: [2090262.335977] free_cma:0 Sep 16 12:12:34 hydra01 kernel: [2090262.335983] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:12:34 hydra01 kernel: [2090262.335992] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:12:34 hydra01 kernel: [2090262.335998] Node 0 DMA32 free:146620kB min:6224kB low:7780kB high:9336kB active_anon:439528kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171156kB slab_unreclaimable:46900kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:34 hydra01 kernel: [2090262.336008] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:12:34 hydra01 kernel: [2090262.336013] Node 0 Normal free:170908kB min:61324kB low:76652kB high:91984kB active_anon:4736892kB inactive_anon:2535524kB active_file:7385044kB inactive_file:12559696kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:272kB writeback:0kB mapped:100552kB shmem:604kB slab_reclaimable:1356764kB slab_unreclaimable:381880kB kernel_stack:7992kB pagetables:36940kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:34 hydra01 kernel: [2090262.336022] lowmem_reserve[]: 0 0 0 0 Sep 16 12:12:34 hydra01 kernel: [2090262.336027] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:12:34 hydra01 kernel: [2090262.336048] Node 0 DMA32: 3007*4kB (UEM) 3182*8kB (UEM) 6027*16kB (UEM) 383*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146684kB Sep 16 12:12:34 hydra01 kernel: [2090262.336066] Node 0 Normal: 16824*4kB (UEMR) 11609*8kB (UEMR) 552*16kB (UEMR) 19*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 171272kB Sep 16 12:12:34 hydra01 kernel: [2090262.336086] 5784820 total pagecache pages Sep 16 12:12:34 hydra01 kernel: [2090262.336089] 432226 pages in swap cache Sep 16 12:12:34 hydra01 kernel: [2090262.336092] Swap cache stats: add 20773464, delete 20341238, find 12661771/14316160 Sep 16 12:12:34 hydra01 kernel: [2090262.336095] Free swap = 185330368kB Sep 16 12:12:34 hydra01 kernel: [2090262.336096] Total swap = 187617276kB Sep 16 12:12:34 hydra01 kernel: [2090262.529673] 8388592 pages RAM Sep 16 12:12:34 hydra01 kernel: [2090262.529677] 151696 pages reserved Sep 16 12:12:34 hydra01 kernel: [2090262.529685] 8683722 pages shared Sep 16 12:12:34 hydra01 kernel: [2090262.529695] 3445304 pages non-shared Sep 16 12:12:38 hydra01 kernel: [2090266.849258] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:12:38 hydra01 kernel: [2090266.849267] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:12:38 hydra01 kernel: [2090266.849269] Call Trace: Sep 16 12:12:38 hydra01 kernel: [2090266.849283] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:12:38 hydra01 kernel: [2090266.849289] [] ? drain_local_pages+0x16/0x20 Sep 16 12:12:38 hydra01 kernel: [2090266.849298] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:12:38 hydra01 kernel: [2090266.849304] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:12:38 hydra01 kernel: [2090266.849311] [] alloc_pages_current+0xb8/0x180 Sep 16 12:12:38 hydra01 kernel: [2090266.849317] [] __get_free_pages+0xe/0x50 Sep 16 12:12:38 hydra01 kernel: [2090266.849324] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:12:38 hydra01 kernel: [2090266.849330] [] __kmalloc+0x161/0x170 Sep 16 12:12:38 hydra01 kernel: [2090266.849336] [] ? get_iowait_time+0x60/0x60 Sep 16 12:12:38 hydra01 kernel: [2090266.849341] [] stat_open+0x56/0xc0 Sep 16 12:12:38 hydra01 kernel: [2090266.849347] [] proc_reg_open+0xa0/0x180 Sep 16 12:12:38 hydra01 kernel: [2090266.849354] [] ? seq_release_private+0x60/0x60 Sep 16 12:12:38 hydra01 kernel: [2090266.849359] [] do_dentry_open+0x203/0x290 Sep 16 12:12:38 hydra01 kernel: [2090266.849364] [] ? init_once+0x20/0x20 Sep 16 12:12:38 hydra01 kernel: [2090266.849368] [] vfs_open+0x49/0x50 Sep 16 12:12:38 hydra01 kernel: [2090266.849374] [] do_last+0x296/0xea0 Sep 16 12:12:38 hydra01 kernel: [2090266.849378] [] ? inode_permission+0x18/0x50 Sep 16 12:12:38 hydra01 kernel: [2090266.849383] [] ? link_path_walk+0x6b/0x880 Sep 16 12:12:38 hydra01 kernel: [2090266.849388] [] path_openat+0xb3/0x4b0 Sep 16 12:12:38 hydra01 kernel: [2090266.849393] [] do_filp_open+0x41/0xa0 Sep 16 12:12:38 hydra01 kernel: [2090266.849399] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:12:38 hydra01 kernel: [2090266.849403] [] do_sys_open+0xf3/0x230 Sep 16 12:12:38 hydra01 kernel: [2090266.849409] [] ? sys_select+0xcb/0x110 Sep 16 12:12:38 hydra01 kernel: [2090266.849413] [] sys_open+0x21/0x30 Sep 16 12:12:38 hydra01 kernel: [2090266.849419] [] system_call_fastpath+0x1a/0x1f Sep 16 12:12:38 hydra01 kernel: [2090266.849422] Mem-Info: Sep 16 12:12:38 hydra01 kernel: [2090266.849424] Node 0 DMA per-cpu: Sep 16 12:12:38 hydra01 kernel: [2090266.849429] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849432] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849434] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849437] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849440] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849442] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849445] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849447] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849450] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849452] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849455] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849458] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849460] Node 0 DMA32 per-cpu: Sep 16 12:12:38 hydra01 kernel: [2090266.849464] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849467] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849469] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849472] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849475] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849477] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849480] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849483] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849485] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849488] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849490] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849493] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849495] Node 0 Normal per-cpu: Sep 16 12:12:38 hydra01 kernel: [2090266.849499] CPU 0: hi: 186, btch: 31 usd: 168 Sep 16 12:12:38 hydra01 kernel: [2090266.849501] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849504] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849507] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849509] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849512] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849515] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849517] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849520] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849523] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849525] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849528] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849535] active_anon:1294180 inactive_anon:832199 isolated_anon:0 Sep 16 12:12:38 hydra01 kernel: [2090266.849535] active_file:1870062 inactive_file:3482489 isolated_file:0 Sep 16 12:12:38 hydra01 kernel: [2090266.849535] unevictable:0 dirty:103 writeback:0 unstable:0 Sep 16 12:12:38 hydra01 kernel: [2090266.849535] free:82687 slab_reclaimable:381980 slab_unreclaimable:107195 Sep 16 12:12:38 hydra01 kernel: [2090266.849535] mapped:26046 shmem:151 pagetables:9672 bounce:0 Sep 16 12:12:38 hydra01 kernel: [2090266.849535] free_cma:0 Sep 16 12:12:38 hydra01 kernel: [2090266.849542] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:12:38 hydra01 kernel: [2090266.849551] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:12:38 hydra01 kernel: [2090266.849557] Node 0 DMA32 free:146460kB min:6224kB low:7780kB high:9336kB active_anon:439528kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171156kB slab_unreclaimable:46900kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:38 hydra01 kernel: [2090266.849566] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:12:38 hydra01 kernel: [2090266.849571] Node 0 Normal free:168444kB min:61324kB low:76652kB high:91984kB active_anon:4737192kB inactive_anon:2535528kB active_file:7385052kB inactive_file:12560188kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:412kB writeback:0kB mapped:100260kB shmem:604kB slab_reclaimable:1356764kB slab_unreclaimable:381880kB kernel_stack:8008kB pagetables:37160kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:12:38 hydra01 kernel: [2090266.849581] lowmem_reserve[]: 0 0 0 0 Sep 16 12:12:38 hydra01 kernel: [2090266.849586] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:12:38 hydra01 kernel: [2090266.849607] Node 0 DMA32: 3007*4kB (UEM) 3182*8kB (UEM) 6027*16kB (UEM) 383*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146684kB Sep 16 12:12:38 hydra01 kernel: [2090266.849626] Node 0 Normal: 16459*4kB (UEMR) 11617*8kB (UEMR) 557*16kB (UEMR) 23*32kB (UER) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170084kB Sep 16 12:12:38 hydra01 kernel: [2090266.849646] 5784925 total pagecache pages Sep 16 12:12:38 hydra01 kernel: [2090266.849648] 432226 pages in swap cache Sep 16 12:12:38 hydra01 kernel: [2090266.849651] Swap cache stats: add 20773464, delete 20341238, find 12661771/14316160 Sep 16 12:12:38 hydra01 kernel: [2090266.849654] Free swap = 185330368kB Sep 16 12:12:38 hydra01 kernel: [2090266.849656] Total swap = 187617276kB Sep 16 12:12:38 hydra01 kernel: [2090267.024485] 8388592 pages RAM Sep 16 12:12:38 hydra01 kernel: [2090267.024488] 151696 pages reserved Sep 16 12:12:38 hydra01 kernel: [2090267.024490] 8681221 pages shared Sep 16 12:12:38 hydra01 kernel: [2090267.024491] 3446592 pages non-shared Sep 16 12:12:44 hydra01 kernel: [2090272.330256] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:12:44 hydra01 kernel: [2090272.330264] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:12:44 hydra01 kernel: [2090272.330267] Call Trace: Sep 16 12:12:44 hydra01 kernel: [2090272.330280] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:12:44 hydra01 kernel: [2090272.330287] [] ? drain_local_pages+0x16/0x20 Sep 16 12:12:44 hydra01 kernel: [2090272.330295] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:12:44 hydra01 kernel: [2090272.330301] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:12:44 hydra01 kernel: [2090272.330309] [] alloc_pages_current+0xb8/0x180 Sep 16 12:12:44 hydra01 kernel: [2090272.330314] [] __get_free_pages+0xe/0x50 Sep 16 12:12:44 hydra01 kernel: [2090272.330321] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:12:44 hydra01 kernel: [2090272.330327] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:12:44 hydra01 kernel: [2090272.330333] [] __kmalloc+0x161/0x170 Sep 16 12:12:44 hydra01 kernel: [2090272.330339] [] ? get_iowait_time+0x60/0x60 Sep 16 12:12:44 hydra01 kernel: [2090272.330344] [] stat_open+0x56/0xc0 Sep 16 12:12:44 hydra01 kernel: [2090272.330350] [] proc_reg_open+0xa0/0x180 Sep 16 12:12:44 hydra01 kernel: [2090272.330357] [] ? seq_release_private+0x60/0x60 Sep 16 12:12:44 hydra01 kernel: [2090272.330363] [] do_dentry_open+0x203/0x290 Sep 16 12:12:44 hydra01 kernel: [2090272.330368] [] ? init_once+0x20/0x20 Sep 16 12:12:44 hydra01 kernel: [2090272.330372] [] vfs_open+0x49/0x50 Sep 16 12:12:44 hydra01 kernel: [2090272.330378] [] do_last+0x296/0xea0 Sep 16 12:12:44 hydra01 kernel: [2090272.330382] [] ? inode_permission+0x18/0x50 Sep 16 12:12:44 hydra01 kernel: [2090272.330387] [] ? link_path_walk+0x6b/0x880 Sep 16 12:12:44 hydra01 kernel: [2090272.330392] [] path_openat+0xb3/0x4b0 Sep 16 12:12:44 hydra01 kernel: [2090272.330398] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:12:44 hydra01 kernel: [2090272.330404] [] ? unmap_region+0xdb/0x120 Sep 16 12:12:44 hydra01 kernel: [2090272.330408] [] do_filp_open+0x41/0xa0 Sep 16 12:12:44 hydra01 kernel: [2090272.330414] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:12:44 hydra01 kernel: [2090272.330418] [] do_sys_open+0xf3/0x230 Sep 16 12:12:44 hydra01 kernel: [2090272.330423] [] sys_open+0x21/0x30 Sep 16 12:12:44 hydra01 kernel: [2090272.330429] [] system_call_fastpath+0x1a/0x1f Sep 16 12:12:44 hydra01 kernel: [2090272.330431] Mem-Info: Sep 16 12:12:44 hydra01 kernel: [2090272.330434] Node 0 DMA per-cpu: Sep 16 12:12:44 hydra01 kernel: [2090272.330439] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330442] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330444] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330447] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330449] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330452] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330455] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330457] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330460] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330462] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330465] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330468] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330470] Node 0 DMA32 per-cpu: Sep 16 12:12:44 hydra01 kernel: [2090272.330474] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330477] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330479] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330482] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330485] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330487] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330490] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330492] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330495] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330498] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330500] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330503] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330505] Node 0 Normal per-cpu: Sep 16 12:12:44 hydra01 kernel: [2090272.330509] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330511] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330514] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330517] CPU 3: hi: 186, btch: 31 usd: 171 Sep 16 12:12:44 hydra01 kernel: [2090272.330519] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330522] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330525] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330527] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330530] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330532] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330535] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330538] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330545] active_anon:1294207 inactive_anon:832295 isolated_anon:0 Sep 16 12:12:44 hydra01 kernel: [2090272.330545] active_file:1870064 inactive_file:3480090 isolated_file:0 Sep 16 12:12:44 hydra01 kernel: [2090272.330545] unevictable:0 dirty:72 writeback:0 unstable:0 Sep 16 12:12:44 hydra01 kernel: [2090272.330545] free:84964 slab_reclaimable:381978 slab_unreclaimable:107203 Sep 16 12:12:44 hydra01 kernel: [2090272.330545] mapped:25823 shmem:151 pagetables:9563 bounce:0 Sep 16 12:12:44 hydra01 kernel: [2090272.330545] free_cma:0 Sep 16 12:12:44 hydra01 kernel: [2090272.330552] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:12:44 hydra01 kernel: [2090272.330560] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:12:44 hydra01 kernel: [2090272.330567] Node 0 DMA32 free:146428kB min:6224kB low:7780kB high:9336kB active_anon:439524kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171156kB slab_unreclaimable:46932kB kernel_stack:800kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:12:44 hydra01 kernel: [2090272.330576] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:12:44 hydra01 kernel: [2090272.330581] Node 0 Normal free:177584kB min:61324kB low:76652kB high:91984kB active_anon:4737304kB inactive_anon:2535912kB active_file:7385060kB inactive_file:12550208kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:288kB writeback:0kB mapped:99368kB shmem:604kB slab_reclaimable:1356756kB slab_unreclaimable:381880kB kernel_stack:8016kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:12:44 hydra01 kernel: [2090272.330591] lowmem_reserve[]: 0 0 0 0 Sep 16 12:12:44 hydra01 kernel: [2090272.330596] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:12:44 hydra01 kernel: [2090272.330617] Node 0 DMA32: 3008*4kB (UEM) 3182*8kB (UEM) 6025*16kB (UEM) 384*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146688kB Sep 16 12:12:44 hydra01 kernel: [2090272.330636] Node 0 Normal: 14717*4kB (UEMR) 12753*8kB (UEMR) 949*16kB (UEMR) 44*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 179148kB Sep 16 12:12:44 hydra01 kernel: [2090272.330656] 5782339 total pagecache pages Sep 16 12:12:44 hydra01 kernel: [2090272.330658] 432226 pages in swap cache Sep 16 12:12:44 hydra01 kernel: [2090272.330661] Swap cache stats: add 20773464, delete 20341238, find 12661771/14316160 Sep 16 12:12:44 hydra01 kernel: [2090272.330663] Free swap = 185330368kB Sep 16 12:12:44 hydra01 kernel: [2090272.330666] Total swap = 187617276kB Sep 16 12:12:44 hydra01 kernel: [2090272.527691] 8388592 pages RAM Sep 16 12:12:44 hydra01 kernel: [2090272.527704] 151696 pages reserved Sep 16 12:12:44 hydra01 kernel: [2090272.527714] 8679728 pages shared Sep 16 12:12:44 hydra01 kernel: [2090272.527724] 3447032 pages non-shared Sep 16 12:13:13 hydra01 kernel: [2090301.845677] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:13:13 hydra01 kernel: [2090301.845686] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:13:13 hydra01 kernel: [2090301.845688] Call Trace: Sep 16 12:13:13 hydra01 kernel: [2090301.845702] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:13:13 hydra01 kernel: [2090301.845709] [] ? drain_local_pages+0x16/0x20 Sep 16 12:13:13 hydra01 kernel: [2090301.845716] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:13:13 hydra01 kernel: [2090301.845722] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:13:13 hydra01 kernel: [2090301.845730] [] alloc_pages_current+0xb8/0x180 Sep 16 12:13:13 hydra01 kernel: [2090301.845735] [] __get_free_pages+0xe/0x50 Sep 16 12:13:13 hydra01 kernel: [2090301.845742] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:13:13 hydra01 kernel: [2090301.845748] [] __kmalloc+0x161/0x170 Sep 16 12:13:13 hydra01 kernel: [2090301.845755] [] ? get_iowait_time+0x60/0x60 Sep 16 12:13:13 hydra01 kernel: [2090301.845759] [] stat_open+0x56/0xc0 Sep 16 12:13:13 hydra01 kernel: [2090301.845765] [] proc_reg_open+0xa0/0x180 Sep 16 12:13:13 hydra01 kernel: [2090301.845772] [] ? seq_release_private+0x60/0x60 Sep 16 12:13:13 hydra01 kernel: [2090301.845778] [] do_dentry_open+0x203/0x290 Sep 16 12:13:13 hydra01 kernel: [2090301.845783] [] ? init_once+0x20/0x20 Sep 16 12:13:13 hydra01 kernel: [2090301.845787] [] vfs_open+0x49/0x50 Sep 16 12:13:13 hydra01 kernel: [2090301.845792] [] do_last+0x296/0xea0 Sep 16 12:13:13 hydra01 kernel: [2090301.845797] [] ? inode_permission+0x18/0x50 Sep 16 12:13:13 hydra01 kernel: [2090301.845802] [] ? link_path_walk+0x6b/0x880 Sep 16 12:13:13 hydra01 kernel: [2090301.845807] [] path_openat+0xb3/0x4b0 Sep 16 12:13:13 hydra01 kernel: [2090301.845812] [] do_filp_open+0x41/0xa0 Sep 16 12:13:13 hydra01 kernel: [2090301.845817] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:13:13 hydra01 kernel: [2090301.845822] [] do_sys_open+0xf3/0x230 Sep 16 12:13:13 hydra01 kernel: [2090301.845830] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:13:13 hydra01 kernel: [2090301.845834] [] sys_open+0x21/0x30 Sep 16 12:13:13 hydra01 kernel: [2090301.845840] [] system_call_fastpath+0x1a/0x1f Sep 16 12:13:13 hydra01 kernel: [2090301.845843] Mem-Info: Sep 16 12:13:13 hydra01 kernel: [2090301.845845] Node 0 DMA per-cpu: Sep 16 12:13:13 hydra01 kernel: [2090301.845850] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845853] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845855] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845858] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845861] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845863] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845866] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845868] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845871] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845873] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845876] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845879] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845881] Node 0 DMA32 per-cpu: Sep 16 12:13:13 hydra01 kernel: [2090301.845885] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845888] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845890] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845893] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845895] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845898] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845901] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845903] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845906] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845909] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845911] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845914] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845916] Node 0 Normal per-cpu: Sep 16 12:13:13 hydra01 kernel: [2090301.845919] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845922] CPU 1: hi: 186, btch: 31 usd: 169 Sep 16 12:13:13 hydra01 kernel: [2090301.845925] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845928] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845930] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845933] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845935] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845938] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845941] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845943] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845946] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845949] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:13:13 hydra01 kernel: [2090301.845956] active_anon:1294400 inactive_anon:832421 isolated_anon:0 Sep 16 12:13:13 hydra01 kernel: [2090301.845956] active_file:1870071 inactive_file:3482246 isolated_file:0 Sep 16 12:13:13 hydra01 kernel: [2090301.845956] unevictable:0 dirty:67 writeback:0 unstable:0 Sep 16 12:13:13 hydra01 kernel: [2090301.845956] free:82664 slab_reclaimable:381931 slab_unreclaimable:107151 Sep 16 12:13:13 hydra01 kernel: [2090301.845956] mapped:25915 shmem:149 pagetables:9647 bounce:0 Sep 16 12:13:13 hydra01 kernel: [2090301.845956] free_cma:0 Sep 16 12:13:13 hydra01 kernel: [2090301.845963] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:13:13 hydra01 kernel: [2090301.845971] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:13:13 hydra01 kernel: [2090301.845978] Node 0 DMA32 free:146680kB min:6224kB low:7780kB high:9336kB active_anon:439508kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171156kB slab_unreclaimable:46884kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:13:13 hydra01 kernel: [2090301.845987] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:13:13 hydra01 kernel: [2090301.845992] Node 0 Normal free:168424kB min:61324kB low:76652kB high:91984kB active_anon:4738092kB inactive_anon:2536416kB active_file:7385088kB inactive_file:12558832kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:268kB writeback:0kB mapped:99736kB shmem:596kB slab_reclaimable:1356568kB slab_unreclaimable:381720kB kernel_stack:8016kB pagetables:37060kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:13:13 hydra01 kernel: [2090301.846002] lowmem_reserve[]: 0 0 0 0 Sep 16 12:13:13 hydra01 kernel: [2090301.846007] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:13:13 hydra01 kernel: [2090301.846028] Node 0 DMA32: 3010*4kB (UEM) 3184*8kB (UEM) 6024*16kB (UEM) 389*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146856kB Sep 16 12:13:13 hydra01 kernel: [2090301.846046] Node 0 Normal: 16023*4kB (UEMR) 11711*8kB (UEMR) 617*16kB (UEMR) 27*32kB (UEMR) 7*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170244kB Sep 16 12:13:13 hydra01 kernel: [2090301.846066] 5784473 total pagecache pages Sep 16 12:13:13 hydra01 kernel: [2090301.846069] 432226 pages in swap cache Sep 16 12:13:13 hydra01 kernel: [2090301.846072] Swap cache stats: add 20773464, delete 20341238, find 12661771/14316160 Sep 16 12:13:13 hydra01 kernel: [2090301.846074] Free swap = 185330368kB Sep 16 12:13:13 hydra01 kernel: [2090301.846076] Total swap = 187617276kB Sep 16 12:13:13 hydra01 kernel: [2090302.039099] 8388592 pages RAM Sep 16 12:13:13 hydra01 kernel: [2090302.039116] 151696 pages reserved Sep 16 12:13:13 hydra01 kernel: [2090302.039119] 8676178 pages shared Sep 16 12:13:13 hydra01 kernel: [2090302.039121] 3450947 pages non-shared Sep 16 12:13:53 hydra01 kernel: [2090341.843733] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:13:53 hydra01 kernel: [2090341.843742] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:13:53 hydra01 kernel: [2090341.843746] Call Trace: Sep 16 12:13:53 hydra01 kernel: [2090341.843762] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:13:53 hydra01 kernel: [2090341.843769] [] ? drain_local_pages+0x16/0x20 Sep 16 12:13:53 hydra01 kernel: [2090341.843778] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:13:53 hydra01 kernel: [2090341.843785] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:13:53 hydra01 kernel: [2090341.843794] [] alloc_pages_current+0xb8/0x180 Sep 16 12:13:53 hydra01 kernel: [2090341.843802] [] __get_free_pages+0xe/0x50 Sep 16 12:13:53 hydra01 kernel: [2090341.843811] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:13:53 hydra01 kernel: [2090341.843819] [] __kmalloc+0x161/0x170 Sep 16 12:13:53 hydra01 kernel: [2090341.843828] [] ? get_iowait_time+0x60/0x60 Sep 16 12:13:53 hydra01 kernel: [2090341.843834] [] stat_open+0x56/0xc0 Sep 16 12:13:53 hydra01 kernel: [2090341.843842] [] proc_reg_open+0xa0/0x180 Sep 16 12:13:53 hydra01 kernel: [2090341.843852] [] ? seq_release_private+0x60/0x60 Sep 16 12:13:53 hydra01 kernel: [2090341.843859] [] do_dentry_open+0x203/0x290 Sep 16 12:13:53 hydra01 kernel: [2090341.843866] [] ? init_once+0x20/0x20 Sep 16 12:13:53 hydra01 kernel: [2090341.843871] [] vfs_open+0x49/0x50 Sep 16 12:13:53 hydra01 kernel: [2090341.843879] [] do_last+0x296/0xea0 Sep 16 12:13:53 hydra01 kernel: [2090341.843885] [] ? inode_permission+0x18/0x50 Sep 16 12:13:53 hydra01 kernel: [2090341.843891] [] ? link_path_walk+0x6b/0x880 Sep 16 12:13:53 hydra01 kernel: [2090341.843899] [] path_openat+0xb3/0x4b0 Sep 16 12:13:53 hydra01 kernel: [2090341.843907] [] do_filp_open+0x41/0xa0 Sep 16 12:13:53 hydra01 kernel: [2090341.843914] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:13:53 hydra01 kernel: [2090341.843921] [] do_sys_open+0xf3/0x230 Sep 16 12:13:53 hydra01 kernel: [2090341.843929] [] ? sys_select+0xcb/0x110 Sep 16 12:13:53 hydra01 kernel: [2090341.843936] [] sys_open+0x21/0x30 Sep 16 12:13:53 hydra01 kernel: [2090341.843944] [] system_call_fastpath+0x1a/0x1f Sep 16 12:13:53 hydra01 kernel: [2090341.843948] Mem-Info: Sep 16 12:13:53 hydra01 kernel: [2090341.843951] Node 0 DMA per-cpu: Sep 16 12:13:53 hydra01 kernel: [2090341.843958] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843963] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843967] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843972] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843976] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843980] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843984] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843988] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843992] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843995] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.843998] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844001] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844003] Node 0 DMA32 per-cpu: Sep 16 12:13:53 hydra01 kernel: [2090341.844007] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844009] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844012] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844015] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844017] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844020] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844023] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844025] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844028] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844030] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844033] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844036] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844038] Node 0 Normal per-cpu: Sep 16 12:13:53 hydra01 kernel: [2090341.844041] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844044] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844046] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844049] CPU 3: hi: 186, btch: 31 usd: 171 Sep 16 12:13:53 hydra01 kernel: [2090341.844052] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844054] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844057] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844060] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844063] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844065] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844068] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844071] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844078] active_anon:1294486 inactive_anon:832548 isolated_anon:0 Sep 16 12:13:53 hydra01 kernel: [2090341.844078] active_file:1870093 inactive_file:3490761 isolated_file:0 Sep 16 12:13:53 hydra01 kernel: [2090341.844078] unevictable:0 dirty:75 writeback:0 unstable:0 Sep 16 12:13:53 hydra01 kernel: [2090341.844078] free:74164 slab_reclaimable:381826 slab_unreclaimable:107162 Sep 16 12:13:53 hydra01 kernel: [2090341.844078] mapped:26121 shmem:151 pagetables:9538 bounce:0 Sep 16 12:13:53 hydra01 kernel: [2090341.844078] free_cma:0 Sep 16 12:13:53 hydra01 kernel: [2090341.844085] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:13:53 hydra01 kernel: [2090341.844094] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:13:53 hydra01 kernel: [2090341.844100] Node 0 DMA32 free:146296kB min:6224kB low:7780kB high:9336kB active_anon:439500kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171268kB slab_unreclaimable:46948kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:13:53 hydra01 kernel: [2090341.844109] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:13:53 hydra01 kernel: [2090341.844115] Node 0 Normal free:134808kB min:61324kB low:76652kB high:91984kB active_anon:4738444kB inactive_anon:2536924kB active_file:7385176kB inactive_file:12593276kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:292kB writeback:0kB mapped:100560kB shmem:604kB slab_reclaimable:1356036kB slab_unreclaimable:381700kB kernel_stack:8024kB pagetables:36624kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:13:53 hydra01 kernel: [2090341.844124] lowmem_reserve[]: 0 0 0 0 Sep 16 12:13:53 hydra01 kernel: [2090341.844129] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:13:53 hydra01 kernel: [2090341.844151] Node 0 DMA32: 3012*4kB (UEM) 3186*8kB (UEM) 6030*16kB (UEM) 374*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146560kB Sep 16 12:13:53 hydra01 kernel: [2090341.844170] Node 0 Normal: 16587*4kB (UEMR) 8562*8kB (UEMR) 16*16kB (UR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 136956kB Sep 16 12:13:53 hydra01 kernel: [2090341.844189] 5793156 total pagecache pages Sep 16 12:13:53 hydra01 kernel: [2090341.844192] 432226 pages in swap cache Sep 16 12:13:53 hydra01 kernel: [2090341.844195] Swap cache stats: add 20773464, delete 20341238, find 12661771/14316160 Sep 16 12:13:53 hydra01 kernel: [2090341.844197] Free swap = 185330368kB Sep 16 12:13:53 hydra01 kernel: [2090341.844199] Total swap = 187617276kB Sep 16 12:13:53 hydra01 kernel: [2090342.038397] 8388592 pages RAM Sep 16 12:13:53 hydra01 kernel: [2090342.038408] 151696 pages reserved Sep 16 12:13:53 hydra01 kernel: [2090342.038418] 8670099 pages shared Sep 16 12:13:53 hydra01 kernel: [2090342.038428] 3457213 pages non-shared Sep 16 12:13:58 hydra01 kernel: [2090346.841087] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:13:58 hydra01 kernel: [2090346.841096] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:13:58 hydra01 kernel: [2090346.841099] Call Trace: Sep 16 12:13:58 hydra01 kernel: [2090346.841112] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:13:58 hydra01 kernel: [2090346.841119] [] ? drain_local_pages+0x16/0x20 Sep 16 12:13:58 hydra01 kernel: [2090346.841127] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:13:58 hydra01 kernel: [2090346.841133] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:13:58 hydra01 kernel: [2090346.841140] [] alloc_pages_current+0xb8/0x180 Sep 16 12:13:58 hydra01 kernel: [2090346.841146] [] __get_free_pages+0xe/0x50 Sep 16 12:13:58 hydra01 kernel: [2090346.841153] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:13:58 hydra01 kernel: [2090346.841159] [] __kmalloc+0x161/0x170 Sep 16 12:13:58 hydra01 kernel: [2090346.841165] [] ? get_iowait_time+0x60/0x60 Sep 16 12:13:58 hydra01 kernel: [2090346.841169] [] stat_open+0x56/0xc0 Sep 16 12:13:58 hydra01 kernel: [2090346.841176] [] proc_reg_open+0xa0/0x180 Sep 16 12:13:58 hydra01 kernel: [2090346.841183] [] ? seq_release_private+0x60/0x60 Sep 16 12:13:58 hydra01 kernel: [2090346.841188] [] do_dentry_open+0x203/0x290 Sep 16 12:13:58 hydra01 kernel: [2090346.841193] [] ? init_once+0x20/0x20 Sep 16 12:13:58 hydra01 kernel: [2090346.841198] [] vfs_open+0x49/0x50 Sep 16 12:13:58 hydra01 kernel: [2090346.841203] [] do_last+0x296/0xea0 Sep 16 12:13:58 hydra01 kernel: [2090346.841208] [] ? inode_permission+0x18/0x50 Sep 16 12:13:58 hydra01 kernel: [2090346.841213] [] ? link_path_walk+0x6b/0x880 Sep 16 12:13:58 hydra01 kernel: [2090346.841218] [] path_openat+0xb3/0x4b0 Sep 16 12:13:58 hydra01 kernel: [2090346.841223] [] do_filp_open+0x41/0xa0 Sep 16 12:13:58 hydra01 kernel: [2090346.841228] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:13:58 hydra01 kernel: [2090346.841233] [] do_sys_open+0xf3/0x230 Sep 16 12:13:58 hydra01 kernel: [2090346.841239] [] ? sys_select+0xcb/0x110 Sep 16 12:13:58 hydra01 kernel: [2090346.841243] [] sys_open+0x21/0x30 Sep 16 12:13:58 hydra01 kernel: [2090346.841249] [] system_call_fastpath+0x1a/0x1f Sep 16 12:13:58 hydra01 kernel: [2090346.841251] Mem-Info: Sep 16 12:13:58 hydra01 kernel: [2090346.841254] Node 0 DMA per-cpu: Sep 16 12:13:58 hydra01 kernel: [2090346.841259] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841262] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841264] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841267] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841269] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841272] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841275] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841277] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841280] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841282] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841285] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841288] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841290] Node 0 DMA32 per-cpu: Sep 16 12:13:58 hydra01 kernel: [2090346.841294] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841297] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841299] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841302] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841305] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841307] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841310] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841312] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841315] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841318] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841320] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841323] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841325] Node 0 Normal per-cpu: Sep 16 12:13:58 hydra01 kernel: [2090346.841328] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841331] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841334] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841336] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841339] CPU 4: hi: 186, btch: 31 usd: 169 Sep 16 12:13:58 hydra01 kernel: [2090346.841342] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841344] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841347] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841350] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841352] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841355] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841358] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841365] active_anon:1294537 inactive_anon:832579 isolated_anon:0 Sep 16 12:13:58 hydra01 kernel: [2090346.841365] active_file:1870095 inactive_file:3481751 isolated_file:0 Sep 16 12:13:58 hydra01 kernel: [2090346.841365] unevictable:0 dirty:58 writeback:0 unstable:0 Sep 16 12:13:58 hydra01 kernel: [2090346.841365] free:83685 slab_reclaimable:381802 slab_unreclaimable:107133 Sep 16 12:13:58 hydra01 kernel: [2090346.841365] mapped:26145 shmem:151 pagetables:9673 bounce:0 Sep 16 12:13:58 hydra01 kernel: [2090346.841365] free_cma:0 Sep 16 12:13:58 hydra01 kernel: [2090346.841372] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:13:58 hydra01 kernel: [2090346.841381] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:13:58 hydra01 kernel: [2090346.841387] Node 0 DMA32 free:146448kB min:6224kB low:7780kB high:9336kB active_anon:439496kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:12kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171172kB slab_unreclaimable:46900kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:13:58 hydra01 kernel: [2090346.841396] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:13:58 hydra01 kernel: [2090346.841401] Node 0 Normal free:172448kB min:61324kB low:76652kB high:91984kB active_anon:4738652kB inactive_anon:2537048kB active_file:7385184kB inactive_file:12557236kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:220kB writeback:0kB mapped:100656kB shmem:604kB slab_reclaimable:1356036kB slab_unreclaimable:381632kB kernel_stack:8016kB pagetables:37164kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:13:58 hydra01 kernel: [2090346.841411] lowmem_reserve[]: 0 0 0 0 Sep 16 12:13:58 hydra01 kernel: [2090346.841416] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:13:58 hydra01 kernel: [2090346.841437] Node 0 DMA32: 3011*4kB (UEM) 3185*8kB (UEM) 6030*16kB (UEM) 377*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146644kB Sep 16 12:13:58 hydra01 kernel: [2090346.841456] Node 0 Normal: 24998*4kB (UEMR) 8864*8kB (UEMR) 32*16kB (UEMR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 173272kB Sep 16 12:13:58 hydra01 kernel: [2090346.841476] 5784209 total pagecache pages Sep 16 12:13:58 hydra01 kernel: [2090346.841478] 432226 pages in swap cache Sep 16 12:13:58 hydra01 kernel: [2090346.841481] Swap cache stats: add 20773464, delete 20341238, find 12661771/14316160 Sep 16 12:13:58 hydra01 kernel: [2090346.841483] Free swap = 185330368kB Sep 16 12:13:58 hydra01 kernel: [2090346.841485] Total swap = 187617276kB Sep 16 12:13:58 hydra01 kernel: [2090347.039650] 8388592 pages RAM Sep 16 12:13:58 hydra01 kernel: [2090347.039662] 151696 pages reserved Sep 16 12:13:58 hydra01 kernel: [2090347.039672] 8669661 pages shared Sep 16 12:13:58 hydra01 kernel: [2090347.039682] 3458009 pages non-shared Sep 16 12:14:23 hydra01 kernel: [2090371.835114] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:14:23 hydra01 kernel: [2090371.835123] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:14:23 hydra01 kernel: [2090371.835126] Call Trace: Sep 16 12:14:23 hydra01 kernel: [2090371.835139] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:14:23 hydra01 kernel: [2090371.835146] [] ? drain_local_pages+0x16/0x20 Sep 16 12:14:23 hydra01 kernel: [2090371.835154] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:14:23 hydra01 kernel: [2090371.835160] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:14:23 hydra01 kernel: [2090371.835168] [] alloc_pages_current+0xb8/0x180 Sep 16 12:14:23 hydra01 kernel: [2090371.835173] [] __get_free_pages+0xe/0x50 Sep 16 12:14:23 hydra01 kernel: [2090371.835181] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:14:23 hydra01 kernel: [2090371.835186] [] __kmalloc+0x161/0x170 Sep 16 12:14:23 hydra01 kernel: [2090371.835193] [] ? get_iowait_time+0x60/0x60 Sep 16 12:14:23 hydra01 kernel: [2090371.835197] [] stat_open+0x56/0xc0 Sep 16 12:14:23 hydra01 kernel: [2090371.835204] [] proc_reg_open+0xa0/0x180 Sep 16 12:14:23 hydra01 kernel: [2090371.835211] [] ? seq_release_private+0x60/0x60 Sep 16 12:14:23 hydra01 kernel: [2090371.835216] [] do_dentry_open+0x203/0x290 Sep 16 12:14:23 hydra01 kernel: [2090371.835221] [] ? init_once+0x20/0x20 Sep 16 12:14:23 hydra01 kernel: [2090371.835225] [] vfs_open+0x49/0x50 Sep 16 12:14:23 hydra01 kernel: [2090371.835231] [] do_last+0x296/0xea0 Sep 16 12:14:23 hydra01 kernel: [2090371.835235] [] ? inode_permission+0x18/0x50 Sep 16 12:14:23 hydra01 kernel: [2090371.835240] [] ? link_path_walk+0x6b/0x880 Sep 16 12:14:23 hydra01 kernel: [2090371.835245] [] path_openat+0xb3/0x4b0 Sep 16 12:14:23 hydra01 kernel: [2090371.835250] [] do_filp_open+0x41/0xa0 Sep 16 12:14:23 hydra01 kernel: [2090371.835255] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:14:23 hydra01 kernel: [2090371.835260] [] do_sys_open+0xf3/0x230 Sep 16 12:14:23 hydra01 kernel: [2090371.835265] [] ? sys_select+0xcb/0x110 Sep 16 12:14:23 hydra01 kernel: [2090371.835270] [] sys_open+0x21/0x30 Sep 16 12:14:23 hydra01 kernel: [2090371.835276] [] system_call_fastpath+0x1a/0x1f Sep 16 12:14:23 hydra01 kernel: [2090371.835278] Mem-Info: Sep 16 12:14:23 hydra01 kernel: [2090371.835281] Node 0 DMA per-cpu: Sep 16 12:14:23 hydra01 kernel: [2090371.835286] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835288] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835291] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835294] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835296] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835299] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835301] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835304] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835306] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835309] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835312] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835314] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835316] Node 0 DMA32 per-cpu: Sep 16 12:14:23 hydra01 kernel: [2090371.835320] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835323] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835326] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835328] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835331] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835333] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835336] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835339] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835341] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835344] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835346] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835349] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835351] Node 0 Normal per-cpu: Sep 16 12:14:23 hydra01 kernel: [2090371.835354] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835357] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835360] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835363] CPU 3: hi: 186, btch: 31 usd: 173 Sep 16 12:14:23 hydra01 kernel: [2090371.835365] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835368] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835371] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835373] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835376] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835379] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835381] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835384] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835391] active_anon:1294617 inactive_anon:832674 isolated_anon:0 Sep 16 12:14:23 hydra01 kernel: [2090371.835391] active_file:1870108 inactive_file:3485233 isolated_file:0 Sep 16 12:14:23 hydra01 kernel: [2090371.835391] unevictable:0 dirty:79 writeback:0 unstable:0 Sep 16 12:14:23 hydra01 kernel: [2090371.835391] free:80139 slab_reclaimable:381748 slab_unreclaimable:107126 Sep 16 12:14:23 hydra01 kernel: [2090371.835391] mapped:26177 shmem:151 pagetables:9666 bounce:0 Sep 16 12:14:23 hydra01 kernel: [2090371.835391] free_cma:0 Sep 16 12:14:23 hydra01 kernel: [2090371.835398] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:14:23 hydra01 kernel: [2090371.835407] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:14:23 hydra01 kernel: [2090371.835413] Node 0 DMA32 free:146520kB min:6224kB low:7780kB high:9336kB active_anon:439492kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171172kB slab_unreclaimable:46900kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:14:23 hydra01 kernel: [2090371.835449] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:14:23 hydra01 kernel: [2090371.835457] Node 0 Normal free:158484kB min:61324kB low:76652kB high:91984kB active_anon:4738976kB inactive_anon:2537428kB active_file:7385236kB inactive_file:12570780kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:316kB writeback:0kB mapped:100784kB shmem:604kB slab_reclaimable:1355820kB slab_unreclaimable:381604kB kernel_stack:8016kB pagetables:37136kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:14:23 hydra01 kernel: [2090371.835471] lowmem_reserve[]: 0 0 0 0 Sep 16 12:14:23 hydra01 kernel: [2090371.835479] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:14:23 hydra01 kernel: [2090371.835613] Node 0 DMA32: 3012*4kB (UEM) 3185*8kB (UEM) 6028*16kB (UEM) 378*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146648kB Sep 16 12:14:23 hydra01 kernel: [2090371.835632] Node 0 Normal: 17491*4kB (UEMR) 10745*8kB (UEMR) 120*16kB (UEMR) 18*32kB (UEMR) 9*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 160276kB Sep 16 12:14:23 hydra01 kernel: [2090371.835652] 5787451 total pagecache pages Sep 16 12:14:23 hydra01 kernel: [2090371.835655] 432225 pages in swap cache Sep 16 12:14:23 hydra01 kernel: [2090371.835658] Swap cache stats: add 20773464, delete 20341239, find 12661772/14316161 Sep 16 12:14:23 hydra01 kernel: [2090371.835660] Free swap = 185330372kB Sep 16 12:14:23 hydra01 kernel: [2090371.835662] Total swap = 187617276kB Sep 16 12:14:23 hydra01 kernel: [2090372.030853] 8388592 pages RAM Sep 16 12:14:23 hydra01 kernel: [2090372.030862] 151696 pages reserved Sep 16 12:14:23 hydra01 kernel: [2090372.030864] 8665087 pages shared Sep 16 12:14:23 hydra01 kernel: [2090372.030866] 3462346 pages non-shared Sep 16 12:14:43 hydra01 kernel: [2090391.828117] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:14:43 hydra01 kernel: [2090391.828126] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:14:43 hydra01 kernel: [2090391.828129] Call Trace: Sep 16 12:14:43 hydra01 kernel: [2090391.828143] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:14:43 hydra01 kernel: [2090391.828149] [] ? drain_local_pages+0x16/0x20 Sep 16 12:14:43 hydra01 kernel: [2090391.828157] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:14:43 hydra01 kernel: [2090391.828163] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:14:43 hydra01 kernel: [2090391.828171] [] alloc_pages_current+0xb8/0x180 Sep 16 12:14:43 hydra01 kernel: [2090391.828177] [] __get_free_pages+0xe/0x50 Sep 16 12:14:43 hydra01 kernel: [2090391.828184] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:14:43 hydra01 kernel: [2090391.828190] [] __kmalloc+0x161/0x170 Sep 16 12:14:43 hydra01 kernel: [2090391.828196] [] ? get_iowait_time+0x60/0x60 Sep 16 12:14:43 hydra01 kernel: [2090391.828201] [] stat_open+0x56/0xc0 Sep 16 12:14:43 hydra01 kernel: [2090391.828207] [] proc_reg_open+0xa0/0x180 Sep 16 12:14:43 hydra01 kernel: [2090391.828215] [] ? seq_release_private+0x60/0x60 Sep 16 12:14:43 hydra01 kernel: [2090391.828220] [] do_dentry_open+0x203/0x290 Sep 16 12:14:43 hydra01 kernel: [2090391.828225] [] ? init_once+0x20/0x20 Sep 16 12:14:43 hydra01 kernel: [2090391.828229] [] vfs_open+0x49/0x50 Sep 16 12:14:43 hydra01 kernel: [2090391.828234] [] do_last+0x296/0xea0 Sep 16 12:14:43 hydra01 kernel: [2090391.828239] [] ? inode_permission+0x18/0x50 Sep 16 12:14:43 hydra01 kernel: [2090391.828244] [] ? link_path_walk+0x6b/0x880 Sep 16 12:14:43 hydra01 kernel: [2090391.828249] [] path_openat+0xb3/0x4b0 Sep 16 12:14:43 hydra01 kernel: [2090391.828264] [] do_filp_open+0x41/0xa0 Sep 16 12:14:43 hydra01 kernel: [2090391.828267] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:14:43 hydra01 kernel: [2090391.828270] [] do_sys_open+0xf3/0x230 Sep 16 12:14:43 hydra01 kernel: [2090391.828273] [] ? sys_select+0xcb/0x110 Sep 16 12:14:43 hydra01 kernel: [2090391.828275] [] sys_open+0x21/0x30 Sep 16 12:14:43 hydra01 kernel: [2090391.828279] [] system_call_fastpath+0x1a/0x1f Sep 16 12:14:43 hydra01 kernel: [2090391.828280] Mem-Info: Sep 16 12:14:43 hydra01 kernel: [2090391.828282] Node 0 DMA per-cpu: Sep 16 12:14:43 hydra01 kernel: [2090391.828284] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828286] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828287] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828288] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828290] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828291] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828292] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828294] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828295] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828296] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828298] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828299] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828300] Node 0 DMA32 per-cpu: Sep 16 12:14:43 hydra01 kernel: [2090391.828302] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828304] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828305] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828306] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828308] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828309] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828311] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828312] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828313] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828315] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828316] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828317] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828318] Node 0 Normal per-cpu: Sep 16 12:14:43 hydra01 kernel: [2090391.828320] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828322] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828323] CPU 2: hi: 186, btch: 31 usd: 186 Sep 16 12:14:43 hydra01 kernel: [2090391.828325] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828326] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828327] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828329] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828343] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828346] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828348] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828351] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828354] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828361] active_anon:1294377 inactive_anon:832702 isolated_anon:0 Sep 16 12:14:43 hydra01 kernel: [2090391.828361] active_file:1870113 inactive_file:3482507 isolated_file:0 Sep 16 12:14:43 hydra01 kernel: [2090391.828361] unevictable:0 dirty:77 writeback:0 unstable:0 Sep 16 12:14:43 hydra01 kernel: [2090391.828361] free:82648 slab_reclaimable:381727 slab_unreclaimable:107101 Sep 16 12:14:43 hydra01 kernel: [2090391.828361] mapped:26171 shmem:151 pagetables:9672 bounce:0 Sep 16 12:14:43 hydra01 kernel: [2090391.828361] free_cma:0 Sep 16 12:14:43 hydra01 kernel: [2090391.828368] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:14:43 hydra01 kernel: [2090391.828377] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:14:43 hydra01 kernel: [2090391.828383] Node 0 DMA32 free:146536kB min:6224kB low:7780kB high:9336kB active_anon:439380kB inactive_anon:793268kB active_file:95196kB inactive_file:1369768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171140kB slab_unreclaimable:46820kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:14:43 hydra01 kernel: [2090391.828392] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:14:43 hydra01 kernel: [2090391.828397] Node 0 Normal free:168212kB min:61324kB low:76652kB high:91984kB active_anon:4738128kB inactive_anon:2537540kB active_file:7385256kB inactive_file:12559876kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:308kB writeback:0kB mapped:100760kB shmem:604kB slab_reclaimable:1355768kB slab_unreclaimable:381584kB kernel_stack:8016kB pagetables:37160kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:14:43 hydra01 kernel: [2090391.828407] lowmem_reserve[]: 0 0 0 0 Sep 16 12:14:43 hydra01 kernel: [2090391.828412] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:14:43 hydra01 kernel: [2090391.828433] Node 0 DMA32: 3039*4kB (UEM) 3186*8kB (UEM) 6031*16kB (UEM) 380*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146876kB Sep 16 12:14:43 hydra01 kernel: [2090391.828453] Node 0 Normal: 14955*4kB (UEMR) 12955*8kB (UEMR) 230*16kB (UEMR) 27*32kB (UEMR) 9*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 169860kB Sep 16 12:14:43 hydra01 kernel: [2090391.828472] 5784837 total pagecache pages Sep 16 12:14:43 hydra01 kernel: [2090391.828475] 432225 pages in swap cache Sep 16 12:14:43 hydra01 kernel: [2090391.828478] Swap cache stats: add 20773464, delete 20341239, find 12661772/14316161 Sep 16 12:14:43 hydra01 kernel: [2090391.828480] Free swap = 185330372kB Sep 16 12:14:43 hydra01 kernel: [2090391.828482] Total swap = 187617276kB Sep 16 12:14:43 hydra01 kernel: [2090392.022270] 8388592 pages RAM Sep 16 12:14:43 hydra01 kernel: [2090392.022276] 151696 pages reserved Sep 16 12:14:43 hydra01 kernel: [2090392.022278] 8661529 pages shared Sep 16 12:14:43 hydra01 kernel: [2090392.022281] 3466238 pages non-shared Sep 16 12:15:08 hydra01 kernel: [2090416.818731] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:15:08 hydra01 kernel: [2090416.818740] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:15:08 hydra01 kernel: [2090416.818743] Call Trace: Sep 16 12:15:08 hydra01 kernel: [2090416.818756] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:15:08 hydra01 kernel: [2090416.818763] [] ? drain_local_pages+0x16/0x20 Sep 16 12:15:08 hydra01 kernel: [2090416.818771] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:15:08 hydra01 kernel: [2090416.818777] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:15:08 hydra01 kernel: [2090416.818784] [] alloc_pages_current+0xb8/0x180 Sep 16 12:15:08 hydra01 kernel: [2090416.818790] [] __get_free_pages+0xe/0x50 Sep 16 12:15:08 hydra01 kernel: [2090416.818797] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:15:08 hydra01 kernel: [2090416.818803] [] __kmalloc+0x161/0x170 Sep 16 12:15:08 hydra01 kernel: [2090416.818809] [] ? get_iowait_time+0x60/0x60 Sep 16 12:15:08 hydra01 kernel: [2090416.818814] [] stat_open+0x56/0xc0 Sep 16 12:15:08 hydra01 kernel: [2090416.818820] [] proc_reg_open+0xa0/0x180 Sep 16 12:15:08 hydra01 kernel: [2090416.818827] [] ? seq_release_private+0x60/0x60 Sep 16 12:15:08 hydra01 kernel: [2090416.818833] [] do_dentry_open+0x203/0x290 Sep 16 12:15:08 hydra01 kernel: [2090416.818838] [] ? init_once+0x20/0x20 Sep 16 12:15:08 hydra01 kernel: [2090416.818842] [] vfs_open+0x49/0x50 Sep 16 12:15:08 hydra01 kernel: [2090416.818847] [] do_last+0x296/0xea0 Sep 16 12:15:08 hydra01 kernel: [2090416.818852] [] ? inode_permission+0x18/0x50 Sep 16 12:15:08 hydra01 kernel: [2090416.818857] [] ? link_path_walk+0x6b/0x880 Sep 16 12:15:08 hydra01 kernel: [2090416.818862] [] path_openat+0xb3/0x4b0 Sep 16 12:15:08 hydra01 kernel: [2090416.818867] [] do_filp_open+0x41/0xa0 Sep 16 12:15:08 hydra01 kernel: [2090416.818873] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:15:08 hydra01 kernel: [2090416.818878] [] do_sys_open+0xf3/0x230 Sep 16 12:15:08 hydra01 kernel: [2090416.818883] [] ? sys_select+0xcb/0x110 Sep 16 12:15:08 hydra01 kernel: [2090416.818887] [] sys_open+0x21/0x30 Sep 16 12:15:08 hydra01 kernel: [2090416.818893] [] system_call_fastpath+0x1a/0x1f Sep 16 12:15:08 hydra01 kernel: [2090416.818896] Mem-Info: Sep 16 12:15:08 hydra01 kernel: [2090416.818898] Node 0 DMA per-cpu: Sep 16 12:15:08 hydra01 kernel: [2090416.818903] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818906] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818908] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818911] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818913] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818916] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818919] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818921] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818924] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818926] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818929] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818932] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818934] Node 0 DMA32 per-cpu: Sep 16 12:15:08 hydra01 kernel: [2090416.818938] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818940] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818943] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818946] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818948] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818951] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818954] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818956] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818959] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818961] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818964] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818967] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818969] Node 0 Normal per-cpu: Sep 16 12:15:08 hydra01 kernel: [2090416.818972] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818975] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818977] CPU 2: hi: 186, btch: 31 usd: 172 Sep 16 12:15:08 hydra01 kernel: [2090416.818980] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818983] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818985] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818988] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818991] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818993] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818996] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.818999] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.819001] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:08 hydra01 kernel: [2090416.819009] active_anon:1294610 inactive_anon:832722 isolated_anon:0 Sep 16 12:15:08 hydra01 kernel: [2090416.819009] active_file:1870133 inactive_file:3496237 isolated_file:0 Sep 16 12:15:08 hydra01 kernel: [2090416.819009] unevictable:0 dirty:90 writeback:0 unstable:0 Sep 16 12:15:08 hydra01 kernel: [2090416.819009] free:65556 slab_reclaimable:381711 slab_unreclaimable:107115 Sep 16 12:15:08 hydra01 kernel: [2090416.819009] mapped:26205 shmem:151 pagetables:9563 bounce:0 Sep 16 12:15:08 hydra01 kernel: [2090416.819009] free_cma:0 Sep 16 12:15:08 hydra01 kernel: [2090416.819015] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:15:08 hydra01 kernel: [2090416.819024] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:15:08 hydra01 kernel: [2090416.819030] Node 0 DMA32 free:146620kB min:6224kB low:7780kB high:9336kB active_anon:439364kB inactive_anon:793268kB active_file:95204kB inactive_file:1369760kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171076kB slab_unreclaimable:46948kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:08 hydra01 kernel: [2090416.819039] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:15:08 hydra01 kernel: [2090416.819044] Node 0 Normal free:99760kB min:61324kB low:76652kB high:91984kB active_anon:4739076kB inactive_anon:2537620kB active_file:7385328kB inactive_file:12615188kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:360kB writeback:0kB mapped:100896kB shmem:604kB slab_reclaimable:1355768kB slab_unreclaimable:381512kB kernel_stack:8016kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:08 hydra01 kernel: [2090416.819054] lowmem_reserve[]: 0 0 0 0 Sep 16 12:15:08 hydra01 kernel: [2090416.819059] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:15:08 hydra01 kernel: [2090416.819080] Node 0 DMA32: 3042*4kB (UEM) 3186*8kB (UEM) 6031*16kB (UEM) 378*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146824kB Sep 16 12:15:08 hydra01 kernel: [2090416.819099] Node 0 Normal: 11374*4kB (UEMR) 6820*8kB (UMR) 9*16kB (UMR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 102056kB Sep 16 12:15:08 hydra01 kernel: [2090416.819119] 5798624 total pagecache pages Sep 16 12:15:08 hydra01 kernel: [2090416.819121] 432225 pages in swap cache Sep 16 12:15:08 hydra01 kernel: [2090416.819124] Swap cache stats: add 20773466, delete 20341241, find 12661772/14316161 Sep 16 12:15:08 hydra01 kernel: [2090416.819127] Free swap = 185330364kB Sep 16 12:15:08 hydra01 kernel: [2090416.819129] Total swap = 187617276kB Sep 16 12:15:08 hydra01 kernel: [2090417.012799] 8388592 pages RAM Sep 16 12:15:08 hydra01 kernel: [2090417.012812] 151696 pages reserved Sep 16 12:15:08 hydra01 kernel: [2090417.012822] 8649112 pages shared Sep 16 12:15:08 hydra01 kernel: [2090417.012832] 3478015 pages non-shared Sep 16 12:15:48 hydra01 kernel: [2090456.658480] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:15:48 hydra01 kernel: [2090456.658486] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:15:48 hydra01 kernel: [2090456.658487] Call Trace: Sep 16 12:15:48 hydra01 kernel: [2090456.658496] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:15:48 hydra01 kernel: [2090456.658501] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:15:48 hydra01 kernel: [2090456.658504] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:15:48 hydra01 kernel: [2090456.658509] [] alloc_pages_current+0xb8/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.658512] [] __get_free_pages+0xe/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.658516] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:15:48 hydra01 kernel: [2090456.658520] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:15:48 hydra01 kernel: [2090456.658523] [] __kmalloc+0x161/0x170 Sep 16 12:15:48 hydra01 kernel: [2090456.658527] [] ? setkey+0x81/0x100 Sep 16 12:15:48 hydra01 kernel: [2090456.658531] [] ? get_iowait_time+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.658534] [] stat_open+0x56/0xc0 Sep 16 12:15:48 hydra01 kernel: [2090456.658537] [] proc_reg_open+0xa0/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.658541] [] ? seq_release_private+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.658545] [] do_dentry_open+0x203/0x290 Sep 16 12:15:48 hydra01 kernel: [2090456.658547] [] ? init_once+0x20/0x20 Sep 16 12:15:48 hydra01 kernel: [2090456.658550] [] vfs_open+0x49/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.658553] [] do_last+0x296/0xea0 Sep 16 12:15:48 hydra01 kernel: [2090456.658555] [] ? inode_permission+0x18/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.658557] [] ? link_path_walk+0x6b/0x880 Sep 16 12:15:48 hydra01 kernel: [2090456.658560] [] path_openat+0xb3/0x4b0 Sep 16 12:15:48 hydra01 kernel: [2090456.658564] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:15:48 hydra01 kernel: [2090456.658568] [] ? unmap_region+0xdb/0x120 Sep 16 12:15:48 hydra01 kernel: [2090456.658570] [] do_filp_open+0x41/0xa0 Sep 16 12:15:48 hydra01 kernel: [2090456.658573] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:15:48 hydra01 kernel: [2090456.658576] [] do_sys_open+0xf3/0x230 Sep 16 12:15:48 hydra01 kernel: [2090456.658578] [] sys_open+0x21/0x30 Sep 16 12:15:48 hydra01 kernel: [2090456.658581] [] system_call_fastpath+0x1a/0x1f Sep 16 12:15:48 hydra01 kernel: [2090456.658583] Mem-Info: Sep 16 12:15:48 hydra01 kernel: [2090456.658584] Node 0 DMA per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.658587] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658589] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658590] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658592] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658593] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658594] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658596] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658597] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658599] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658600] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658602] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658603] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658604] Node 0 DMA32 per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.658607] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658608] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658609] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658611] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658612] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658613] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658615] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658616] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658618] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658619] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658620] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658622] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658623] Node 0 Normal per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.658625] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658626] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658627] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658629] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658630] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658632] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658633] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658635] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658636] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658638] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658639] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658640] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658645] active_anon:1294790 inactive_anon:832999 isolated_anon:0 Sep 16 12:15:48 hydra01 kernel: [2090456.658645] active_file:1870149 inactive_file:3478789 isolated_file:0 Sep 16 12:15:48 hydra01 kernel: [2090456.658645] unevictable:0 dirty:103 writeback:0 unstable:0 Sep 16 12:15:48 hydra01 kernel: [2090456.658645] free:83936 slab_reclaimable:381478 slab_unreclaimable:106989 Sep 16 12:15:48 hydra01 kernel: [2090456.658645] mapped:26213 shmem:151 pagetables:9601 bounce:0 Sep 16 12:15:48 hydra01 kernel: [2090456.658645] free_cma:0 Sep 16 12:15:48 hydra01 kernel: [2090456.658648] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:15:48 hydra01 kernel: [2090456.658653] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:15:48 hydra01 kernel: [2090456.658656] Node 0 DMA32 free:146968kB min:6224kB low:7780kB high:9336kB active_anon:439336kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171068kB slab_unreclaimable:46804kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.658661] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:15:48 hydra01 kernel: [2090456.658664] Node 0 Normal free:172932kB min:61324kB low:76652kB high:91984kB active_anon:4739824kB inactive_anon:2538728kB active_file:7385380kB inactive_file:12545408kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:392kB writeback:0kB mapped:100928kB shmem:604kB slab_reclaimable:1354844kB slab_unreclaimable:381152kB kernel_stack:8016kB pagetables:36876kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.658669] lowmem_reserve[]: 0 0 0 0 Sep 16 12:15:48 hydra01 kernel: [2090456.658671] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:15:48 hydra01 kernel: [2090456.658682] Node 0 DMA32: 3051*4kB (UEM) 3186*8kB (UEM) 6035*16kB (UEM) 384*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 147052kB Sep 16 12:15:48 hydra01 kernel: [2090456.658692] Node 0 Normal: 11642*4kB (UEMR) 9559*8kB (UEMR) 2275*16kB (UEMR) 377*32kB (UEMR) 25*64kB (UER) 3*128kB (UR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 174512kB Sep 16 12:15:48 hydra01 kernel: [2090456.658703] 5781309 total pagecache pages Sep 16 12:15:48 hydra01 kernel: [2090456.658704] 432224 pages in swap cache Sep 16 12:15:48 hydra01 kernel: [2090456.658706] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:15:48 hydra01 kernel: [2090456.658707] Free swap = 185330368kB Sep 16 12:15:48 hydra01 kernel: [2090456.658708] Total swap = 187617276kB Sep 16 12:15:48 hydra01 kernel: [2090456.764667] 8388592 pages RAM Sep 16 12:15:48 hydra01 kernel: [2090456.764674] 151696 pages reserved Sep 16 12:15:48 hydra01 kernel: [2090456.764681] 8644168 pages shared Sep 16 12:15:48 hydra01 kernel: [2090456.764689] 3482926 pages non-shared Sep 16 12:15:48 hydra01 kernel: [2090456.765540] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:15:48 hydra01 kernel: [2090456.765544] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:15:48 hydra01 kernel: [2090456.765545] Call Trace: Sep 16 12:15:48 hydra01 kernel: [2090456.765553] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:15:48 hydra01 kernel: [2090456.765556] [] ? drain_local_pages+0x16/0x20 Sep 16 12:15:48 hydra01 kernel: [2090456.765561] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:15:48 hydra01 kernel: [2090456.765564] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:15:48 hydra01 kernel: [2090456.765569] [] alloc_pages_current+0xb8/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.765572] [] __get_free_pages+0xe/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.765576] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:15:48 hydra01 kernel: [2090456.765579] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:15:48 hydra01 kernel: [2090456.765582] [] __kmalloc+0x161/0x170 Sep 16 12:15:48 hydra01 kernel: [2090456.765587] [] ? setkey+0x80/0x100 Sep 16 12:15:48 hydra01 kernel: [2090456.765591] [] ? get_iowait_time+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.765593] [] stat_open+0x56/0xc0 Sep 16 12:15:48 hydra01 kernel: [2090456.765597] [] proc_reg_open+0xa0/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.765601] [] ? seq_release_private+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.765604] [] do_dentry_open+0x203/0x290 Sep 16 12:15:48 hydra01 kernel: [2090456.765607] [] ? init_once+0x20/0x20 Sep 16 12:15:48 hydra01 kernel: [2090456.765609] [] vfs_open+0x49/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.765612] [] do_last+0x296/0xea0 Sep 16 12:15:48 hydra01 kernel: [2090456.765614] [] ? inode_permission+0x18/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.765617] [] ? link_path_walk+0x6b/0x880 Sep 16 12:15:48 hydra01 kernel: [2090456.765619] [] path_openat+0xb3/0x4b0 Sep 16 12:15:48 hydra01 kernel: [2090456.765623] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:15:48 hydra01 kernel: [2090456.765626] [] ? unmap_region+0xdb/0x120 Sep 16 12:15:48 hydra01 kernel: [2090456.765629] [] do_filp_open+0x41/0xa0 Sep 16 12:15:48 hydra01 kernel: [2090456.765632] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:15:48 hydra01 kernel: [2090456.765634] [] do_sys_open+0xf3/0x230 Sep 16 12:15:48 hydra01 kernel: [2090456.765637] [] sys_open+0x21/0x30 Sep 16 12:15:48 hydra01 kernel: [2090456.765640] [] system_call_fastpath+0x1a/0x1f Sep 16 12:15:48 hydra01 kernel: [2090456.765641] Mem-Info: Sep 16 12:15:48 hydra01 kernel: [2090456.765643] Node 0 DMA per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.765645] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765647] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765648] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765649] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765651] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765652] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765653] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765655] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765656] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765657] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765659] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765660] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765661] Node 0 DMA32 per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.765663] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765665] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765666] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765667] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765669] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765670] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765671] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765673] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765674] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765675] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765677] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765678] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765679] Node 0 Normal per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.765681] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765682] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765684] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765685] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765687] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765688] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765689] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765691] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765692] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765693] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765695] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765696] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765700] active_anon:1294790 inactive_anon:832999 isolated_anon:0 Sep 16 12:15:48 hydra01 kernel: [2090456.765700] active_file:1870149 inactive_file:3478789 isolated_file:0 Sep 16 12:15:48 hydra01 kernel: [2090456.765700] unevictable:0 dirty:103 writeback:0 unstable:0 Sep 16 12:15:48 hydra01 kernel: [2090456.765700] free:83936 slab_reclaimable:381478 slab_unreclaimable:106989 Sep 16 12:15:48 hydra01 kernel: [2090456.765700] mapped:26213 shmem:151 pagetables:9601 bounce:0 Sep 16 12:15:48 hydra01 kernel: [2090456.765700] free_cma:0 Sep 16 12:15:48 hydra01 kernel: [2090456.765703] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:15:48 hydra01 kernel: [2090456.765708] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:15:48 hydra01 kernel: [2090456.765711] Node 0 DMA32 free:146968kB min:6224kB low:7780kB high:9336kB active_anon:439336kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171068kB slab_unreclaimable:46804kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.765716] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:15:48 hydra01 kernel: [2090456.765719] Node 0 Normal free:172932kB min:61324kB low:76652kB high:91984kB active_anon:4739824kB inactive_anon:2538728kB active_file:7385380kB inactive_file:12545408kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:392kB writeback:0kB mapped:100928kB shmem:604kB slab_reclaimable:1354844kB slab_unreclaimable:381152kB kernel_stack:8016kB pagetables:36876kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.765724] lowmem_reserve[]: 0 0 0 0 Sep 16 12:15:48 hydra01 kernel: [2090456.765726] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:15:48 hydra01 kernel: [2090456.765738] Node 0 DMA32: 3051*4kB (UEM) 3186*8kB (UEM) 6035*16kB (UEM) 384*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 147052kB Sep 16 12:15:48 hydra01 kernel: [2090456.765747] Node 0 Normal: 11642*4kB (UEMR) 9559*8kB (UEMR) 2275*16kB (UEMR) 377*32kB (UEMR) 25*64kB (UER) 3*128kB (UR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 174512kB Sep 16 12:15:48 hydra01 kernel: [2090456.765758] 5781309 total pagecache pages Sep 16 12:15:48 hydra01 kernel: [2090456.765759] 432224 pages in swap cache Sep 16 12:15:48 hydra01 kernel: [2090456.765761] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:15:48 hydra01 kernel: [2090456.765762] Free swap = 185330368kB Sep 16 12:15:48 hydra01 kernel: [2090456.765763] Total swap = 187617276kB Sep 16 12:15:48 hydra01 kernel: [2090456.811818] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:15:48 hydra01 kernel: [2090456.811823] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:15:48 hydra01 kernel: [2090456.811824] Call Trace: Sep 16 12:15:48 hydra01 kernel: [2090456.811833] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:15:48 hydra01 kernel: [2090456.811838] [] ? on_each_cpu_mask+0x37/0x70 Sep 16 12:15:48 hydra01 kernel: [2090456.811842] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:15:48 hydra01 kernel: [2090456.811846] [] alloc_pages_current+0xb8/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.811849] [] __get_free_pages+0xe/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.811853] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:15:48 hydra01 kernel: [2090456.811856] [] __kmalloc+0x161/0x170 Sep 16 12:15:48 hydra01 kernel: [2090456.811860] [] ? get_iowait_time+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.811863] [] stat_open+0x56/0xc0 Sep 16 12:15:48 hydra01 kernel: [2090456.811866] [] proc_reg_open+0xa0/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.811870] [] ? seq_release_private+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.811873] [] do_dentry_open+0x203/0x290 Sep 16 12:15:48 hydra01 kernel: [2090456.811876] [] ? init_once+0x20/0x20 Sep 16 12:15:48 hydra01 kernel: [2090456.811878] [] vfs_open+0x49/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.811881] [] do_last+0x296/0xea0 Sep 16 12:15:48 hydra01 kernel: [2090456.811884] [] ? inode_permission+0x18/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.811886] [] ? link_path_walk+0x6b/0x880 Sep 16 12:15:48 hydra01 kernel: [2090456.811889] [] path_openat+0xb3/0x4b0 Sep 16 12:15:48 hydra01 kernel: [2090456.811892] [] do_filp_open+0x41/0xa0 Sep 16 12:15:48 hydra01 kernel: [2090456.811894] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:15:48 hydra01 kernel: [2090456.811897] [] do_sys_open+0xf3/0x230 Sep 16 12:15:48 hydra01 kernel: [2090456.811902] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:15:48 hydra01 kernel: [2090456.811904] [] sys_open+0x21/0x30 Sep 16 12:15:48 hydra01 kernel: [2090456.811907] [] system_call_fastpath+0x1a/0x1f Sep 16 12:15:48 hydra01 kernel: [2090456.811909] Mem-Info: Sep 16 12:15:48 hydra01 kernel: [2090456.811910] Node 0 DMA per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.811913] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811914] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811916] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811917] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811918] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811920] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811921] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811922] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811924] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811925] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811927] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811928] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811929] Node 0 DMA32 per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.811931] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811932] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811934] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811935] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811936] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811938] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811939] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811941] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811942] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811943] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811945] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811946] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811947] Node 0 Normal per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.811949] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811950] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811952] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811953] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811954] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811956] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811957] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811958] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811960] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811961] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811962] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811964] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811968] active_anon:1294790 inactive_anon:832999 isolated_anon:0 Sep 16 12:15:48 hydra01 kernel: [2090456.811968] active_file:1870149 inactive_file:3478789 isolated_file:0 Sep 16 12:15:48 hydra01 kernel: [2090456.811968] unevictable:0 dirty:103 writeback:0 unstable:0 Sep 16 12:15:48 hydra01 kernel: [2090456.811968] free:83936 slab_reclaimable:381478 slab_unreclaimable:106989 Sep 16 12:15:48 hydra01 kernel: [2090456.811968] mapped:26213 shmem:151 pagetables:9601 bounce:0 Sep 16 12:15:48 hydra01 kernel: [2090456.811968] free_cma:0 Sep 16 12:15:48 hydra01 kernel: [2090456.811971] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:15:48 hydra01 kernel: [2090456.811976] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:15:48 hydra01 kernel: [2090456.811979] Node 0 DMA32 free:146968kB min:6224kB low:7780kB high:9336kB active_anon:439336kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171068kB slab_unreclaimable:46804kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.811984] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:15:48 hydra01 kernel: [2090456.811987] Node 0 Normal free:172932kB min:61324kB low:76652kB high:91984kB active_anon:4739824kB inactive_anon:2538728kB active_file:7385380kB inactive_file:12545408kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:392kB writeback:0kB mapped:100928kB shmem:604kB slab_reclaimable:1354844kB slab_unreclaimable:381152kB kernel_stack:8016kB pagetables:36876kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.811992] lowmem_reserve[]: 0 0 0 0 Sep 16 12:15:48 hydra01 kernel: [2090456.811994] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:15:48 hydra01 kernel: [2090456.812005] Node 0 DMA32: 3051*4kB (UEM) 3186*8kB (UEM) 6035*16kB (UEM) 384*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 147052kB Sep 16 12:15:48 hydra01 kernel: [2090456.812015] Node 0 Normal: 11641*4kB (UEMR) 9559*8kB (UEMR) 2275*16kB (UEMR) 377*32kB (UEMR) 25*64kB (UER) 3*128kB (UR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 174508kB Sep 16 12:15:48 hydra01 kernel: [2090456.812025] 5781309 total pagecache pages Sep 16 12:15:48 hydra01 kernel: [2090456.812027] 432224 pages in swap cache Sep 16 12:15:48 hydra01 kernel: [2090456.812028] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:15:48 hydra01 kernel: [2090456.812030] Free swap = 185330368kB Sep 16 12:15:48 hydra01 kernel: [2090456.812031] Total swap = 187617276kB Sep 16 12:15:48 hydra01 kernel: [2090456.871866] 8388592 pages RAM Sep 16 12:15:48 hydra01 kernel: [2090456.871870] 151696 pages reserved Sep 16 12:15:48 hydra01 kernel: [2090456.871872] 8644065 pages shared Sep 16 12:15:48 hydra01 kernel: [2090456.871873] 3482941 pages non-shared Sep 16 12:15:48 hydra01 kernel: [2090456.872786] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:15:48 hydra01 kernel: [2090456.872789] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:15:48 hydra01 kernel: [2090456.872791] Call Trace: Sep 16 12:15:48 hydra01 kernel: [2090456.872799] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:15:48 hydra01 kernel: [2090456.872802] [] ? drain_local_pages+0x16/0x20 Sep 16 12:15:48 hydra01 kernel: [2090456.872807] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:15:48 hydra01 kernel: [2090456.872811] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:15:48 hydra01 kernel: [2090456.872815] [] alloc_pages_current+0xb8/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.872818] [] __get_free_pages+0xe/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.872822] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:15:48 hydra01 kernel: [2090456.872825] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:15:48 hydra01 kernel: [2090456.872828] [] __kmalloc+0x161/0x170 Sep 16 12:15:48 hydra01 kernel: [2090456.872832] [] ? amiga_partition+0x2c3/0x4d0 Sep 16 12:15:48 hydra01 kernel: [2090456.872836] [] ? get_iowait_time+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.872838] [] stat_open+0x56/0xc0 Sep 16 12:15:48 hydra01 kernel: [2090456.872842] [] proc_reg_open+0xa0/0x180 Sep 16 12:15:48 hydra01 kernel: [2090456.872846] [] ? seq_release_private+0x60/0x60 Sep 16 12:15:48 hydra01 kernel: [2090456.872849] [] do_dentry_open+0x203/0x290 Sep 16 12:15:48 hydra01 kernel: [2090456.872852] [] ? init_once+0x20/0x20 Sep 16 12:15:48 hydra01 kernel: [2090456.872854] [] vfs_open+0x49/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.872857] [] do_last+0x296/0xea0 Sep 16 12:15:48 hydra01 kernel: [2090456.872859] [] ? inode_permission+0x18/0x50 Sep 16 12:15:48 hydra01 kernel: [2090456.872862] [] ? link_path_walk+0x6b/0x880 Sep 16 12:15:48 hydra01 kernel: [2090456.872864] [] path_openat+0xb3/0x4b0 Sep 16 12:15:48 hydra01 kernel: [2090456.872868] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:15:48 hydra01 kernel: [2090456.872871] [] ? unmap_region+0xdb/0x120 Sep 16 12:15:48 hydra01 kernel: [2090456.872874] [] do_filp_open+0x41/0xa0 Sep 16 12:15:48 hydra01 kernel: [2090456.872877] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:15:48 hydra01 kernel: [2090456.872879] [] do_sys_open+0xf3/0x230 Sep 16 12:15:48 hydra01 kernel: [2090456.872882] [] sys_open+0x21/0x30 Sep 16 12:15:48 hydra01 kernel: [2090456.872886] [] system_call_fastpath+0x1a/0x1f Sep 16 12:15:48 hydra01 kernel: [2090456.872887] Mem-Info: Sep 16 12:15:48 hydra01 kernel: [2090456.872888] Node 0 DMA per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.872891] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872892] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872893] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872895] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872896] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872897] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872899] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872900] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872902] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872903] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872904] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872906] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872907] Node 0 DMA32 per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.872909] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872910] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872912] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872913] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872914] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872916] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872917] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872918] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872920] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872921] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872923] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872924] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872925] Node 0 Normal per-cpu: Sep 16 12:15:48 hydra01 kernel: [2090456.872927] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872928] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872929] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872931] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872932] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872934] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872935] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872936] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872938] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872939] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872940] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872942] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872946] active_anon:1294790 inactive_anon:832999 isolated_anon:0 Sep 16 12:15:48 hydra01 kernel: [2090456.872946] active_file:1870149 inactive_file:3478789 isolated_file:0 Sep 16 12:15:48 hydra01 kernel: [2090456.872946] unevictable:0 dirty:103 writeback:0 unstable:0 Sep 16 12:15:48 hydra01 kernel: [2090456.872946] free:84009 slab_reclaimable:381478 slab_unreclaimable:106989 Sep 16 12:15:48 hydra01 kernel: [2090456.872946] mapped:26213 shmem:151 pagetables:9601 bounce:0 Sep 16 12:15:48 hydra01 kernel: [2090456.872946] free_cma:0 Sep 16 12:15:48 hydra01 kernel: [2090456.872949] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:15:48 hydra01 kernel: [2090456.872954] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:15:48 hydra01 kernel: [2090456.872957] Node 0 DMA32 free:146968kB min:6224kB low:7780kB high:9336kB active_anon:439336kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171068kB slab_unreclaimable:46804kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.872962] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:15:48 hydra01 kernel: [2090456.872964] Node 0 Normal free:173224kB min:61324kB low:76652kB high:91984kB active_anon:4739824kB inactive_anon:2538728kB active_file:7385380kB inactive_file:12545408kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:392kB writeback:0kB mapped:100928kB shmem:604kB slab_reclaimable:1354844kB slab_unreclaimable:381152kB kernel_stack:8016kB pagetables:36876kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:15:48 hydra01 kernel: [2090456.872969] lowmem_reserve[]: 0 0 0 0 Sep 16 12:15:48 hydra01 kernel: [2090456.872972] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:15:48 hydra01 kernel: [2090456.872983] Node 0 DMA32: 3051*4kB (UEM) 3186*8kB (UEM) 6035*16kB (UEM) 384*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 147052kB Sep 16 12:15:48 hydra01 kernel: [2090456.872993] Node 0 Normal: 11652*4kB (UEMR) 9559*8kB (UEMR) 2275*16kB (UEMR) 377*32kB (UEMR) 25*64kB (UER) 3*128kB (UR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 174552kB Sep 16 12:15:48 hydra01 kernel: [2090456.873003] 5781309 total pagecache pages Sep 16 12:15:48 hydra01 kernel: [2090456.873005] 432224 pages in swap cache Sep 16 12:15:48 hydra01 kernel: [2090456.873006] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:15:48 hydra01 kernel: [2090456.873007] Free swap = 185330368kB Sep 16 12:15:48 hydra01 kernel: [2090456.873008] Total swap = 187617276kB Sep 16 12:15:48 hydra01 kernel: [2090456.978061] 8388592 pages RAM Sep 16 12:15:48 hydra01 kernel: [2090456.978070] 151696 pages reserved Sep 16 12:15:48 hydra01 kernel: [2090456.978077] 8643828 pages shared Sep 16 12:15:48 hydra01 kernel: [2090456.978083] 3482926 pages non-shared Sep 16 12:15:48 hydra01 kernel: [2090457.084084] 8388592 pages RAM Sep 16 12:15:48 hydra01 kernel: [2090457.084088] 151696 pages reserved Sep 16 12:15:48 hydra01 kernel: [2090457.084089] 8643842 pages shared Sep 16 12:15:48 hydra01 kernel: [2090457.084090] 3482910 pages non-shared Sep 16 12:16:18 hydra01 kernel: [2090487.075451] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:16:18 hydra01 kernel: [2090487.075462] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:16:18 hydra01 kernel: [2090487.075466] Call Trace: Sep 16 12:16:18 hydra01 kernel: [2090487.075482] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:16:18 hydra01 kernel: [2090487.075491] [] ? drain_local_pages+0x16/0x20 Sep 16 12:16:18 hydra01 kernel: [2090487.075501] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:16:18 hydra01 kernel: [2090487.075510] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:16:18 hydra01 kernel: [2090487.075520] [] alloc_pages_current+0xb8/0x180 Sep 16 12:16:18 hydra01 kernel: [2090487.075529] [] __get_free_pages+0xe/0x50 Sep 16 12:16:18 hydra01 kernel: [2090487.075538] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:16:18 hydra01 kernel: [2090487.075546] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:16:18 hydra01 kernel: [2090487.075554] [] __kmalloc+0x161/0x170 Sep 16 12:16:18 hydra01 kernel: [2090487.075564] [] ? setkey+0x80/0x100 Sep 16 12:16:18 hydra01 kernel: [2090487.075573] [] ? get_iowait_time+0x60/0x60 Sep 16 12:16:18 hydra01 kernel: [2090487.075580] [] stat_open+0x56/0xc0 Sep 16 12:16:18 hydra01 kernel: [2090487.075589] [] proc_reg_open+0xa0/0x180 Sep 16 12:16:18 hydra01 kernel: [2090487.075599] [] ? seq_release_private+0x60/0x60 Sep 16 12:16:18 hydra01 kernel: [2090487.075607] [] do_dentry_open+0x203/0x290 Sep 16 12:16:18 hydra01 kernel: [2090487.075614] [] ? init_once+0x20/0x20 Sep 16 12:16:18 hydra01 kernel: [2090487.075620] [] vfs_open+0x49/0x50 Sep 16 12:16:18 hydra01 kernel: [2090487.075628] [] do_last+0x296/0xea0 Sep 16 12:16:18 hydra01 kernel: [2090487.075635] [] ? inode_permission+0x18/0x50 Sep 16 12:16:18 hydra01 kernel: [2090487.075642] [] ? link_path_walk+0x6b/0x880 Sep 16 12:16:18 hydra01 kernel: [2090487.075649] [] path_openat+0xb3/0x4b0 Sep 16 12:16:18 hydra01 kernel: [2090487.075658] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:16:18 hydra01 kernel: [2090487.075667] [] ? unmap_region+0xdb/0x120 Sep 16 12:16:18 hydra01 kernel: [2090487.075674] [] do_filp_open+0x41/0xa0 Sep 16 12:16:18 hydra01 kernel: [2090487.075682] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:16:18 hydra01 kernel: [2090487.075716] [] do_sys_open+0xf3/0x230 Sep 16 12:16:18 hydra01 kernel: [2090487.075724] [] sys_open+0x21/0x30 Sep 16 12:16:18 hydra01 kernel: [2090487.075733] [] system_call_fastpath+0x1a/0x1f Sep 16 12:16:18 hydra01 kernel: [2090487.075737] Mem-Info: Sep 16 12:16:18 hydra01 kernel: [2090487.075740] Node 0 DMA per-cpu: Sep 16 12:16:18 hydra01 kernel: [2090487.075764] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075777] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075789] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075801] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075813] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075825] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075837] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075845] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075849] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075854] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075858] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075863] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075867] Node 0 DMA32 per-cpu: Sep 16 12:16:18 hydra01 kernel: [2090487.075872] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075877] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075882] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075886] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075890] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075894] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075898] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075903] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075907] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075911] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075916] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075920] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075924] Node 0 Normal per-cpu: Sep 16 12:16:18 hydra01 kernel: [2090487.075929] CPU 0: hi: 186, btch: 31 usd: 165 Sep 16 12:16:18 hydra01 kernel: [2090487.075934] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075938] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075942] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075946] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075950] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075954] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075959] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075964] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075968] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075973] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075977] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:18 hydra01 kernel: [2090487.075989] active_anon:1289116 inactive_anon:830721 isolated_anon:0 Sep 16 12:16:18 hydra01 kernel: [2090487.075989] active_file:1870166 inactive_file:3487344 isolated_file:0 Sep 16 12:16:18 hydra01 kernel: [2090487.075989] unevictable:0 dirty:55 writeback:0 unstable:0 Sep 16 12:16:18 hydra01 kernel: [2090487.075989] free:82938 slab_reclaimable:381522 slab_unreclaimable:106913 Sep 16 12:16:18 hydra01 kernel: [2090487.075989] mapped:26303 shmem:152 pagetables:9602 bounce:0 Sep 16 12:16:18 hydra01 kernel: [2090487.075989] free_cma:0 Sep 16 12:16:18 hydra01 kernel: [2090487.075999] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:16:18 hydra01 kernel: [2090487.076013] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:16:18 hydra01 kernel: [2090487.076022] Node 0 DMA32 free:147716kB min:6224kB low:7780kB high:9336kB active_anon:438292kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171196kB slab_unreclaimable:46596kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:18 hydra01 kernel: [2090487.076036] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:16:18 hydra01 kernel: [2090487.076044] Node 0 Normal free:168484kB min:61324kB low:76652kB high:91984kB active_anon:4718172kB inactive_anon:2529616kB active_file:7385448kB inactive_file:12579628kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:216kB writeback:0kB mapped:101288kB shmem:608kB slab_reclaimable:1354892kB slab_unreclaimable:381056kB kernel_stack:8024kB pagetables:36880kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:18 hydra01 kernel: [2090487.076079] lowmem_reserve[]: 0 0 0 0 Sep 16 12:16:18 hydra01 kernel: [2090487.076159] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:16:18 hydra01 kernel: [2090487.076209] Node 0 DMA32: 3315*4kB (UEM) 3188*8kB (UEM) 6035*16kB (UEM) 383*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 148156kB Sep 16 12:16:18 hydra01 kernel: [2090487.076237] Node 0 Normal: 14679*4kB (UEMR) 9156*8kB (UEMR) 2041*16kB (UEMR) 111*32kB (UMR) 7*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 169900kB Sep 16 12:16:18 hydra01 kernel: [2090487.076273] 5789715 total pagecache pages Sep 16 12:16:18 hydra01 kernel: [2090487.076277] 432224 pages in swap cache Sep 16 12:16:18 hydra01 kernel: [2090487.076282] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:16:18 hydra01 kernel: [2090487.076286] Free swap = 185330368kB Sep 16 12:16:18 hydra01 kernel: [2090487.076289] Total swap = 187617276kB Sep 16 12:16:19 hydra01 kernel: [2090487.271131] 8388592 pages RAM Sep 16 12:16:19 hydra01 kernel: [2090487.271137] 151696 pages reserved Sep 16 12:16:19 hydra01 kernel: [2090487.271139] 8654838 pages shared Sep 16 12:16:19 hydra01 kernel: [2090487.271141] 3479148 pages non-shared Sep 16 12:16:19 hydra01 kernel: [2090487.321816] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:16:19 hydra01 kernel: [2090487.321821] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:16:19 hydra01 kernel: [2090487.321822] Call Trace: Sep 16 12:16:19 hydra01 kernel: [2090487.321831] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:16:19 hydra01 kernel: [2090487.321835] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:16:19 hydra01 kernel: [2090487.321839] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:16:19 hydra01 kernel: [2090487.321843] [] alloc_pages_current+0xb8/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.321846] [] __get_free_pages+0xe/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.321850] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:16:19 hydra01 kernel: [2090487.321853] [] __kmalloc+0x161/0x170 Sep 16 12:16:19 hydra01 kernel: [2090487.321857] [] ? get_iowait_time+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.321860] [] stat_open+0x56/0xc0 Sep 16 12:16:19 hydra01 kernel: [2090487.321864] [] proc_reg_open+0xa0/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.321868] [] ? seq_release_private+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.321871] [] do_dentry_open+0x203/0x290 Sep 16 12:16:19 hydra01 kernel: [2090487.321873] [] ? init_once+0x20/0x20 Sep 16 12:16:19 hydra01 kernel: [2090487.321876] [] vfs_open+0x49/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.321879] [] do_last+0x296/0xea0 Sep 16 12:16:19 hydra01 kernel: [2090487.321881] [] ? inode_permission+0x18/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.321883] [] ? link_path_walk+0x6b/0x880 Sep 16 12:16:19 hydra01 kernel: [2090487.321886] [] path_openat+0xb3/0x4b0 Sep 16 12:16:19 hydra01 kernel: [2090487.321890] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.321893] [] ? unmap_region+0xdb/0x120 Sep 16 12:16:19 hydra01 kernel: [2090487.321896] [] do_filp_open+0x41/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.321899] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:16:19 hydra01 kernel: [2090487.321901] [] do_sys_open+0xf3/0x230 Sep 16 12:16:19 hydra01 kernel: [2090487.321904] [] sys_open+0x21/0x30 Sep 16 12:16:19 hydra01 kernel: [2090487.321907] [] system_call_fastpath+0x1a/0x1f Sep 16 12:16:19 hydra01 kernel: [2090487.321908] Mem-Info: Sep 16 12:16:19 hydra01 kernel: [2090487.321910] Node 0 DMA per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.321912] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321914] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321915] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321917] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321918] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321919] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321921] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321922] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321924] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321925] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321927] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321928] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321929] Node 0 DMA32 per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.321932] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321933] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321934] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321936] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321937] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321939] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321940] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321941] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321943] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321944] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321945] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321947] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321948] Node 0 Normal per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.321950] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321951] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321952] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321954] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321955] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321956] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321958] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321959] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321961] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321962] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321963] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321965] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321969] active_anon:1288967 inactive_anon:830708 isolated_anon:0 Sep 16 12:16:19 hydra01 kernel: [2090487.321969] active_file:1870166 inactive_file:3487056 isolated_file:0 Sep 16 12:16:19 hydra01 kernel: [2090487.321969] unevictable:0 dirty:57 writeback:0 unstable:0 Sep 16 12:16:19 hydra01 kernel: [2090487.321969] free:83524 slab_reclaimable:381519 slab_unreclaimable:106921 Sep 16 12:16:19 hydra01 kernel: [2090487.321969] mapped:26315 shmem:151 pagetables:9563 bounce:0 Sep 16 12:16:19 hydra01 kernel: [2090487.321969] free_cma:0 Sep 16 12:16:19 hydra01 kernel: [2090487.321972] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:16:19 hydra01 kernel: [2090487.321977] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:16:19 hydra01 kernel: [2090487.321980] Node 0 DMA32 free:147784kB min:6224kB low:7780kB high:9336kB active_anon:438264kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171196kB slab_unreclaimable:46628kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.321985] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:16:19 hydra01 kernel: [2090487.321988] Node 0 Normal free:170468kB min:61324kB low:76652kB high:91984kB active_anon:4717604kB inactive_anon:2529564kB active_file:7385448kB inactive_file:12578476kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:224kB writeback:0kB mapped:101336kB shmem:604kB slab_reclaimable:1354880kB slab_unreclaimable:381056kB kernel_stack:8024kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.321993] lowmem_reserve[]: 0 0 0 0 Sep 16 12:16:19 hydra01 kernel: [2090487.321995] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:16:19 hydra01 kernel: [2090487.322007] Node 0 DMA32: 3321*4kB (UEM) 3188*8kB (UEM) 6035*16kB (UEM) 383*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 148180kB Sep 16 12:16:19 hydra01 kernel: [2090487.322017] Node 0 Normal: 11955*4kB (UEMR) 9667*8kB (UEMR) 2363*16kB (UEMR) 205*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 171444kB Sep 16 12:16:19 hydra01 kernel: [2090487.322028] 5789638 total pagecache pages Sep 16 12:16:19 hydra01 kernel: [2090487.322029] 432224 pages in swap cache Sep 16 12:16:19 hydra01 kernel: [2090487.322031] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:16:19 hydra01 kernel: [2090487.322032] Free swap = 185330368kB Sep 16 12:16:19 hydra01 kernel: [2090487.322033] Total swap = 187617276kB Sep 16 12:16:19 hydra01 kernel: [2090487.427773] 8388592 pages RAM Sep 16 12:16:19 hydra01 kernel: [2090487.427779] 151696 pages reserved Sep 16 12:16:19 hydra01 kernel: [2090487.427784] 8655648 pages shared Sep 16 12:16:19 hydra01 kernel: [2090487.427789] 3479181 pages non-shared Sep 16 12:16:19 hydra01 kernel: [2090487.428758] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:16:19 hydra01 kernel: [2090487.428761] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:16:19 hydra01 kernel: [2090487.428762] Call Trace: Sep 16 12:16:19 hydra01 kernel: [2090487.428771] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:16:19 hydra01 kernel: [2090487.428774] [] ? drain_local_pages+0x16/0x20 Sep 16 12:16:19 hydra01 kernel: [2090487.428779] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:16:19 hydra01 kernel: [2090487.428782] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:16:19 hydra01 kernel: [2090487.428786] [] alloc_pages_current+0xb8/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.428789] [] __get_free_pages+0xe/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.428794] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:16:19 hydra01 kernel: [2090487.428797] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:16:19 hydra01 kernel: [2090487.428800] [] __kmalloc+0x161/0x170 Sep 16 12:16:19 hydra01 kernel: [2090487.428805] [] ? setkey+0x80/0x100 Sep 16 12:16:19 hydra01 kernel: [2090487.428808] [] ? get_iowait_time+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.428811] [] stat_open+0x56/0xc0 Sep 16 12:16:19 hydra01 kernel: [2090487.428814] [] proc_reg_open+0xa0/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.428818] [] ? seq_release_private+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.428822] [] do_dentry_open+0x203/0x290 Sep 16 12:16:19 hydra01 kernel: [2090487.428824] [] ? init_once+0x20/0x20 Sep 16 12:16:19 hydra01 kernel: [2090487.428827] [] vfs_open+0x49/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.428830] [] do_last+0x296/0xea0 Sep 16 12:16:19 hydra01 kernel: [2090487.428832] [] ? inode_permission+0x18/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.428835] [] ? link_path_walk+0x6b/0x880 Sep 16 12:16:19 hydra01 kernel: [2090487.428837] [] path_openat+0xb3/0x4b0 Sep 16 12:16:19 hydra01 kernel: [2090487.428841] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.428845] [] ? unmap_region+0xdb/0x120 Sep 16 12:16:19 hydra01 kernel: [2090487.428847] [] do_filp_open+0x41/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.428851] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:16:19 hydra01 kernel: [2090487.428853] [] do_sys_open+0xf3/0x230 Sep 16 12:16:19 hydra01 kernel: [2090487.428856] [] sys_open+0x21/0x30 Sep 16 12:16:19 hydra01 kernel: [2090487.428859] [] system_call_fastpath+0x1a/0x1f Sep 16 12:16:19 hydra01 kernel: [2090487.428861] Mem-Info: Sep 16 12:16:19 hydra01 kernel: [2090487.428862] Node 0 DMA per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.428864] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428866] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428867] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428869] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428870] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428871] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428873] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428874] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428875] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428877] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428878] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428879] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428880] Node 0 DMA32 per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.428883] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428884] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428885] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428887] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428888] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428889] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428891] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428892] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428893] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428895] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428896] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428898] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428899] Node 0 Normal per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.428901] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428902] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428903] CPU 2: hi: 186, btch: 31 usd: 30 Sep 16 12:16:19 hydra01 kernel: [2090487.428905] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428906] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428907] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428909] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428910] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428911] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428913] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428914] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428916] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428919] active_anon:1288894 inactive_anon:830708 isolated_anon:0 Sep 16 12:16:19 hydra01 kernel: [2090487.428919] active_file:1870166 inactive_file:3487129 isolated_file:0 Sep 16 12:16:19 hydra01 kernel: [2090487.428919] unevictable:0 dirty:57 writeback:0 unstable:0 Sep 16 12:16:19 hydra01 kernel: [2090487.428919] free:83334 slab_reclaimable:381519 slab_unreclaimable:106921 Sep 16 12:16:19 hydra01 kernel: [2090487.428919] mapped:26315 shmem:151 pagetables:9563 bounce:0 Sep 16 12:16:19 hydra01 kernel: [2090487.428919] free_cma:0 Sep 16 12:16:19 hydra01 kernel: [2090487.428923] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:16:19 hydra01 kernel: [2090487.428927] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:16:19 hydra01 kernel: [2090487.428931] Node 0 DMA32 free:147784kB min:6224kB low:7780kB high:9336kB active_anon:438264kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171196kB slab_unreclaimable:46628kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.428936] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:16:19 hydra01 kernel: [2090487.428938] Node 0 Normal free:169708kB min:61324kB low:76652kB high:91984kB active_anon:4717312kB inactive_anon:2529564kB active_file:7385448kB inactive_file:12578768kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:224kB writeback:0kB mapped:101336kB shmem:604kB slab_reclaimable:1354880kB slab_unreclaimable:381056kB kernel_stack:8024kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.428943] lowmem_reserve[]: 0 0 0 0 Sep 16 12:16:19 hydra01 kernel: [2090487.428946] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:16:19 hydra01 kernel: [2090487.428957] Node 0 DMA32: 3332*4kB (UEM) 3188*8kB (UEM) 6035*16kB (UEM) 383*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 148224kB Sep 16 12:16:19 hydra01 kernel: [2090487.428967] Node 0 Normal: 11807*4kB (UEMR) 9669*8kB (UEMR) 2363*16kB (UEMR) 205*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170868kB Sep 16 12:16:19 hydra01 kernel: [2090487.428978] 5789638 total pagecache pages Sep 16 12:16:19 hydra01 kernel: [2090487.428979] 432224 pages in swap cache Sep 16 12:16:19 hydra01 kernel: [2090487.428981] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:16:19 hydra01 kernel: [2090487.428982] Free swap = 185330368kB Sep 16 12:16:19 hydra01 kernel: [2090487.428983] Total swap = 187617276kB Sep 16 12:16:19 hydra01 kernel: [2090487.534805] 8388592 pages RAM Sep 16 12:16:19 hydra01 kernel: [2090487.534813] 151696 pages reserved Sep 16 12:16:19 hydra01 kernel: [2090487.534818] 8655617 pages shared Sep 16 12:16:19 hydra01 kernel: [2090487.534824] 3479323 pages non-shared Sep 16 12:16:19 hydra01 kernel: [2090487.536666] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:16:19 hydra01 kernel: [2090487.536669] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:16:19 hydra01 kernel: [2090487.536671] Call Trace: Sep 16 12:16:19 hydra01 kernel: [2090487.536679] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:16:19 hydra01 kernel: [2090487.536683] [] ? drain_local_pages+0x16/0x20 Sep 16 12:16:19 hydra01 kernel: [2090487.536688] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:16:19 hydra01 kernel: [2090487.536691] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:16:19 hydra01 kernel: [2090487.536695] [] alloc_pages_current+0xb8/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.536698] [] __get_free_pages+0xe/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.536702] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:16:19 hydra01 kernel: [2090487.536706] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:16:19 hydra01 kernel: [2090487.536709] [] __kmalloc+0x161/0x170 Sep 16 12:16:19 hydra01 kernel: [2090487.536713] [] ? amiga_partition+0x2c3/0x4d0 Sep 16 12:16:19 hydra01 kernel: [2090487.536716] [] ? get_iowait_time+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.536719] [] stat_open+0x56/0xc0 Sep 16 12:16:19 hydra01 kernel: [2090487.536722] [] proc_reg_open+0xa0/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.536726] [] ? seq_release_private+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.536729] [] do_dentry_open+0x203/0x290 Sep 16 12:16:19 hydra01 kernel: [2090487.536732] [] ? init_once+0x20/0x20 Sep 16 12:16:19 hydra01 kernel: [2090487.536734] [] vfs_open+0x49/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.536737] [] do_last+0x296/0xea0 Sep 16 12:16:19 hydra01 kernel: [2090487.536740] [] ? inode_permission+0x18/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.536742] [] ? link_path_walk+0x6b/0x880 Sep 16 12:16:19 hydra01 kernel: [2090487.536745] [] path_openat+0xb3/0x4b0 Sep 16 12:16:19 hydra01 kernel: [2090487.536748] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.536752] [] ? unmap_region+0xdb/0x120 Sep 16 12:16:19 hydra01 kernel: [2090487.536754] [] do_filp_open+0x41/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.536757] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:16:19 hydra01 kernel: [2090487.536760] [] do_sys_open+0xf3/0x230 Sep 16 12:16:19 hydra01 kernel: [2090487.536762] [] sys_open+0x21/0x30 Sep 16 12:16:19 hydra01 kernel: [2090487.536766] [] system_call_fastpath+0x1a/0x1f Sep 16 12:16:19 hydra01 kernel: [2090487.536767] Mem-Info: Sep 16 12:16:19 hydra01 kernel: [2090487.536769] Node 0 DMA per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.536771] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536773] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536774] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536775] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536777] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536778] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536779] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536781] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536782] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536783] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536785] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536786] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536787] Node 0 DMA32 per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.536789] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536791] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536792] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536794] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536795] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536796] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536798] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536799] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536800] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536802] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536803] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536805] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536806] Node 0 Normal per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.536807] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536809] CPU 1: hi: 186, btch: 31 usd: 23 Sep 16 12:16:19 hydra01 kernel: [2090487.536810] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536811] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536813] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536814] CPU 5: hi: 186, btch: 31 usd: 163 Sep 16 12:16:19 hydra01 kernel: [2090487.536816] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536817] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536818] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536820] CPU 9: hi: 186, btch: 31 usd: 23 Sep 16 12:16:19 hydra01 kernel: [2090487.536821] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536823] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536826] active_anon:1289186 inactive_anon:830708 isolated_anon:0 Sep 16 12:16:19 hydra01 kernel: [2090487.536826] active_file:1870166 inactive_file:3485287 isolated_file:0 Sep 16 12:16:19 hydra01 kernel: [2090487.536826] unevictable:0 dirty:57 writeback:0 unstable:0 Sep 16 12:16:19 hydra01 kernel: [2090487.536826] free:84800 slab_reclaimable:381519 slab_unreclaimable:106921 Sep 16 12:16:19 hydra01 kernel: [2090487.536826] mapped:26315 shmem:151 pagetables:9563 bounce:0 Sep 16 12:16:19 hydra01 kernel: [2090487.536826] free_cma:0 Sep 16 12:16:19 hydra01 kernel: [2090487.536830] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:16:19 hydra01 kernel: [2090487.536834] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:16:19 hydra01 kernel: [2090487.536838] Node 0 DMA32 free:147784kB min:6224kB low:7780kB high:9336kB active_anon:438264kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171196kB slab_unreclaimable:46628kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.536843] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:16:19 hydra01 kernel: [2090487.536845] Node 0 Normal free:175572kB min:61324kB low:76652kB high:91984kB active_anon:4718480kB inactive_anon:2529564kB active_file:7385448kB inactive_file:12571400kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:224kB writeback:0kB mapped:101336kB shmem:604kB slab_reclaimable:1354880kB slab_unreclaimable:381056kB kernel_stack:8024kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.536850] lowmem_reserve[]: 0 0 0 0 Sep 16 12:16:19 hydra01 kernel: [2090487.536853] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:16:19 hydra01 kernel: [2090487.536864] Node 0 DMA32: 3332*4kB (UEM) 3188*8kB (UEM) 6035*16kB (UEM) 383*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 148224kB Sep 16 12:16:19 hydra01 kernel: [2090487.536874] Node 0 Normal: 13310*4kB (UEMR) 9578*8kB (UEMR) 2363*16kB (UEMR) 205*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 176152kB Sep 16 12:16:19 hydra01 kernel: [2090487.536884] 5787894 total pagecache pages Sep 16 12:16:19 hydra01 kernel: [2090487.536886] 432224 pages in swap cache Sep 16 12:16:19 hydra01 kernel: [2090487.536887] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:16:19 hydra01 kernel: [2090487.536889] Free swap = 185330368kB Sep 16 12:16:19 hydra01 kernel: [2090487.536890] Total swap = 187617276kB Sep 16 12:16:19 hydra01 kernel: [2090487.642742] 8388592 pages RAM Sep 16 12:16:19 hydra01 kernel: [2090487.642747] 151696 pages reserved Sep 16 12:16:19 hydra01 kernel: [2090487.642752] 8653477 pages shared Sep 16 12:16:19 hydra01 kernel: [2090487.642757] 3479681 pages non-shared Sep 16 12:16:19 hydra01 kernel: [2090487.643788] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:16:19 hydra01 kernel: [2090487.643792] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:16:19 hydra01 kernel: [2090487.643793] Call Trace: Sep 16 12:16:19 hydra01 kernel: [2090487.643802] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:16:19 hydra01 kernel: [2090487.643805] [] ? drain_local_pages+0x16/0x20 Sep 16 12:16:19 hydra01 kernel: [2090487.643810] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:16:19 hydra01 kernel: [2090487.643813] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:16:19 hydra01 kernel: [2090487.643817] [] alloc_pages_current+0xb8/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.643820] [] __get_free_pages+0xe/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.643825] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:16:19 hydra01 kernel: [2090487.643828] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:16:19 hydra01 kernel: [2090487.643831] [] __kmalloc+0x161/0x170 Sep 16 12:16:19 hydra01 kernel: [2090487.643835] [] ? amiga_partition+0x2c3/0x4d0 Sep 16 12:16:19 hydra01 kernel: [2090487.643839] [] ? get_iowait_time+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.643841] [] stat_open+0x56/0xc0 Sep 16 12:16:19 hydra01 kernel: [2090487.643845] [] proc_reg_open+0xa0/0x180 Sep 16 12:16:19 hydra01 kernel: [2090487.643849] [] ? seq_release_private+0x60/0x60 Sep 16 12:16:19 hydra01 kernel: [2090487.643852] [] do_dentry_open+0x203/0x290 Sep 16 12:16:19 hydra01 kernel: [2090487.643855] [] ? init_once+0x20/0x20 Sep 16 12:16:19 hydra01 kernel: [2090487.643857] [] vfs_open+0x49/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.643860] [] do_last+0x296/0xea0 Sep 16 12:16:19 hydra01 kernel: [2090487.643862] [] ? inode_permission+0x18/0x50 Sep 16 12:16:19 hydra01 kernel: [2090487.643865] [] ? link_path_walk+0x6b/0x880 Sep 16 12:16:19 hydra01 kernel: [2090487.643867] [] path_openat+0xb3/0x4b0 Sep 16 12:16:19 hydra01 kernel: [2090487.643871] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.643875] [] ? unmap_region+0xdb/0x120 Sep 16 12:16:19 hydra01 kernel: [2090487.643877] [] do_filp_open+0x41/0xa0 Sep 16 12:16:19 hydra01 kernel: [2090487.643880] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:16:19 hydra01 kernel: [2090487.643883] [] do_sys_open+0xf3/0x230 Sep 16 12:16:19 hydra01 kernel: [2090487.643885] [] sys_open+0x21/0x30 Sep 16 12:16:19 hydra01 kernel: [2090487.643889] [] system_call_fastpath+0x1a/0x1f Sep 16 12:16:19 hydra01 kernel: [2090487.643890] Mem-Info: Sep 16 12:16:19 hydra01 kernel: [2090487.643892] Node 0 DMA per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.643894] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643896] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643897] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643898] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643900] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643901] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643902] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643904] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643905] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643906] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643908] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643909] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643910] Node 0 DMA32 per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.643912] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643914] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643915] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643916] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643918] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643919] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643920] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643922] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643923] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643924] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643926] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643927] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643928] Node 0 Normal per-cpu: Sep 16 12:16:19 hydra01 kernel: [2090487.643930] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643932] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643933] CPU 2: hi: 186, btch: 31 usd: 30 Sep 16 12:16:19 hydra01 kernel: [2090487.643934] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643936] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643937] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643938] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643940] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643941] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643943] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643944] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643945] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643949] active_anon:1289040 inactive_anon:830708 isolated_anon:0 Sep 16 12:16:19 hydra01 kernel: [2090487.643949] active_file:1870166 inactive_file:3485725 isolated_file:0 Sep 16 12:16:19 hydra01 kernel: [2090487.643949] unevictable:0 dirty:57 writeback:0 unstable:0 Sep 16 12:16:19 hydra01 kernel: [2090487.643949] free:84779 slab_reclaimable:381519 slab_unreclaimable:106921 Sep 16 12:16:19 hydra01 kernel: [2090487.643949] mapped:26315 shmem:151 pagetables:9563 bounce:0 Sep 16 12:16:19 hydra01 kernel: [2090487.643949] free_cma:0 Sep 16 12:16:19 hydra01 kernel: [2090487.643953] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:16:19 hydra01 kernel: [2090487.643957] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:16:19 hydra01 kernel: [2090487.643961] Node 0 DMA32 free:147784kB min:6224kB low:7780kB high:9336kB active_anon:438264kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171196kB slab_unreclaimable:46628kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.643965] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:16:19 hydra01 kernel: [2090487.643968] Node 0 Normal free:175488kB min:61324kB low:76652kB high:91984kB active_anon:4717896kB inactive_anon:2529564kB active_file:7385448kB inactive_file:12573152kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:224kB writeback:0kB mapped:101336kB shmem:604kB slab_reclaimable:1354880kB slab_unreclaimable:381056kB kernel_stack:8024kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:16:19 hydra01 kernel: [2090487.643973] lowmem_reserve[]: 0 0 0 0 Sep 16 12:16:19 hydra01 kernel: [2090487.643976] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:16:19 hydra01 kernel: [2090487.643987] Node 0 DMA32: 3341*4kB (UEM) 3188*8kB (UEM) 6035*16kB (UEM) 383*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 148260kB Sep 16 12:16:19 hydra01 kernel: [2090487.643997] Node 0 Normal: 13593*4kB (UEMR) 9565*8kB (UEMR) 2363*16kB (UEMR) 205*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 177180kB Sep 16 12:16:19 hydra01 kernel: [2090487.644007] 5788330 total pagecache pages Sep 16 12:16:19 hydra01 kernel: [2090487.644008] 432224 pages in swap cache Sep 16 12:16:19 hydra01 kernel: [2090487.644010] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:16:19 hydra01 kernel: [2090487.644011] Free swap = 185330368kB Sep 16 12:16:19 hydra01 kernel: [2090487.644012] Total swap = 187617276kB Sep 16 12:16:19 hydra01 kernel: [2090487.749921] 8388592 pages RAM Sep 16 12:16:19 hydra01 kernel: [2090487.749927] 151696 pages reserved Sep 16 12:16:19 hydra01 kernel: [2090487.749933] 8653385 pages shared Sep 16 12:16:19 hydra01 kernel: [2090487.749939] 3480165 pages non-shared Sep 16 12:17:38 hydra01 kernel: [2090566.792820] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:17:38 hydra01 kernel: [2090566.792829] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:17:38 hydra01 kernel: [2090566.792832] Call Trace: Sep 16 12:17:38 hydra01 kernel: [2090566.792846] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:17:38 hydra01 kernel: [2090566.792853] [] ? drain_local_pages+0x16/0x20 Sep 16 12:17:38 hydra01 kernel: [2090566.792861] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:17:38 hydra01 kernel: [2090566.792867] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:17:38 hydra01 kernel: [2090566.792874] [] alloc_pages_current+0xb8/0x180 Sep 16 12:17:38 hydra01 kernel: [2090566.792880] [] __get_free_pages+0xe/0x50 Sep 16 12:17:38 hydra01 kernel: [2090566.792887] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:17:38 hydra01 kernel: [2090566.792893] [] __kmalloc+0x161/0x170 Sep 16 12:17:38 hydra01 kernel: [2090566.792899] [] ? get_iowait_time+0x60/0x60 Sep 16 12:17:38 hydra01 kernel: [2090566.792904] [] stat_open+0x56/0xc0 Sep 16 12:17:38 hydra01 kernel: [2090566.792910] [] proc_reg_open+0xa0/0x180 Sep 16 12:17:38 hydra01 kernel: [2090566.792917] [] ? seq_release_private+0x60/0x60 Sep 16 12:17:38 hydra01 kernel: [2090566.792923] [] do_dentry_open+0x203/0x290 Sep 16 12:17:38 hydra01 kernel: [2090566.792928] [] ? init_once+0x20/0x20 Sep 16 12:17:38 hydra01 kernel: [2090566.792932] [] vfs_open+0x49/0x50 Sep 16 12:17:38 hydra01 kernel: [2090566.792937] [] do_last+0x296/0xea0 Sep 16 12:17:38 hydra01 kernel: [2090566.792942] [] ? inode_permission+0x18/0x50 Sep 16 12:17:38 hydra01 kernel: [2090566.792946] [] ? link_path_walk+0x6b/0x880 Sep 16 12:17:38 hydra01 kernel: [2090566.792951] [] path_openat+0xb3/0x4b0 Sep 16 12:17:38 hydra01 kernel: [2090566.792956] [] do_filp_open+0x41/0xa0 Sep 16 12:17:38 hydra01 kernel: [2090566.792961] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:17:38 hydra01 kernel: [2090566.792966] [] do_sys_open+0xf3/0x230 Sep 16 12:17:38 hydra01 kernel: [2090566.792973] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:17:38 hydra01 kernel: [2090566.792977] [] sys_open+0x21/0x30 Sep 16 12:17:38 hydra01 kernel: [2090566.792983] [] system_call_fastpath+0x1a/0x1f Sep 16 12:17:38 hydra01 kernel: [2090566.792985] Mem-Info: Sep 16 12:17:38 hydra01 kernel: [2090566.792988] Node 0 DMA per-cpu: Sep 16 12:17:38 hydra01 kernel: [2090566.792992] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.792995] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.792998] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793000] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793003] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793006] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793008] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793011] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793013] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793016] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793019] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793021] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793023] Node 0 DMA32 per-cpu: Sep 16 12:17:38 hydra01 kernel: [2090566.793027] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793030] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793033] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793035] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793038] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793040] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793043] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793046] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793048] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793051] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793054] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793056] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793058] Node 0 Normal per-cpu: Sep 16 12:17:38 hydra01 kernel: [2090566.793062] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793064] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793067] CPU 2: hi: 186, btch: 31 usd: 168 Sep 16 12:17:38 hydra01 kernel: [2090566.793070] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793072] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793075] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793077] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793080] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793083] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793085] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793088] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793091] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793098] active_anon:1272569 inactive_anon:830468 isolated_anon:0 Sep 16 12:17:38 hydra01 kernel: [2090566.793098] active_file:1870203 inactive_file:3496764 isolated_file:0 Sep 16 12:17:38 hydra01 kernel: [2090566.793098] unevictable:0 dirty:85 writeback:0 unstable:0 Sep 16 12:17:38 hydra01 kernel: [2090566.793098] free:86332 slab_reclaimable:381468 slab_unreclaimable:106746 Sep 16 12:17:38 hydra01 kernel: [2090566.793098] mapped:26091 shmem:151 pagetables:9673 bounce:0 Sep 16 12:17:38 hydra01 kernel: [2090566.793098] free_cma:0 Sep 16 12:17:38 hydra01 kernel: [2090566.793104] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:17:38 hydra01 kernel: [2090566.793113] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:17:38 hydra01 kernel: [2090566.793119] Node 0 DMA32 free:153308kB min:6224kB low:7780kB high:9336kB active_anon:432476kB inactive_anon:793268kB active_file:95216kB inactive_file:1369748kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171476kB slab_unreclaimable:46420kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:17:38 hydra01 kernel: [2090566.793129] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:17:38 hydra01 kernel: [2090566.793134] Node 0 Normal free:176176kB min:61324kB low:76652kB high:91984kB active_anon:4657800kB inactive_anon:2528604kB active_file:7385596kB inactive_file:12617308kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:332kB writeback:0kB mapped:100440kB shmem:604kB slab_reclaimable:1354396kB slab_unreclaimable:380564kB kernel_stack:8024kB pagetables:37164kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:17:38 hydra01 kernel: [2090566.793143] lowmem_reserve[]: 0 0 0 0 Sep 16 12:17:38 hydra01 kernel: [2090566.793148] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:17:38 hydra01 kernel: [2090566.793170] Node 0 DMA32: 4244*4kB (UEM) 3448*8kB (UEM) 6054*16kB (UEM) 369*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 153808kB Sep 16 12:17:38 hydra01 kernel: [2090566.793189] Node 0 Normal: 15126*4kB (UEMR) 11935*8kB (UEMR) 997*16kB (UEMR) 118*32kB (UEMR) 11*64kB (UMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 177696kB Sep 16 12:17:38 hydra01 kernel: [2090566.793208] 5799257 total pagecache pages Sep 16 12:17:38 hydra01 kernel: [2090566.793211] 432224 pages in swap cache Sep 16 12:17:38 hydra01 kernel: [2090566.793214] Swap cache stats: add 20773466, delete 20341242, find 12661773/14316162 Sep 16 12:17:38 hydra01 kernel: [2090566.793216] Free swap = 185330368kB Sep 16 12:17:38 hydra01 kernel: [2090566.793218] Total swap = 187617276kB Sep 16 12:17:38 hydra01 kernel: [2090566.986454] 8388592 pages RAM Sep 16 12:17:38 hydra01 kernel: [2090566.986459] 151696 pages reserved Sep 16 12:17:38 hydra01 kernel: [2090566.986462] 8661886 pages shared Sep 16 12:17:38 hydra01 kernel: [2090566.986465] 3486391 pages non-shared Sep 16 12:18:03 hydra01 kernel: [2090591.793149] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:18:03 hydra01 kernel: [2090591.793158] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:18:03 hydra01 kernel: [2090591.793161] Call Trace: Sep 16 12:18:03 hydra01 kernel: [2090591.793175] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:18:03 hydra01 kernel: [2090591.793182] [] ? drain_local_pages+0x16/0x20 Sep 16 12:18:03 hydra01 kernel: [2090591.793189] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:18:03 hydra01 kernel: [2090591.793195] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:18:03 hydra01 kernel: [2090591.793203] [] alloc_pages_current+0xb8/0x180 Sep 16 12:18:03 hydra01 kernel: [2090591.793208] [] __get_free_pages+0xe/0x50 Sep 16 12:18:03 hydra01 kernel: [2090591.793216] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:18:03 hydra01 kernel: [2090591.793221] [] __kmalloc+0x161/0x170 Sep 16 12:18:03 hydra01 kernel: [2090591.793228] [] ? get_iowait_time+0x60/0x60 Sep 16 12:18:03 hydra01 kernel: [2090591.793232] [] stat_open+0x56/0xc0 Sep 16 12:18:03 hydra01 kernel: [2090591.793239] [] proc_reg_open+0xa0/0x180 Sep 16 12:18:03 hydra01 kernel: [2090591.793245] [] ? seq_release_private+0x60/0x60 Sep 16 12:18:03 hydra01 kernel: [2090591.793251] [] do_dentry_open+0x203/0x290 Sep 16 12:18:03 hydra01 kernel: [2090591.793256] [] ? init_once+0x20/0x20 Sep 16 12:18:03 hydra01 kernel: [2090591.793260] [] vfs_open+0x49/0x50 Sep 16 12:18:03 hydra01 kernel: [2090591.793265] [] do_last+0x296/0xea0 Sep 16 12:18:03 hydra01 kernel: [2090591.793270] [] ? inode_permission+0x18/0x50 Sep 16 12:18:03 hydra01 kernel: [2090591.793274] [] ? link_path_walk+0x6b/0x880 Sep 16 12:18:03 hydra01 kernel: [2090591.793279] [] path_openat+0xb3/0x4b0 Sep 16 12:18:03 hydra01 kernel: [2090591.793285] [] do_filp_open+0x41/0xa0 Sep 16 12:18:03 hydra01 kernel: [2090591.793290] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:18:03 hydra01 kernel: [2090591.793295] [] do_sys_open+0xf3/0x230 Sep 16 12:18:03 hydra01 kernel: [2090591.793300] [] ? sys_select+0xcb/0x110 Sep 16 12:18:03 hydra01 kernel: [2090591.793305] [] sys_open+0x21/0x30 Sep 16 12:18:03 hydra01 kernel: [2090591.793310] [] system_call_fastpath+0x1a/0x1f Sep 16 12:18:03 hydra01 kernel: [2090591.793313] Mem-Info: Sep 16 12:18:03 hydra01 kernel: [2090591.793316] Node 0 DMA per-cpu: Sep 16 12:18:03 hydra01 kernel: [2090591.793320] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793323] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793326] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793328] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793331] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793333] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793336] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793338] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793341] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793343] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793346] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793349] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793351] Node 0 DMA32 per-cpu: Sep 16 12:18:03 hydra01 kernel: [2090591.793355] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793358] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793360] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793363] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793366] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793368] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793371] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793373] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793376] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793379] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793381] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793384] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793386] Node 0 Normal per-cpu: Sep 16 12:18:03 hydra01 kernel: [2090591.793389] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793392] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793394] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793397] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793400] CPU 4: hi: 186, btch: 31 usd: 171 Sep 16 12:18:03 hydra01 kernel: [2090591.793403] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793405] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793408] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793411] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793413] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793416] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793418] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793426] active_anon:1272891 inactive_anon:830465 isolated_anon:0 Sep 16 12:18:03 hydra01 kernel: [2090591.793426] active_file:1870196 inactive_file:3515893 isolated_file:0 Sep 16 12:18:03 hydra01 kernel: [2090591.793426] unevictable:0 dirty:56 writeback:0 unstable:0 Sep 16 12:18:03 hydra01 kernel: [2090591.793426] free:62012 slab_reclaimable:381404 slab_unreclaimable:106727 Sep 16 12:18:03 hydra01 kernel: [2090591.793426] mapped:25853 shmem:151 pagetables:9565 bounce:0 Sep 16 12:18:03 hydra01 kernel: [2090591.793426] free_cma:0 Sep 16 12:18:03 hydra01 kernel: [2090591.793432] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:18:03 hydra01 kernel: [2090591.793441] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:18:03 hydra01 kernel: [2090591.793448] Node 0 DMA32 free:144832kB min:6224kB low:7780kB high:9336kB active_anon:434720kB inactive_anon:793268kB active_file:95232kB inactive_file:1373652kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:16kB slab_reclaimable:171476kB slab_unreclaimable:46324kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:18:03 hydra01 kernel: [2090591.793475] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:18:03 hydra01 kernel: [2090591.793480] Node 0 Normal free:87664kB min:61324kB low:76652kB high:91984kB active_anon:4656844kB inactive_anon:2528592kB active_file:7385552kB inactive_file:12689536kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:224kB writeback:0kB mapped:99488kB shmem:588kB slab_reclaimable:1354140kB slab_unreclaimable:380584kB kernel_stack:8016kB pagetables:36732kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:18:03 hydra01 kernel: [2090591.793490] lowmem_reserve[]: 0 0 0 0 Sep 16 12:18:03 hydra01 kernel: [2090591.793495] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:18:03 hydra01 kernel: [2090591.793516] Node 0 DMA32: 2399*4kB (UEM) 3361*8kB (UEM) 6054*16kB (UEM) 369*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 145732kB Sep 16 12:18:03 hydra01 kernel: [2090591.793535] Node 0 Normal: 8127*4kB (UEMR) 6370*8kB (UEMR) 190*16kB (UEMR) 20*32kB (MR) 11*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 89132kB Sep 16 12:18:03 hydra01 kernel: [2090591.793555] 5818407 total pagecache pages Sep 16 12:18:03 hydra01 kernel: [2090591.793558] 432223 pages in swap cache Sep 16 12:18:03 hydra01 kernel: [2090591.793561] Swap cache stats: add 20773468, delete 20341245, find 12661773/14316162 Sep 16 12:18:03 hydra01 kernel: [2090591.793563] Free swap = 185330360kB Sep 16 12:18:03 hydra01 kernel: [2090591.793565] Total swap = 187617276kB Sep 16 12:18:03 hydra01 kernel: [2090591.986470] 8388592 pages RAM Sep 16 12:18:03 hydra01 kernel: [2090591.986474] 151696 pages reserved Sep 16 12:18:03 hydra01 kernel: [2090591.986477] 8655987 pages shared Sep 16 12:18:03 hydra01 kernel: [2090591.986491] 3496391 pages non-shared Sep 16 12:18:28 hydra01 kernel: [2090616.787714] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:18:28 hydra01 kernel: [2090616.787723] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:18:28 hydra01 kernel: [2090616.787726] Call Trace: Sep 16 12:18:28 hydra01 kernel: [2090616.787739] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:18:28 hydra01 kernel: [2090616.787746] [] ? drain_local_pages+0x16/0x20 Sep 16 12:18:28 hydra01 kernel: [2090616.787754] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:18:28 hydra01 kernel: [2090616.787760] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:18:28 hydra01 kernel: [2090616.787768] [] alloc_pages_current+0xb8/0x180 Sep 16 12:18:28 hydra01 kernel: [2090616.787774] [] __get_free_pages+0xe/0x50 Sep 16 12:18:28 hydra01 kernel: [2090616.787781] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:18:28 hydra01 kernel: [2090616.787787] [] __kmalloc+0x161/0x170 Sep 16 12:18:28 hydra01 kernel: [2090616.787793] [] ? get_iowait_time+0x60/0x60 Sep 16 12:18:28 hydra01 kernel: [2090616.787798] [] stat_open+0x56/0xc0 Sep 16 12:18:28 hydra01 kernel: [2090616.787804] [] proc_reg_open+0xa0/0x180 Sep 16 12:18:28 hydra01 kernel: [2090616.787811] [] ? seq_release_private+0x60/0x60 Sep 16 12:18:28 hydra01 kernel: [2090616.787817] [] do_dentry_open+0x203/0x290 Sep 16 12:18:28 hydra01 kernel: [2090616.787822] [] ? init_once+0x20/0x20 Sep 16 12:18:28 hydra01 kernel: [2090616.787826] [] vfs_open+0x49/0x50 Sep 16 12:18:28 hydra01 kernel: [2090616.787831] [] do_last+0x296/0xea0 Sep 16 12:18:28 hydra01 kernel: [2090616.787836] [] ? inode_permission+0x18/0x50 Sep 16 12:18:28 hydra01 kernel: [2090616.787841] [] ? link_path_walk+0x6b/0x880 Sep 16 12:18:28 hydra01 kernel: [2090616.787847] [] ? touch_atime+0x81/0x160 Sep 16 12:18:28 hydra01 kernel: [2090616.787852] [] path_openat+0xb3/0x4b0 Sep 16 12:18:28 hydra01 kernel: [2090616.787857] [] do_filp_open+0x41/0xa0 Sep 16 12:18:28 hydra01 kernel: [2090616.787862] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:18:28 hydra01 kernel: [2090616.787867] [] do_sys_open+0xf3/0x230 Sep 16 12:18:28 hydra01 kernel: [2090616.787875] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:18:28 hydra01 kernel: [2090616.787880] [] sys_open+0x21/0x30 Sep 16 12:18:28 hydra01 kernel: [2090616.787885] [] system_call_fastpath+0x1a/0x1f Sep 16 12:18:28 hydra01 kernel: [2090616.787888] Mem-Info: Sep 16 12:18:28 hydra01 kernel: [2090616.787891] Node 0 DMA per-cpu: Sep 16 12:18:28 hydra01 kernel: [2090616.787895] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787898] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787901] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787903] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787906] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787909] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787911] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787914] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787916] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787919] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787922] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787924] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787926] Node 0 DMA32 per-cpu: Sep 16 12:18:28 hydra01 kernel: [2090616.787930] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787933] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787936] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787938] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787941] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787944] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787946] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787949] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787951] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787954] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787957] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787959] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787961] Node 0 Normal per-cpu: Sep 16 12:18:28 hydra01 kernel: [2090616.787965] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787967] CPU 1: hi: 186, btch: 31 usd: 169 Sep 16 12:18:28 hydra01 kernel: [2090616.787970] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787973] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787975] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787978] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787981] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787983] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787986] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787989] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787991] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.787994] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:18:28 hydra01 kernel: [2090616.788002] active_anon:1273273 inactive_anon:830360 isolated_anon:0 Sep 16 12:18:28 hydra01 kernel: [2090616.788002] active_file:1870205 inactive_file:3506851 isolated_file:0 Sep 16 12:18:28 hydra01 kernel: [2090616.788002] unevictable:0 dirty:32 writeback:0 unstable:0 Sep 16 12:18:28 hydra01 kernel: [2090616.788002] free:71749 slab_reclaimable:381326 slab_unreclaimable:106631 Sep 16 12:18:28 hydra01 kernel: [2090616.788002] mapped:25855 shmem:151 pagetables:9674 bounce:0 Sep 16 12:18:28 hydra01 kernel: [2090616.788002] free_cma:0 Sep 16 12:18:28 hydra01 kernel: [2090616.788008] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:18:28 hydra01 kernel: [2090616.788017] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:18:28 hydra01 kernel: [2090616.788024] Node 0 DMA32 free:147060kB min:6224kB low:7780kB high:9336kB active_anon:432764kB inactive_anon:793268kB active_file:95232kB inactive_file:1373828kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171372kB slab_unreclaimable:46292kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:18:28 hydra01 kernel: [2090616.788033] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:18:28 hydra01 kernel: [2090616.788038] Node 0 Normal free:124092kB min:61324kB low:76652kB high:91984kB active_anon:4660328kB inactive_anon:2528172kB active_file:7385588kB inactive_file:12653192kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:128kB writeback:0kB mapped:99496kB shmem:604kB slab_reclaimable:1353932kB slab_unreclaimable:380232kB kernel_stack:8016kB pagetables:37168kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:18:28 hydra01 kernel: [2090616.788048] lowmem_reserve[]: 0 0 0 0 Sep 16 12:18:28 hydra01 kernel: [2090616.788053] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:18:28 hydra01 kernel: [2090616.788074] Node 0 DMA32: 2801*4kB (UEM) 3362*8kB (UEM) 6058*16kB (UEM) 372*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 147508kB Sep 16 12:18:28 hydra01 kernel: [2090616.788093] Node 0 Normal: 9618*4kB (UEMR) 8068*8kB (UEMR) 977*16kB (UEMR) 157*32kB (UEMR) 9*64kB (UMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 125528kB Sep 16 12:18:28 hydra01 kernel: [2090616.788113] 5809375 total pagecache pages Sep 16 12:18:28 hydra01 kernel: [2090616.788115] 432226 pages in swap cache Sep 16 12:18:28 hydra01 kernel: [2090616.788118] Swap cache stats: add 20773472, delete 20341246, find 12661773/14316162 Sep 16 12:18:28 hydra01 kernel: [2090616.788121] Free swap = 185330344kB Sep 16 12:18:28 hydra01 kernel: [2090616.788123] Total swap = 187617276kB Sep 16 12:18:28 hydra01 kernel: [2090616.981542] 8388592 pages RAM Sep 16 12:18:28 hydra01 kernel: [2090616.981552] 151696 pages reserved Sep 16 12:18:28 hydra01 kernel: [2090616.981560] 8648943 pages shared Sep 16 12:18:28 hydra01 kernel: [2090616.981569] 3503071 pages non-shared Sep 16 12:18:53 hydra01 kernel: [2090641.781604] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:18:53 hydra01 kernel: [2090641.781613] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:18:53 hydra01 kernel: [2090641.781615] Call Trace: Sep 16 12:18:53 hydra01 kernel: [2090641.781629] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:18:53 hydra01 kernel: [2090641.781635] [] ? drain_local_pages+0x16/0x20 Sep 16 12:18:53 hydra01 kernel: [2090641.781643] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:18:53 hydra01 kernel: [2090641.781649] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:18:53 hydra01 kernel: [2090641.781656] [] alloc_pages_current+0xb8/0x180 Sep 16 12:18:53 hydra01 kernel: [2090641.781662] [] __get_free_pages+0xe/0x50 Sep 16 12:18:53 hydra01 kernel: [2090641.781669] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:18:53 hydra01 kernel: [2090641.781675] [] __kmalloc+0x161/0x170 Sep 16 12:18:53 hydra01 kernel: [2090641.781681] [] ? get_iowait_time+0x60/0x60 Sep 16 12:18:53 hydra01 kernel: [2090641.781686] [] stat_open+0x56/0xc0 Sep 16 12:18:53 hydra01 kernel: [2090641.781692] [] proc_reg_open+0xa0/0x180 Sep 16 12:18:53 hydra01 kernel: [2090641.781699] [] ? seq_release_private+0x60/0x60 Sep 16 12:18:53 hydra01 kernel: [2090641.781705] [] do_dentry_open+0x203/0x290 Sep 16 12:18:53 hydra01 kernel: [2090641.781710] [] ? init_once+0x20/0x20 Sep 16 12:18:53 hydra01 kernel: [2090641.781714] [] vfs_open+0x49/0x50 Sep 16 12:18:53 hydra01 kernel: [2090641.781719] [] do_last+0x296/0xea0 Sep 16 12:18:53 hydra01 kernel: [2090641.781724] [] ? inode_permission+0x18/0x50 Sep 16 12:18:53 hydra01 kernel: [2090641.781729] [] ? link_path_walk+0x6b/0x880 Sep 16 12:18:53 hydra01 kernel: [2090641.781734] [] path_openat+0xb3/0x4b0 Sep 16 12:18:53 hydra01 kernel: [2090641.781739] [] do_filp_open+0x41/0xa0 Sep 16 12:18:53 hydra01 kernel: [2090641.781744] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:18:53 hydra01 kernel: [2090641.781749] [] do_sys_open+0xf3/0x230 Sep 16 12:18:53 hydra01 kernel: [2090641.781755] [] ? sys_select+0xcb/0x110 Sep 16 12:18:53 hydra01 kernel: [2090641.781759] [] sys_open+0x21/0x30 Sep 16 12:18:53 hydra01 kernel: [2090641.781765] [] system_call_fastpath+0x1a/0x1f Sep 16 12:18:53 hydra01 kernel: [2090641.781767] Mem-Info: Sep 16 12:18:53 hydra01 kernel: [2090641.781770] Node 0 DMA per-cpu: Sep 16 12:18:53 hydra01 kernel: [2090641.781775] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781777] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781780] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781783] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781785] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781788] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781790] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781793] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781796] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781798] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781801] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781804] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781806] Node 0 DMA32 per-cpu: Sep 16 12:18:53 hydra01 kernel: [2090641.781810] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781812] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781815] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781818] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781820] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781823] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781825] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781828] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781831] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781833] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781836] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781839] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781840] Node 0 Normal per-cpu: Sep 16 12:18:53 hydra01 kernel: [2090641.781844] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781847] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781849] CPU 2: hi: 186, btch: 31 usd: 186 Sep 16 12:18:53 hydra01 kernel: [2090641.781852] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781855] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781857] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781860] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781863] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781865] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781868] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781871] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781873] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781881] active_anon:1273515 inactive_anon:830358 isolated_anon:0 Sep 16 12:18:53 hydra01 kernel: [2090641.781881] active_file:1870216 inactive_file:3497854 isolated_file:0 Sep 16 12:18:53 hydra01 kernel: [2090641.781881] unevictable:0 dirty:43 writeback:0 unstable:0 Sep 16 12:18:53 hydra01 kernel: [2090641.781881] free:75349 slab_reclaimable:381183 slab_unreclaimable:106396 Sep 16 12:18:53 hydra01 kernel: [2090641.781881] mapped:25817 shmem:151 pagetables:9670 bounce:0 Sep 16 12:18:53 hydra01 kernel: [2090641.781881] free_cma:0 Sep 16 12:18:53 hydra01 kernel: [2090641.781887] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:18:53 hydra01 kernel: [2090641.781896] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:18:53 hydra01 kernel: [2090641.781902] Node 0 DMA32 free:147580kB min:6224kB low:7780kB high:9336kB active_anon:432684kB inactive_anon:793268kB active_file:95232kB inactive_file:1373828kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171364kB slab_unreclaimable:45972kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:18:53 hydra01 kernel: [2090641.781912] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:18:53 hydra01 kernel: [2090641.781917] Node 0 Normal free:137972kB min:61324kB low:76652kB high:91984kB active_anon:4661376kB inactive_anon:2528164kB active_file:7385632kB inactive_file:12617204kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:164kB writeback:0kB mapped:99344kB shmem:604kB slab_reclaimable:1353368kB slab_unreclaimable:379612kB kernel_stack:8016kB pagetables:37152kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:18:53 hydra01 kernel: [2090641.781927] lowmem_reserve[]: 0 0 0 0 Sep 16 12:18:53 hydra01 kernel: [2090641.781932] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:18:53 hydra01 kernel: [2090641.781953] Node 0 DMA32: 2821*4kB (UEM) 3361*8kB (UEM) 6065*16kB (UEM) 379*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 147916kB Sep 16 12:18:53 hydra01 kernel: [2090641.781972] Node 0 Normal: 7930*4kB (UEMR) 9858*8kB (UEMR) 1344*16kB (UEMR) 185*32kB (UEMR) 14*64kB (UMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 140184kB Sep 16 12:18:53 hydra01 kernel: [2090641.781993] 5800233 total pagecache pages Sep 16 12:18:53 hydra01 kernel: [2090641.781995] 432226 pages in swap cache Sep 16 12:18:53 hydra01 kernel: [2090641.781999] Swap cache stats: add 20773472, delete 20341246, find 12661773/14316162 Sep 16 12:18:53 hydra01 kernel: [2090641.782001] Free swap = 185330344kB Sep 16 12:18:53 hydra01 kernel: [2090641.782003] Total swap = 187617276kB Sep 16 12:18:53 hydra01 kernel: [2090641.975420] 8388592 pages RAM Sep 16 12:18:53 hydra01 kernel: [2090641.975424] 151696 pages reserved Sep 16 12:18:53 hydra01 kernel: [2090641.975426] 8636351 pages shared Sep 16 12:18:53 hydra01 kernel: [2090641.975427] 3513787 pages non-shared Sep 16 12:20:19 hydra01 kernel: [2090727.720888] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:20:19 hydra01 kernel: [2090727.720897] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:20:19 hydra01 kernel: [2090727.720899] Call Trace: Sep 16 12:20:19 hydra01 kernel: [2090727.720913] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:20:19 hydra01 kernel: [2090727.720919] [] ? drain_local_pages+0x16/0x20 Sep 16 12:20:19 hydra01 kernel: [2090727.720927] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:20:19 hydra01 kernel: [2090727.720933] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:20:19 hydra01 kernel: [2090727.720941] [] alloc_pages_current+0xb8/0x180 Sep 16 12:20:19 hydra01 kernel: [2090727.720946] [] __get_free_pages+0xe/0x50 Sep 16 12:20:19 hydra01 kernel: [2090727.720953] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:20:19 hydra01 kernel: [2090727.720959] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:20:19 hydra01 kernel: [2090727.720965] [] __kmalloc+0x161/0x170 Sep 16 12:20:19 hydra01 kernel: [2090727.720973] [] ? setkey+0x80/0x100 Sep 16 12:20:19 hydra01 kernel: [2090727.720979] [] ? get_iowait_time+0x60/0x60 Sep 16 12:20:19 hydra01 kernel: [2090727.720984] [] stat_open+0x56/0xc0 Sep 16 12:20:19 hydra01 kernel: [2090727.720990] [] proc_reg_open+0xa0/0x180 Sep 16 12:20:19 hydra01 kernel: [2090727.720997] [] ? seq_release_private+0x60/0x60 Sep 16 12:20:19 hydra01 kernel: [2090727.721003] [] do_dentry_open+0x203/0x290 Sep 16 12:20:19 hydra01 kernel: [2090727.721008] [] ? init_once+0x20/0x20 Sep 16 12:20:19 hydra01 kernel: [2090727.721012] [] vfs_open+0x49/0x50 Sep 16 12:20:19 hydra01 kernel: [2090727.721018] [] do_last+0x296/0xea0 Sep 16 12:20:19 hydra01 kernel: [2090727.721022] [] ? inode_permission+0x18/0x50 Sep 16 12:20:19 hydra01 kernel: [2090727.721027] [] ? link_path_walk+0x6b/0x880 Sep 16 12:20:19 hydra01 kernel: [2090727.721032] [] path_openat+0xb3/0x4b0 Sep 16 12:20:19 hydra01 kernel: [2090727.721038] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:20:19 hydra01 kernel: [2090727.721044] [] ? unmap_region+0xdb/0x120 Sep 16 12:20:19 hydra01 kernel: [2090727.721049] [] do_filp_open+0x41/0xa0 Sep 16 12:20:19 hydra01 kernel: [2090727.721055] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:20:19 hydra01 kernel: [2090727.721059] [] do_sys_open+0xf3/0x230 Sep 16 12:20:19 hydra01 kernel: [2090727.721064] [] sys_open+0x21/0x30 Sep 16 12:20:19 hydra01 kernel: [2090727.721070] [] system_call_fastpath+0x1a/0x1f Sep 16 12:20:19 hydra01 kernel: [2090727.721072] Mem-Info: Sep 16 12:20:19 hydra01 kernel: [2090727.721075] Node 0 DMA per-cpu: Sep 16 12:20:19 hydra01 kernel: [2090727.721079] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721082] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721085] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721087] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721090] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721093] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721095] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721098] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721101] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721103] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721106] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721108] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721111] Node 0 DMA32 per-cpu: Sep 16 12:20:19 hydra01 kernel: [2090727.721114] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721117] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721120] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721122] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721125] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721128] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721130] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721133] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721136] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721138] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721141] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721143] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721145] Node 0 Normal per-cpu: Sep 16 12:20:19 hydra01 kernel: [2090727.721149] CPU 0: hi: 186, btch: 31 usd: 167 Sep 16 12:20:19 hydra01 kernel: [2090727.721152] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721154] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721157] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721160] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721162] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721165] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721167] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721170] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721173] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721175] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721178] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721186] active_anon:1274582 inactive_anon:830240 isolated_anon:0 Sep 16 12:20:19 hydra01 kernel: [2090727.721186] active_file:1869697 inactive_file:3500530 isolated_file:0 Sep 16 12:20:19 hydra01 kernel: [2090727.721186] unevictable:0 dirty:22 writeback:0 unstable:0 Sep 16 12:20:19 hydra01 kernel: [2090727.721186] free:64355 slab_reclaimable:380930 slab_unreclaimable:106185 Sep 16 12:20:19 hydra01 kernel: [2090727.721186] mapped:25967 shmem:151 pagetables:9562 bounce:0 Sep 16 12:20:19 hydra01 kernel: [2090727.721186] free_cma:0 Sep 16 12:20:19 hydra01 kernel: [2090727.721192] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:20:19 hydra01 kernel: [2090727.721201] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:20:19 hydra01 kernel: [2090727.721207] Node 0 DMA32 free:145908kB min:6224kB low:7780kB high:9336kB active_anon:432604kB inactive_anon:793268kB active_file:95236kB inactive_file:1376160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171116kB slab_unreclaimable:45796kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:20:19 hydra01 kernel: [2090727.721217] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:20:19 hydra01 kernel: [2090727.721222] Node 0 Normal free:95960kB min:61324kB low:76652kB high:91984kB active_anon:4665724kB inactive_anon:2527692kB active_file:7383552kB inactive_file:12625576kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:88kB writeback:0kB mapped:99944kB shmem:604kB slab_reclaimable:1352604kB slab_unreclaimable:378944kB kernel_stack:8016kB pagetables:36720kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:20:19 hydra01 kernel: [2090727.721231] lowmem_reserve[]: 0 0 0 0 Sep 16 12:20:19 hydra01 kernel: [2090727.721236] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:20:19 hydra01 kernel: [2090727.721258] Node 0 DMA32: 2262*4kB (UEM) 3361*8kB (UEM) 6074*16kB (UEM) 389*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146080kB Sep 16 12:20:19 hydra01 kernel: [2090727.721276] Node 0 Normal: 8045*4kB (UEMR) 5204*8kB (UEMR) 1233*16kB (UEMR) 94*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 98212kB Sep 16 12:20:19 hydra01 kernel: [2090727.721296] 5802415 total pagecache pages Sep 16 12:20:19 hydra01 kernel: [2090727.721298] 432210 pages in swap cache Sep 16 12:20:19 hydra01 kernel: [2090727.721301] Swap cache stats: add 20773479, delete 20341269, find 12661773/14316162 Sep 16 12:20:19 hydra01 kernel: [2090727.721304] Free swap = 185330320kB Sep 16 12:20:19 hydra01 kernel: [2090727.721305] Total swap = 187617276kB Sep 16 12:20:19 hydra01 kernel: [2090727.916777] 8388592 pages RAM Sep 16 12:20:19 hydra01 kernel: [2090727.916789] 151696 pages reserved Sep 16 12:20:19 hydra01 kernel: [2090727.916799] 8609348 pages shared Sep 16 12:20:19 hydra01 kernel: [2090727.916809] 3540114 pages non-shared Sep 16 12:20:33 hydra01 kernel: [2090741.763075] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:20:33 hydra01 kernel: [2090741.763081] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:20:33 hydra01 kernel: [2090741.763082] Call Trace: Sep 16 12:20:33 hydra01 kernel: [2090741.763091] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:20:33 hydra01 kernel: [2090741.763094] [] ? drain_local_pages+0x16/0x20 Sep 16 12:20:33 hydra01 kernel: [2090741.763099] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:20:33 hydra01 kernel: [2090741.763102] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:20:33 hydra01 kernel: [2090741.763107] [] alloc_pages_current+0xb8/0x180 Sep 16 12:20:33 hydra01 kernel: [2090741.763110] [] __get_free_pages+0xe/0x50 Sep 16 12:20:33 hydra01 kernel: [2090741.763114] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:20:33 hydra01 kernel: [2090741.763117] [] __kmalloc+0x161/0x170 Sep 16 12:20:33 hydra01 kernel: [2090741.763121] [] ? get_iowait_time+0x60/0x60 Sep 16 12:20:33 hydra01 kernel: [2090741.763123] [] stat_open+0x56/0xc0 Sep 16 12:20:33 hydra01 kernel: [2090741.763127] [] proc_reg_open+0xa0/0x180 Sep 16 12:20:33 hydra01 kernel: [2090741.763131] [] ? seq_release_private+0x60/0x60 Sep 16 12:20:33 hydra01 kernel: [2090741.763134] [] do_dentry_open+0x203/0x290 Sep 16 12:20:33 hydra01 kernel: [2090741.763136] [] ? init_once+0x20/0x20 Sep 16 12:20:33 hydra01 kernel: [2090741.763139] [] vfs_open+0x49/0x50 Sep 16 12:20:33 hydra01 kernel: [2090741.763142] [] do_last+0x296/0xea0 Sep 16 12:20:33 hydra01 kernel: [2090741.763144] [] ? inode_permission+0x18/0x50 Sep 16 12:20:33 hydra01 kernel: [2090741.763146] [] ? link_path_walk+0x6b/0x880 Sep 16 12:20:33 hydra01 kernel: [2090741.763149] [] path_openat+0xb3/0x4b0 Sep 16 12:20:33 hydra01 kernel: [2090741.763152] [] do_filp_open+0x41/0xa0 Sep 16 12:20:33 hydra01 kernel: [2090741.763155] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:20:33 hydra01 kernel: [2090741.763157] [] do_sys_open+0xf3/0x230 Sep 16 12:20:33 hydra01 kernel: [2090741.763160] [] ? sys_select+0xcb/0x110 Sep 16 12:20:33 hydra01 kernel: [2090741.763163] [] sys_open+0x21/0x30 Sep 16 12:20:33 hydra01 kernel: [2090741.763166] [] system_call_fastpath+0x1a/0x1f Sep 16 12:20:33 hydra01 kernel: [2090741.763167] Mem-Info: Sep 16 12:20:33 hydra01 kernel: [2090741.763169] Node 0 DMA per-cpu: Sep 16 12:20:33 hydra01 kernel: [2090741.763171] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763173] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763174] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763175] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763177] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763178] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763179] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763181] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763182] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763184] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763185] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763186] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763187] Node 0 DMA32 per-cpu: Sep 16 12:20:33 hydra01 kernel: [2090741.763190] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763191] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763192] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763194] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763195] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763196] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763198] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763199] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763201] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763202] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763203] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763205] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763206] Node 0 Normal per-cpu: Sep 16 12:20:33 hydra01 kernel: [2090741.763208] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763209] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763210] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763212] CPU 3: hi: 186, btch: 31 usd: 169 Sep 16 12:20:33 hydra01 kernel: [2090741.763213] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763215] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763216] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763217] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763219] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763220] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763222] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763223] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763227] active_anon:1274550 inactive_anon:830211 isolated_anon:0 Sep 16 12:20:33 hydra01 kernel: [2090741.763227] active_file:1869705 inactive_file:3489505 isolated_file:0 Sep 16 12:20:33 hydra01 kernel: [2090741.763227] unevictable:0 dirty:22 writeback:0 unstable:0 Sep 16 12:20:33 hydra01 kernel: [2090741.763227] free:76498 slab_reclaimable:380822 slab_unreclaimable:105937 Sep 16 12:20:33 hydra01 kernel: [2090741.763227] mapped:26106 shmem:149 pagetables:9609 bounce:0 Sep 16 12:20:33 hydra01 kernel: [2090741.763227] free_cma:0 Sep 16 12:20:33 hydra01 kernel: [2090741.763230] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:20:33 hydra01 kernel: [2090741.763235] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:20:33 hydra01 kernel: [2090741.763239] Node 0 DMA32 free:146176kB min:6224kB low:7780kB high:9336kB active_anon:432596kB inactive_anon:793268kB active_file:95236kB inactive_file:1376160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3924kB shmem:0kB slab_reclaimable:171080kB slab_unreclaimable:45508kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:20:33 hydra01 kernel: [2090741.763244] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:20:33 hydra01 kernel: [2090741.763246] Node 0 Normal free:144264kB min:61324kB low:76652kB high:91984kB active_anon:4665604kB inactive_anon:2527576kB active_file:7383584kB inactive_file:12581476kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:88kB writeback:0kB mapped:100500kB shmem:596kB slab_reclaimable:1352208kB slab_unreclaimable:378240kB kernel_stack:8016kB pagetables:36908kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:20:33 hydra01 kernel: [2090741.763251] lowmem_reserve[]: 0 0 0 0 Sep 16 12:20:33 hydra01 kernel: [2090741.763254] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:20:33 hydra01 kernel: [2090741.763265] Node 0 DMA32: 2266*4kB (UEM) 3361*8kB (UEM) 6080*16kB (UEM) 394*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 146416kB Sep 16 12:20:33 hydra01 kernel: [2090741.763275] Node 0 Normal: 8191*4kB (UEMR) 5180*8kB (UEMR) 4140*16kB (UEMR) 107*32kB (UEMR) 8*64kB (ER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 145660kB Sep 16 12:20:33 hydra01 kernel: [2090741.763285] 5791533 total pagecache pages Sep 16 12:20:33 hydra01 kernel: [2090741.763287] 432210 pages in swap cache Sep 16 12:20:33 hydra01 kernel: [2090741.763288] Swap cache stats: add 20773479, delete 20341269, find 12661773/14316162 Sep 16 12:20:33 hydra01 kernel: [2090741.763290] Free swap = 185330320kB Sep 16 12:20:33 hydra01 kernel: [2090741.763291] Total swap = 187617276kB Sep 16 12:20:33 hydra01 kernel: [2090741.957775] 8388592 pages RAM Sep 16 12:20:33 hydra01 kernel: [2090741.957785] 151696 pages reserved Sep 16 12:20:33 hydra01 kernel: [2090741.957794] 8605708 pages shared Sep 16 12:20:33 hydra01 kernel: [2090741.957802] 3544763 pages non-shared Sep 16 12:25:50 hydra01 kernel: [2091057.892792] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:25:50 hydra01 kernel: [2091057.892800] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:25:50 hydra01 kernel: [2091057.892803] Call Trace: Sep 16 12:25:50 hydra01 kernel: [2091057.892817] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:25:50 hydra01 kernel: [2091057.892824] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:25:50 hydra01 kernel: [2091057.892831] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:25:50 hydra01 kernel: [2091057.892838] [] alloc_pages_current+0xb8/0x180 Sep 16 12:25:50 hydra01 kernel: [2091057.892844] [] __get_free_pages+0xe/0x50 Sep 16 12:25:50 hydra01 kernel: [2091057.892851] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:25:50 hydra01 kernel: [2091057.892857] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:25:50 hydra01 kernel: [2091057.892863] [] __kmalloc+0x161/0x170 Sep 16 12:25:50 hydra01 kernel: [2091057.892870] [] ? amiga_partition+0x2c4/0x4d0 Sep 16 12:25:50 hydra01 kernel: [2091057.892876] [] ? get_iowait_time+0x60/0x60 Sep 16 12:25:50 hydra01 kernel: [2091057.892881] [] stat_open+0x56/0xc0 Sep 16 12:25:50 hydra01 kernel: [2091057.892887] [] proc_reg_open+0xa0/0x180 Sep 16 12:25:50 hydra01 kernel: [2091057.892894] [] ? seq_release_private+0x60/0x60 Sep 16 12:25:50 hydra01 kernel: [2091057.892899] [] do_dentry_open+0x203/0x290 Sep 16 12:25:50 hydra01 kernel: [2091057.892904] [] ? init_once+0x20/0x20 Sep 16 12:25:50 hydra01 kernel: [2091057.892909] [] vfs_open+0x49/0x50 Sep 16 12:25:50 hydra01 kernel: [2091057.892914] [] do_last+0x296/0xea0 Sep 16 12:25:50 hydra01 kernel: [2091057.892919] [] ? inode_permission+0x18/0x50 Sep 16 12:25:50 hydra01 kernel: [2091057.892923] [] ? link_path_walk+0x6b/0x880 Sep 16 12:25:50 hydra01 kernel: [2091057.892928] [] path_openat+0xb3/0x4b0 Sep 16 12:25:50 hydra01 kernel: [2091057.892935] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:25:50 hydra01 kernel: [2091057.892941] [] ? unmap_region+0xdb/0x120 Sep 16 12:25:50 hydra01 kernel: [2091057.892946] [] do_filp_open+0x41/0xa0 Sep 16 12:25:50 hydra01 kernel: [2091057.892952] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:25:50 hydra01 kernel: [2091057.892957] [] do_sys_open+0xf3/0x230 Sep 16 12:25:50 hydra01 kernel: [2091057.892961] [] sys_open+0x21/0x30 Sep 16 12:25:50 hydra01 kernel: [2091057.892967] [] system_call_fastpath+0x1a/0x1f Sep 16 12:25:50 hydra01 kernel: [2091057.892970] Mem-Info: Sep 16 12:25:50 hydra01 kernel: [2091057.892972] Node 0 DMA per-cpu: Sep 16 12:25:50 hydra01 kernel: [2091057.892977] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892980] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892982] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892985] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892988] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892990] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892993] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892995] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.892998] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893000] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893003] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893006] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893008] Node 0 DMA32 per-cpu: Sep 16 12:25:50 hydra01 kernel: [2091057.893012] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893015] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893017] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893020] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893023] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893025] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893028] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893031] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893033] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893036] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893039] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893041] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893043] Node 0 Normal per-cpu: Sep 16 12:25:50 hydra01 kernel: [2091057.893046] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893049] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893052] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893054] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893057] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893059] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893062] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893065] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893067] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893070] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893073] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893075] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893083] active_anon:1278274 inactive_anon:830210 isolated_anon:0 Sep 16 12:25:50 hydra01 kernel: [2091057.893083] active_file:1869787 inactive_file:3480018 isolated_file:0 Sep 16 12:25:50 hydra01 kernel: [2091057.893083] unevictable:0 dirty:16 writeback:0 unstable:0 Sep 16 12:25:50 hydra01 kernel: [2091057.893083] free:82221 slab_reclaimable:379944 slab_unreclaimable:104682 Sep 16 12:25:50 hydra01 kernel: [2091057.893083] mapped:25848 shmem:151 pagetables:9572 bounce:0 Sep 16 12:25:50 hydra01 kernel: [2091057.893083] free_cma:0 Sep 16 12:25:50 hydra01 kernel: [2091057.893089] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:25:50 hydra01 kernel: [2091057.893098] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:25:50 hydra01 kernel: [2091057.893104] Node 0 DMA32 free:139820kB min:6224kB low:7780kB high:9336kB active_anon:433824kB inactive_anon:793260kB active_file:95248kB inactive_file:1380984kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3184kB shmem:0kB slab_reclaimable:170944kB slab_unreclaimable:44812kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:25:50 hydra01 kernel: [2091057.893114] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:25:50 hydra01 kernel: [2091057.893119] Node 0 Normal free:173220kB min:61324kB low:76652kB high:91984kB active_anon:4679272kB inactive_anon:2527580kB active_file:7383900kB inactive_file:12539088kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:64kB writeback:0kB mapped:100208kB shmem:604kB slab_reclaimable:1348832kB slab_unreclaimable:373916kB kernel_stack:8032kB pagetables:36760kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:25:50 hydra01 kernel: [2091057.893128] lowmem_reserve[]: 0 0 0 0 Sep 16 12:25:50 hydra01 kernel: [2091057.893133] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:25:50 hydra01 kernel: [2091057.893155] Node 0 DMA32: 2596*4kB (UEM) 2818*8kB (UEM) 5926*16kB (UEM) 383*32kB (UEMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140512kB Sep 16 12:25:50 hydra01 kernel: [2091057.893173] Node 0 Normal: 13445*4kB (UEMR) 10659*8kB (UEMR) 2033*16kB (UEMR) 36*32kB (UEMR) 9*64kB (EMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 174588kB Sep 16 12:25:50 hydra01 kernel: [2091057.893193] 5782114 total pagecache pages Sep 16 12:25:50 hydra01 kernel: [2091057.893196] 432112 pages in swap cache Sep 16 12:25:50 hydra01 kernel: [2091057.893199] Swap cache stats: add 20773554, delete 20341442, find 12661886/14316282 Sep 16 12:25:50 hydra01 kernel: [2091057.893201] Free swap = 185330692kB Sep 16 12:25:50 hydra01 kernel: [2091057.893203] Total swap = 187617276kB Sep 16 12:25:50 hydra01 kernel: [2091058.088610] 8388592 pages RAM Sep 16 12:25:50 hydra01 kernel: [2091058.088625] 151696 pages reserved Sep 16 12:25:50 hydra01 kernel: [2091058.088638] 8513599 pages shared Sep 16 12:25:50 hydra01 kernel: [2091058.088650] 3633642 pages non-shared Sep 16 12:29:50 hydra01 kernel: [2091298.064595] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:29:50 hydra01 kernel: [2091298.064603] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:29:50 hydra01 kernel: [2091298.064605] Call Trace: Sep 16 12:29:50 hydra01 kernel: [2091298.064619] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:29:50 hydra01 kernel: [2091298.064627] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:29:50 hydra01 kernel: [2091298.064633] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:29:50 hydra01 kernel: [2091298.064641] [] alloc_pages_current+0xb8/0x180 Sep 16 12:29:50 hydra01 kernel: [2091298.064646] [] __get_free_pages+0xe/0x50 Sep 16 12:29:50 hydra01 kernel: [2091298.064654] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:29:50 hydra01 kernel: [2091298.064659] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:29:50 hydra01 kernel: [2091298.064665] [] __kmalloc+0x161/0x170 Sep 16 12:29:50 hydra01 kernel: [2091298.064673] [] ? setkey+0x81/0x100 Sep 16 12:29:50 hydra01 kernel: [2091298.064680] [] ? get_iowait_time+0x60/0x60 Sep 16 12:29:50 hydra01 kernel: [2091298.064684] [] stat_open+0x56/0xc0 Sep 16 12:29:50 hydra01 kernel: [2091298.064691] [] proc_reg_open+0xa0/0x180 Sep 16 12:29:50 hydra01 kernel: [2091298.064698] [] ? seq_release_private+0x60/0x60 Sep 16 12:29:50 hydra01 kernel: [2091298.064704] [] do_dentry_open+0x203/0x290 Sep 16 12:29:50 hydra01 kernel: [2091298.064709] [] ? init_once+0x20/0x20 Sep 16 12:29:50 hydra01 kernel: [2091298.064713] [] vfs_open+0x49/0x50 Sep 16 12:29:50 hydra01 kernel: [2091298.064718] [] do_last+0x296/0xea0 Sep 16 12:29:50 hydra01 kernel: [2091298.064723] [] ? inode_permission+0x18/0x50 Sep 16 12:29:50 hydra01 kernel: [2091298.064727] [] ? link_path_walk+0x6b/0x880 Sep 16 12:29:50 hydra01 kernel: [2091298.064732] [] path_openat+0xb3/0x4b0 Sep 16 12:29:50 hydra01 kernel: [2091298.064739] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:29:50 hydra01 kernel: [2091298.064745] [] ? unmap_region+0xdb/0x120 Sep 16 12:29:50 hydra01 kernel: [2091298.064750] [] do_filp_open+0x41/0xa0 Sep 16 12:29:50 hydra01 kernel: [2091298.064755] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:29:50 hydra01 kernel: [2091298.064760] [] do_sys_open+0xf3/0x230 Sep 16 12:29:50 hydra01 kernel: [2091298.064765] [] sys_open+0x21/0x30 Sep 16 12:29:50 hydra01 kernel: [2091298.064770] [] system_call_fastpath+0x1a/0x1f Sep 16 12:29:50 hydra01 kernel: [2091298.064773] Mem-Info: Sep 16 12:29:50 hydra01 kernel: [2091298.064775] Node 0 DMA per-cpu: Sep 16 12:29:50 hydra01 kernel: [2091298.064780] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064782] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064785] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064788] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064790] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064793] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064796] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064798] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064801] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064804] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064806] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064809] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064811] Node 0 DMA32 per-cpu: Sep 16 12:29:50 hydra01 kernel: [2091298.064815] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064818] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064820] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064823] CPU 3: hi: 186, btch: 31 usd: 85 Sep 16 12:29:50 hydra01 kernel: [2091298.064826] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064828] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064831] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064834] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064836] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064839] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064841] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064844] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064846] Node 0 Normal per-cpu: Sep 16 12:29:50 hydra01 kernel: [2091298.064849] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064852] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064855] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064858] CPU 3: hi: 186, btch: 31 usd: 159 Sep 16 12:29:50 hydra01 kernel: [2091298.064860] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064863] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064865] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064868] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064871] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064873] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064876] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064879] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064886] active_anon:1283377 inactive_anon:826967 isolated_anon:0 Sep 16 12:29:50 hydra01 kernel: [2091298.064886] active_file:1869871 inactive_file:3486433 isolated_file:0 Sep 16 12:29:50 hydra01 kernel: [2091298.064886] unevictable:0 dirty:96 writeback:0 unstable:0 Sep 16 12:29:50 hydra01 kernel: [2091298.064886] free:60353 slab_reclaimable:378392 slab_unreclaimable:104011 Sep 16 12:29:50 hydra01 kernel: [2091298.064886] mapped:25808 shmem:151 pagetables:9731 bounce:0 Sep 16 12:29:50 hydra01 kernel: [2091298.064886] free_cma:0 Sep 16 12:29:50 hydra01 kernel: [2091298.064892] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:29:50 hydra01 kernel: [2091298.064901] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:29:50 hydra01 kernel: [2091298.064907] Node 0 DMA32 free:133352kB min:6224kB low:7780kB high:9336kB active_anon:436468kB inactive_anon:793196kB active_file:95268kB inactive_file:1384324kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:44kB writeback:0kB mapped:3076kB shmem:8kB slab_reclaimable:170840kB slab_unreclaimable:44468kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:84 all_unreclaimable? no Sep 16 12:29:50 hydra01 kernel: [2091298.064917] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:29:50 hydra01 kernel: [2091298.064922] Node 0 Normal free:92508kB min:61324kB low:76652kB high:91984kB active_anon:4697040kB inactive_anon:2514672kB active_file:7384216kB inactive_file:12561024kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:340kB writeback:0kB mapped:100156kB shmem:596kB slab_reclaimable:1342728kB slab_unreclaimable:371576kB kernel_stack:8008kB pagetables:37396kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:29:50 hydra01 kernel: [2091298.064932] lowmem_reserve[]: 0 0 0 0 Sep 16 12:29:50 hydra01 kernel: [2091298.064936] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:29:50 hydra01 kernel: [2091298.064958] Node 0 DMA32: 2223*4kB (UEM) 2219*8kB (UEM) 5809*16kB (UEM) 427*32kB (UEMR) 3*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 133956kB Sep 16 12:29:50 hydra01 kernel: [2091298.064978] Node 0 Normal: 9559*4kB (UEMR) 4387*8kB (UEMR) 1112*16kB (UEMR) 22*32kB (UEMR) 7*64kB (MR) 5*128kB (MR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 93940kB Sep 16 12:29:50 hydra01 kernel: [2091298.064998] 5788034 total pagecache pages Sep 16 12:29:50 hydra01 kernel: [2091298.065000] 431717 pages in swap cache Sep 16 12:29:50 hydra01 kernel: [2091298.065003] Swap cache stats: add 20774101, delete 20342384, find 12661887/14316283 Sep 16 12:29:50 hydra01 kernel: [2091298.065005] Free swap = 185328524kB Sep 16 12:29:50 hydra01 kernel: [2091298.065007] Total swap = 187617276kB Sep 16 12:29:50 hydra01 kernel: [2091298.255168] 8388592 pages RAM Sep 16 12:29:50 hydra01 kernel: [2091298.255172] 151696 pages reserved Sep 16 12:29:50 hydra01 kernel: [2091298.255175] 8434924 pages shared Sep 16 12:29:50 hydra01 kernel: [2091298.255177] 3723275 pages non-shared Sep 16 12:36:34 hydra01 kernel: [2091701.634193] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:36:34 hydra01 kernel: [2091701.634199] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:36:34 hydra01 kernel: [2091701.634200] Call Trace: Sep 16 12:36:34 hydra01 kernel: [2091701.634209] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:36:34 hydra01 kernel: [2091701.634214] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:36:34 hydra01 kernel: [2091701.634217] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:36:34 hydra01 kernel: [2091701.634222] [] alloc_pages_current+0xb8/0x180 Sep 16 12:36:34 hydra01 kernel: [2091701.634225] [] __get_free_pages+0xe/0x50 Sep 16 12:36:34 hydra01 kernel: [2091701.634229] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:36:34 hydra01 kernel: [2091701.634232] [] __kmalloc+0x161/0x170 Sep 16 12:36:34 hydra01 kernel: [2091701.634236] [] ? get_iowait_time+0x60/0x60 Sep 16 12:36:34 hydra01 kernel: [2091701.634238] [] stat_open+0x56/0xc0 Sep 16 12:36:34 hydra01 kernel: [2091701.634242] [] proc_reg_open+0xa0/0x180 Sep 16 12:36:34 hydra01 kernel: [2091701.634246] [] ? seq_release_private+0x60/0x60 Sep 16 12:36:34 hydra01 kernel: [2091701.634249] [] do_dentry_open+0x203/0x290 Sep 16 12:36:34 hydra01 kernel: [2091701.634252] [] ? init_once+0x20/0x20 Sep 16 12:36:34 hydra01 kernel: [2091701.634254] [] vfs_open+0x49/0x50 Sep 16 12:36:34 hydra01 kernel: [2091701.634257] [] do_last+0x296/0xea0 Sep 16 12:36:34 hydra01 kernel: [2091701.634260] [] ? inode_permission+0x18/0x50 Sep 16 12:36:34 hydra01 kernel: [2091701.634262] [] ? link_path_walk+0x6b/0x880 Sep 16 12:36:34 hydra01 kernel: [2091701.634265] [] path_openat+0xb3/0x4b0 Sep 16 12:36:34 hydra01 kernel: [2091701.634268] [] do_filp_open+0x41/0xa0 Sep 16 12:36:34 hydra01 kernel: [2091701.634271] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:36:34 hydra01 kernel: [2091701.634273] [] do_sys_open+0xf3/0x230 Sep 16 12:36:34 hydra01 kernel: [2091701.634276] [] ? sys_select+0xcb/0x110 Sep 16 12:36:34 hydra01 kernel: [2091701.634278] [] sys_open+0x21/0x30 Sep 16 12:36:34 hydra01 kernel: [2091701.634281] [] system_call_fastpath+0x1a/0x1f Sep 16 12:36:34 hydra01 kernel: [2091701.634283] Mem-Info: Sep 16 12:36:34 hydra01 kernel: [2091701.634284] Node 0 DMA per-cpu: Sep 16 12:36:34 hydra01 kernel: [2091701.634287] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634288] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634290] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634291] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634293] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634294] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634295] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634297] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634298] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634299] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634301] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634302] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634303] Node 0 DMA32 per-cpu: Sep 16 12:36:34 hydra01 kernel: [2091701.634305] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634307] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634308] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634309] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634311] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634312] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634313] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634315] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634316] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634317] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634319] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634320] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634321] Node 0 Normal per-cpu: Sep 16 12:36:34 hydra01 kernel: [2091701.634323] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634325] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634326] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634327] CPU 3: hi: 186, btch: 31 usd: 165 Sep 16 12:36:34 hydra01 kernel: [2091701.634329] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634330] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634332] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634333] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634334] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634336] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634337] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634339] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634343] active_anon:1282047 inactive_anon:825461 isolated_anon:0 Sep 16 12:36:34 hydra01 kernel: [2091701.634343] active_file:1869999 inactive_file:3459693 isolated_file:0 Sep 16 12:36:34 hydra01 kernel: [2091701.634343] unevictable:0 dirty:114 writeback:0 unstable:0 Sep 16 12:36:34 hydra01 kernel: [2091701.634343] free:80456 slab_reclaimable:376773 slab_unreclaimable:103412 Sep 16 12:36:34 hydra01 kernel: [2091701.634343] mapped:25870 shmem:151 pagetables:9566 bounce:0 Sep 16 12:36:34 hydra01 kernel: [2091701.634343] free_cma:0 Sep 16 12:36:34 hydra01 kernel: [2091701.634346] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:36:34 hydra01 kernel: [2091701.634351] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:36:34 hydra01 kernel: [2091701.634354] Node 0 DMA32 free:143556kB min:6224kB low:7780kB high:9336kB active_anon:431864kB inactive_anon:793028kB active_file:95300kB inactive_file:1377592kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2900kB shmem:0kB slab_reclaimable:169176kB slab_unreclaimable:44168kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:36:34 hydra01 kernel: [2091701.634359] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:36:34 hydra01 kernel: [2091701.634362] Node 0 Normal free:162716kB min:61324kB low:76652kB high:91984kB active_anon:4696324kB inactive_anon:2508816kB active_file:7384696kB inactive_file:12460796kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:456kB writeback:0kB mapped:100580kB shmem:604kB slab_reclaimable:1337916kB slab_unreclaimable:369480kB kernel_stack:8024kB pagetables:36736kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:36:34 hydra01 kernel: [2091701.634367] lowmem_reserve[]: 0 0 0 0 Sep 16 12:36:34 hydra01 kernel: [2091701.634369] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:36:34 hydra01 kernel: [2091701.634380] Node 0 DMA32: 5439*4kB (UEM) 1844*8kB (UEM) 5818*16kB (UEM) 434*32kB (UEMR) 4*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 144252kB Sep 16 12:36:34 hydra01 kernel: [2091701.634390] Node 0 Normal: 11408*4kB (UEMR) 8032*8kB (UEMR) 3018*16kB (UEMR) 137*32kB (UEMR) 7*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 164288kB Sep 16 12:36:34 hydra01 kernel: [2091701.634401] 5761456 total pagecache pages Sep 16 12:36:34 hydra01 kernel: [2091701.634402] 431675 pages in swap cache Sep 16 12:36:34 hydra01 kernel: [2091701.634404] Swap cache stats: add 20774363, delete 20342688, find 12661889/14316285 Sep 16 12:36:34 hydra01 kernel: [2091701.634405] Free swap = 185327480kB Sep 16 12:36:34 hydra01 kernel: [2091701.634406] Total swap = 187617276kB Sep 16 12:36:34 hydra01 kernel: [2091701.821077] 8388592 pages RAM Sep 16 12:36:34 hydra01 kernel: [2091701.821079] 151696 pages reserved Sep 16 12:36:34 hydra01 kernel: [2091701.821080] 8324931 pages shared Sep 16 12:36:34 hydra01 kernel: [2091701.821081] 3818868 pages non-shared Sep 16 12:37:09 hydra01 kernel: [2091736.739824] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:37:09 hydra01 kernel: [2091736.739830] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:37:09 hydra01 kernel: [2091736.739831] Call Trace: Sep 16 12:37:09 hydra01 kernel: [2091736.739840] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:37:09 hydra01 kernel: [2091736.739845] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:37:09 hydra01 kernel: [2091736.739848] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:37:09 hydra01 kernel: [2091736.739852] [] alloc_pages_current+0xb8/0x180 Sep 16 12:37:09 hydra01 kernel: [2091736.739855] [] __get_free_pages+0xe/0x50 Sep 16 12:37:09 hydra01 kernel: [2091736.739860] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:37:09 hydra01 kernel: [2091736.739863] [] __kmalloc+0x161/0x170 Sep 16 12:37:09 hydra01 kernel: [2091736.739867] [] ? get_iowait_time+0x60/0x60 Sep 16 12:37:09 hydra01 kernel: [2091736.739869] [] stat_open+0x56/0xc0 Sep 16 12:37:09 hydra01 kernel: [2091736.739873] [] proc_reg_open+0xa0/0x180 Sep 16 12:37:09 hydra01 kernel: [2091736.739877] [] ? seq_release_private+0x60/0x60 Sep 16 12:37:09 hydra01 kernel: [2091736.739880] [] do_dentry_open+0x203/0x290 Sep 16 12:37:09 hydra01 kernel: [2091736.739883] [] ? init_once+0x20/0x20 Sep 16 12:37:09 hydra01 kernel: [2091736.739885] [] vfs_open+0x49/0x50 Sep 16 12:37:09 hydra01 kernel: [2091736.739888] [] do_last+0x296/0xea0 Sep 16 12:37:09 hydra01 kernel: [2091736.739890] [] ? inode_permission+0x18/0x50 Sep 16 12:37:09 hydra01 kernel: [2091736.739893] [] ? link_path_walk+0x6b/0x880 Sep 16 12:37:09 hydra01 kernel: [2091736.739896] [] path_openat+0xb3/0x4b0 Sep 16 12:37:09 hydra01 kernel: [2091736.739899] [] do_filp_open+0x41/0xa0 Sep 16 12:37:09 hydra01 kernel: [2091736.739901] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:37:09 hydra01 kernel: [2091736.739904] [] do_sys_open+0xf3/0x230 Sep 16 12:37:09 hydra01 kernel: [2091736.739907] [] ? sys_select+0xcb/0x110 Sep 16 12:37:09 hydra01 kernel: [2091736.739909] [] sys_open+0x21/0x30 Sep 16 12:37:09 hydra01 kernel: [2091736.739912] [] system_call_fastpath+0x1a/0x1f Sep 16 12:37:09 hydra01 kernel: [2091736.739914] Mem-Info: Sep 16 12:37:09 hydra01 kernel: [2091736.739915] Node 0 DMA per-cpu: Sep 16 12:37:09 hydra01 kernel: [2091736.739918] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739919] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739921] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739922] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739924] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739925] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739927] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739928] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739929] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739931] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739932] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739934] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739935] Node 0 DMA32 per-cpu: Sep 16 12:37:09 hydra01 kernel: [2091736.739937] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739938] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739940] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739941] CPU 3: hi: 186, btch: 31 usd: 3 Sep 16 12:37:09 hydra01 kernel: [2091736.739942] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739944] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739945] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739946] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739948] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739949] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739951] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739952] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739953] Node 0 Normal per-cpu: Sep 16 12:37:09 hydra01 kernel: [2091736.739955] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739956] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739958] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739959] CPU 3: hi: 186, btch: 31 usd: 155 Sep 16 12:37:09 hydra01 kernel: [2091736.739960] CPU 4: hi: 186, btch: 31 usd: 30 Sep 16 12:37:09 hydra01 kernel: [2091736.739962] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739963] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739965] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739966] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739967] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739969] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739970] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:09 hydra01 kernel: [2091736.739974] active_anon:1282449 inactive_anon:824585 isolated_anon:0 Sep 16 12:37:09 hydra01 kernel: [2091736.739974] active_file:1869132 inactive_file:3464636 isolated_file:0 Sep 16 12:37:09 hydra01 kernel: [2091736.739974] unevictable:0 dirty:0 writeback:0 unstable:0 Sep 16 12:37:09 hydra01 kernel: [2091736.739974] free:82022 slab_reclaimable:375716 slab_unreclaimable:102899 Sep 16 12:37:09 hydra01 kernel: [2091736.739974] mapped:25703 shmem:151 pagetables:9566 bounce:0 Sep 16 12:37:09 hydra01 kernel: [2091736.739974] free_cma:0 Sep 16 12:37:09 hydra01 kernel: [2091736.739977] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:37:09 hydra01 kernel: [2091736.739982] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:37:09 hydra01 kernel: [2091736.739985] Node 0 DMA32 free:141096kB min:6224kB low:7780kB high:9336kB active_anon:430228kB inactive_anon:793020kB active_file:95304kB inactive_file:1382884kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:8kB slab_reclaimable:168172kB slab_unreclaimable:44148kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:09 hydra01 kernel: [2091736.739990] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:37:09 hydra01 kernel: [2091736.739993] Node 0 Normal free:171148kB min:61324kB low:76652kB high:91984kB active_anon:4699568kB inactive_anon:2505320kB active_file:7381224kB inactive_file:12475660kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:0kB mapped:100040kB shmem:596kB slab_reclaimable:1334692kB slab_unreclaimable:367448kB kernel_stack:8024kB pagetables:36736kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:09 hydra01 kernel: [2091736.739998] lowmem_reserve[]: 0 0 0 0 Sep 16 12:37:09 hydra01 kernel: [2091736.740000] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:37:09 hydra01 kernel: [2091736.740012] Node 0 DMA32: 4891*4kB (UEM) 1845*8kB (UEM) 5822*16kB (UEM) 428*32kB (UEMR) 4*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141940kB Sep 16 12:37:09 hydra01 kernel: [2091736.740022] Node 0 Normal: 14313*4kB (UEMR) 5920*8kB (UEMR) 2955*16kB (UEMR) 527*32kB (UEMR) 34*64kB (UEMR) 5*128kB (MR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 172596kB Sep 16 12:37:09 hydra01 kernel: [2091736.740032] 5765599 total pagecache pages Sep 16 12:37:09 hydra01 kernel: [2091736.740033] 431669 pages in swap cache Sep 16 12:37:09 hydra01 kernel: [2091736.740035] Swap cache stats: add 20774521, delete 20342852, find 12661890/14316286 Sep 16 12:37:09 hydra01 kernel: [2091736.740036] Free swap = 185326852kB Sep 16 12:37:09 hydra01 kernel: [2091736.740037] Total swap = 187617276kB Sep 16 12:37:09 hydra01 kernel: [2091736.847170] 8388592 pages RAM Sep 16 12:37:09 hydra01 kernel: [2091736.847173] 151696 pages reserved Sep 16 12:37:09 hydra01 kernel: [2091736.847174] 8350615 pages shared Sep 16 12:37:09 hydra01 kernel: [2091736.847175] 3791070 pages non-shared Sep 16 12:37:21 hydra01 kernel: [2091748.187392] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:37:21 hydra01 kernel: [2091748.187401] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:37:21 hydra01 kernel: [2091748.187404] Call Trace: Sep 16 12:37:21 hydra01 kernel: [2091748.187417] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.187425] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:37:21 hydra01 kernel: [2091748.187431] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:37:21 hydra01 kernel: [2091748.187439] [] alloc_pages_current+0xb8/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.187444] [] __get_free_pages+0xe/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.187451] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:37:21 hydra01 kernel: [2091748.187457] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.187463] [] __kmalloc+0x161/0x170 Sep 16 12:37:21 hydra01 kernel: [2091748.187471] [] ? setkey+0x80/0x100 Sep 16 12:37:21 hydra01 kernel: [2091748.187477] [] ? get_iowait_time+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.187482] [] stat_open+0x56/0xc0 Sep 16 12:37:21 hydra01 kernel: [2091748.187488] [] proc_reg_open+0xa0/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.187495] [] ? seq_release_private+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.187501] [] do_dentry_open+0x203/0x290 Sep 16 12:37:21 hydra01 kernel: [2091748.187506] [] ? init_once+0x20/0x20 Sep 16 12:37:21 hydra01 kernel: [2091748.187510] [] vfs_open+0x49/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.187515] [] do_last+0x296/0xea0 Sep 16 12:37:21 hydra01 kernel: [2091748.187520] [] ? inode_permission+0x18/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.187524] [] ? link_path_walk+0x6b/0x880 Sep 16 12:37:21 hydra01 kernel: [2091748.187529] [] path_openat+0xb3/0x4b0 Sep 16 12:37:21 hydra01 kernel: [2091748.187536] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.187542] [] ? unmap_region+0xdb/0x120 Sep 16 12:37:21 hydra01 kernel: [2091748.187547] [] do_filp_open+0x41/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.187552] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:37:21 hydra01 kernel: [2091748.187557] [] do_sys_open+0xf3/0x230 Sep 16 12:37:21 hydra01 kernel: [2091748.187562] [] sys_open+0x21/0x30 Sep 16 12:37:21 hydra01 kernel: [2091748.187567] [] system_call_fastpath+0x1a/0x1f Sep 16 12:37:21 hydra01 kernel: [2091748.187569] Mem-Info: Sep 16 12:37:21 hydra01 kernel: [2091748.187572] Node 0 DMA per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.187576] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187579] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187582] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187585] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187587] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187590] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187592] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187595] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187597] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187600] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187603] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187605] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187607] Node 0 DMA32 per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.187612] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187614] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187617] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187620] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187622] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187625] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187627] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187630] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187633] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187635] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187638] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187641] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187643] Node 0 Normal per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.187646] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187649] CPU 1: hi: 186, btch: 31 usd: 174 Sep 16 12:37:21 hydra01 kernel: [2091748.187652] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187654] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187657] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187659] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187662] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187665] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187667] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187670] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187673] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187675] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187683] active_anon:1283119 inactive_anon:824435 isolated_anon:0 Sep 16 12:37:21 hydra01 kernel: [2091748.187683] active_file:1869136 inactive_file:3483846 isolated_file:0 Sep 16 12:37:21 hydra01 kernel: [2091748.187683] unevictable:0 dirty:67 writeback:0 unstable:0 Sep 16 12:37:21 hydra01 kernel: [2091748.187683] free:63764 slab_reclaimable:375415 slab_unreclaimable:102671 Sep 16 12:37:21 hydra01 kernel: [2091748.187683] mapped:25796 shmem:153 pagetables:9607 bounce:0 Sep 16 12:37:21 hydra01 kernel: [2091748.187683] free_cma:0 Sep 16 12:37:21 hydra01 kernel: [2091748.187689] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:37:21 hydra01 kernel: [2091748.187698] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:37:21 hydra01 kernel: [2091748.187704] Node 0 DMA32 free:136964kB min:6224kB low:7780kB high:9336kB active_anon:430256kB inactive_anon:793020kB active_file:95304kB inactive_file:1387508kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:168152kB slab_unreclaimable:43988kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.187713] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:37:21 hydra01 kernel: [2091748.187719] Node 0 Normal free:102248kB min:61324kB low:76652kB high:91984kB active_anon:4702220kB inactive_anon:2504720kB active_file:7381240kB inactive_file:12547876kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:268kB writeback:0kB mapped:100412kB shmem:612kB slab_reclaimable:1333508kB slab_unreclaimable:366696kB kernel_stack:8032kB pagetables:36900kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.187728] lowmem_reserve[]: 0 0 0 0 Sep 16 12:37:21 hydra01 kernel: [2091748.187733] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:37:21 hydra01 kernel: [2091748.187755] Node 0 DMA32: 3761*4kB (UEM) 1845*8kB (UEM) 5834*16kB (UEM) 428*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137420kB Sep 16 12:37:21 hydra01 kernel: [2091748.187774] Node 0 Normal: 14614*4kB (UEMR) 5133*8kB (UEMR) 90*16kB (UEMR) 45*32kB (UEMR) 7*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 104128kB Sep 16 12:37:21 hydra01 kernel: [2091748.187794] 5784662 total pagecache pages Sep 16 12:37:21 hydra01 kernel: [2091748.187796] 431664 pages in swap cache Sep 16 12:37:21 hydra01 kernel: [2091748.187799] Swap cache stats: add 20774541, delete 20342877, find 12661890/14316286 Sep 16 12:37:21 hydra01 kernel: [2091748.187801] Free swap = 185326772kB Sep 16 12:37:21 hydra01 kernel: [2091748.187803] Total swap = 187617276kB Sep 16 12:37:21 hydra01 kernel: [2091748.380737] 8388592 pages RAM Sep 16 12:37:21 hydra01 kernel: [2091748.380743] 151696 pages reserved Sep 16 12:37:21 hydra01 kernel: [2091748.380748] 8359877 pages shared Sep 16 12:37:21 hydra01 kernel: [2091748.380752] 3782940 pages non-shared Sep 16 12:37:21 hydra01 kernel: [2091748.382497] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:37:21 hydra01 kernel: [2091748.382501] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:37:21 hydra01 kernel: [2091748.382502] Call Trace: Sep 16 12:37:21 hydra01 kernel: [2091748.382510] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.382514] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:37:21 hydra01 kernel: [2091748.382518] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:37:21 hydra01 kernel: [2091748.382522] [] alloc_pages_current+0xb8/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.382525] [] __get_free_pages+0xe/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.382529] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:37:21 hydra01 kernel: [2091748.382533] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.382536] [] __kmalloc+0x161/0x170 Sep 16 12:37:21 hydra01 kernel: [2091748.382541] [] ? setkey+0x81/0x100 Sep 16 12:37:21 hydra01 kernel: [2091748.382544] [] ? get_iowait_time+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.382547] [] stat_open+0x56/0xc0 Sep 16 12:37:21 hydra01 kernel: [2091748.382550] [] proc_reg_open+0xa0/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.382554] [] ? seq_release_private+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.382557] [] do_dentry_open+0x203/0x290 Sep 16 12:37:21 hydra01 kernel: [2091748.382560] [] ? init_once+0x20/0x20 Sep 16 12:37:21 hydra01 kernel: [2091748.382562] [] vfs_open+0x49/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.382565] [] do_last+0x296/0xea0 Sep 16 12:37:21 hydra01 kernel: [2091748.382568] [] ? inode_permission+0x18/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.382570] [] ? link_path_walk+0x6b/0x880 Sep 16 12:37:21 hydra01 kernel: [2091748.382573] [] path_openat+0xb3/0x4b0 Sep 16 12:37:21 hydra01 kernel: [2091748.382576] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.382580] [] ? unmap_region+0xdb/0x120 Sep 16 12:37:21 hydra01 kernel: [2091748.382582] [] do_filp_open+0x41/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.382585] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:37:21 hydra01 kernel: [2091748.382588] [] do_sys_open+0xf3/0x230 Sep 16 12:37:21 hydra01 kernel: [2091748.382590] [] sys_open+0x21/0x30 Sep 16 12:37:21 hydra01 kernel: [2091748.382593] [] system_call_fastpath+0x1a/0x1f Sep 16 12:37:21 hydra01 kernel: [2091748.382595] Mem-Info: Sep 16 12:37:21 hydra01 kernel: [2091748.382596] Node 0 DMA per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.382599] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382600] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382601] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382603] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382604] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382605] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382607] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382608] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382609] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382611] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382612] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382613] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382614] Node 0 DMA32 per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.382617] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382618] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382619] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382621] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382622] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382624] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382625] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382626] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382628] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382629] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382630] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382632] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382633] Node 0 Normal per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.382635] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382636] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382637] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382639] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382640] CPU 4: hi: 186, btch: 31 usd: 186 Sep 16 12:37:21 hydra01 kernel: [2091748.382642] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382643] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382645] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382646] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382647] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382649] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382650] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382654] active_anon:1283114 inactive_anon:824435 isolated_anon:0 Sep 16 12:37:21 hydra01 kernel: [2091748.382654] active_file:1869136 inactive_file:3466161 isolated_file:0 Sep 16 12:37:21 hydra01 kernel: [2091748.382654] unevictable:0 dirty:36 writeback:0 unstable:0 Sep 16 12:37:21 hydra01 kernel: [2091748.382654] free:82162 slab_reclaimable:375415 slab_unreclaimable:102663 Sep 16 12:37:21 hydra01 kernel: [2091748.382654] mapped:25806 shmem:153 pagetables:9600 bounce:0 Sep 16 12:37:21 hydra01 kernel: [2091748.382654] free_cma:0 Sep 16 12:37:21 hydra01 kernel: [2091748.382657] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:37:21 hydra01 kernel: [2091748.382662] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:37:21 hydra01 kernel: [2091748.382665] Node 0 DMA32 free:137124kB min:6224kB low:7780kB high:9336kB active_anon:430256kB inactive_anon:793020kB active_file:95304kB inactive_file:1387508kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:168152kB slab_unreclaimable:43956kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.382670] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:37:21 hydra01 kernel: [2091748.382672] Node 0 Normal free:175972kB min:61324kB low:76652kB high:91984kB active_anon:4702200kB inactive_anon:2504720kB active_file:7381240kB inactive_file:12477136kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:144kB writeback:0kB mapped:100452kB shmem:612kB slab_reclaimable:1333508kB slab_unreclaimable:366696kB kernel_stack:8024kB pagetables:36872kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.382677] lowmem_reserve[]: 0 0 0 0 Sep 16 12:37:21 hydra01 kernel: [2091748.382680] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:37:21 hydra01 kernel: [2091748.382691] Node 0 DMA32: 3761*4kB (UEM) 1845*8kB (UEM) 5835*16kB (UEM) 429*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137468kB Sep 16 12:37:21 hydra01 kernel: [2091748.382701] Node 0 Normal: 15348*4kB (UEMR) 13952*8kB (UEMR) 92*16kB (UEMR) 45*32kB (UEMR) 7*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 177648kB Sep 16 12:37:21 hydra01 kernel: [2091748.382727] 5767024 total pagecache pages Sep 16 12:37:21 hydra01 kernel: [2091748.382733] 431664 pages in swap cache Sep 16 12:37:21 hydra01 kernel: [2091748.382739] Swap cache stats: add 20774541, delete 20342877, find 12661890/14316286 Sep 16 12:37:21 hydra01 kernel: [2091748.382744] Free swap = 185326772kB Sep 16 12:37:21 hydra01 kernel: [2091748.382749] Total swap = 187617276kB Sep 16 12:37:21 hydra01 kernel: [2091748.489524] 8388592 pages RAM Sep 16 12:37:21 hydra01 kernel: [2091748.489527] 151696 pages reserved Sep 16 12:37:21 hydra01 kernel: [2091748.489528] 8363215 pages shared Sep 16 12:37:21 hydra01 kernel: [2091748.489530] 3782007 pages non-shared Sep 16 12:37:21 hydra01 kernel: [2091748.491583] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:37:21 hydra01 kernel: [2091748.491590] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:37:21 hydra01 kernel: [2091748.491592] Call Trace: Sep 16 12:37:21 hydra01 kernel: [2091748.491605] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.491612] [] ? drain_local_pages+0x16/0x20 Sep 16 12:37:21 hydra01 kernel: [2091748.491620] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:37:21 hydra01 kernel: [2091748.491626] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:37:21 hydra01 kernel: [2091748.491634] [] alloc_pages_current+0xb8/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.491639] [] __get_free_pages+0xe/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.491646] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:37:21 hydra01 kernel: [2091748.491652] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.491657] [] __kmalloc+0x161/0x170 Sep 16 12:37:21 hydra01 kernel: [2091748.491665] [] ? setkey+0x80/0x100 Sep 16 12:37:21 hydra01 kernel: [2091748.491671] [] ? get_iowait_time+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.491676] [] stat_open+0x56/0xc0 Sep 16 12:37:21 hydra01 kernel: [2091748.491682] [] proc_reg_open+0xa0/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.491689] [] ? seq_release_private+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.491695] [] do_dentry_open+0x203/0x290 Sep 16 12:37:21 hydra01 kernel: [2091748.491700] [] ? init_once+0x20/0x20 Sep 16 12:37:21 hydra01 kernel: [2091748.491704] [] vfs_open+0x49/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.491709] [] do_last+0x296/0xea0 Sep 16 12:37:21 hydra01 kernel: [2091748.491714] [] ? inode_permission+0x18/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.491718] [] ? link_path_walk+0x6b/0x880 Sep 16 12:37:21 hydra01 kernel: [2091748.491723] [] path_openat+0xb3/0x4b0 Sep 16 12:37:21 hydra01 kernel: [2091748.491730] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.491736] [] ? unmap_region+0xdb/0x120 Sep 16 12:37:21 hydra01 kernel: [2091748.491741] [] do_filp_open+0x41/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.491746] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:37:21 hydra01 kernel: [2091748.491751] [] do_sys_open+0xf3/0x230 Sep 16 12:37:21 hydra01 kernel: [2091748.491755] [] sys_open+0x21/0x30 Sep 16 12:37:21 hydra01 kernel: [2091748.491761] [] system_call_fastpath+0x1a/0x1f Sep 16 12:37:21 hydra01 kernel: [2091748.491764] Mem-Info: Sep 16 12:37:21 hydra01 kernel: [2091748.491767] Node 0 DMA per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.491771] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491774] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491777] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491779] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491782] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491785] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491787] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491790] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491792] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491795] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491797] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491800] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491802] Node 0 DMA32 per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.491806] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491808] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491811] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491814] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491816] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491819] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491821] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491824] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491827] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491829] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491832] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491835] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491837] Node 0 Normal per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.491840] CPU 0: hi: 186, btch: 31 usd: 186 Sep 16 12:37:21 hydra01 kernel: [2091748.491843] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491846] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491848] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491851] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491854] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491856] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491859] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491861] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491864] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491867] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491869] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491877] active_anon:1283114 inactive_anon:824435 isolated_anon:0 Sep 16 12:37:21 hydra01 kernel: [2091748.491877] active_file:1869136 inactive_file:3468351 isolated_file:0 Sep 16 12:37:21 hydra01 kernel: [2091748.491877] unevictable:0 dirty:254 writeback:0 unstable:0 Sep 16 12:37:21 hydra01 kernel: [2091748.491877] free:80166 slab_reclaimable:375415 slab_unreclaimable:102663 Sep 16 12:37:21 hydra01 kernel: [2091748.491877] mapped:25806 shmem:153 pagetables:9600 bounce:0 Sep 16 12:37:21 hydra01 kernel: [2091748.491877] free_cma:0 Sep 16 12:37:21 hydra01 kernel: [2091748.491883] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:37:21 hydra01 kernel: [2091748.491892] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:37:21 hydra01 kernel: [2091748.491898] Node 0 DMA32 free:137124kB min:6224kB low:7780kB high:9336kB active_anon:430256kB inactive_anon:793020kB active_file:95304kB inactive_file:1387508kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:168152kB slab_unreclaimable:43956kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.491907] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:37:21 hydra01 kernel: [2091748.491913] Node 0 Normal free:167988kB min:61324kB low:76652kB high:91984kB active_anon:4702200kB inactive_anon:2504720kB active_file:7381240kB inactive_file:12485512kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:1016kB writeback:0kB mapped:100452kB shmem:612kB slab_reclaimable:1333508kB slab_unreclaimable:366696kB kernel_stack:8024kB pagetables:36872kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.491922] lowmem_reserve[]: 0 0 0 0 Sep 16 12:37:21 hydra01 kernel: [2091748.491927] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:37:21 hydra01 kernel: [2091748.491948] Node 0 DMA32: 3761*4kB (UEM) 1845*8kB (UEM) 5835*16kB (UEM) 428*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137436kB Sep 16 12:37:21 hydra01 kernel: [2091748.491968] Node 0 Normal: 16441*4kB (UEMR) 12328*8kB (UEMR) 92*16kB (UEMR) 45*32kB (UEMR) 7*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 169028kB Sep 16 12:37:21 hydra01 kernel: [2091748.491988] 5769204 total pagecache pages Sep 16 12:37:21 hydra01 kernel: [2091748.491990] 431664 pages in swap cache Sep 16 12:37:21 hydra01 kernel: [2091748.491993] Swap cache stats: add 20774541, delete 20342877, find 12661890/14316286 Sep 16 12:37:21 hydra01 kernel: [2091748.491996] Free swap = 185326772kB Sep 16 12:37:21 hydra01 kernel: [2091748.491998] Total swap = 187617276kB Sep 16 12:37:21 hydra01 kernel: [2091748.677117] 8388592 pages RAM Sep 16 12:37:21 hydra01 kernel: [2091748.677130] 151696 pages reserved Sep 16 12:37:21 hydra01 kernel: [2091748.677139] 8362750 pages shared Sep 16 12:37:21 hydra01 kernel: [2091748.677150] 3780876 pages non-shared Sep 16 12:37:21 hydra01 kernel: [2091748.678595] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:37:21 hydra01 kernel: [2091748.678602] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:37:21 hydra01 kernel: [2091748.678604] Call Trace: Sep 16 12:37:21 hydra01 kernel: [2091748.678617] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.678625] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:37:21 hydra01 kernel: [2091748.678631] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:37:21 hydra01 kernel: [2091748.678639] [] alloc_pages_current+0xb8/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.678644] [] __get_free_pages+0xe/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.678651] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:37:21 hydra01 kernel: [2091748.678657] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.678663] [] __kmalloc+0x161/0x170 Sep 16 12:37:21 hydra01 kernel: [2091748.678670] [] ? amiga_partition+0x2c4/0x4d0 Sep 16 12:37:21 hydra01 kernel: [2091748.678676] [] ? get_iowait_time+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.678681] [] stat_open+0x56/0xc0 Sep 16 12:37:21 hydra01 kernel: [2091748.678687] [] proc_reg_open+0xa0/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.678694] [] ? seq_release_private+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.678700] [] do_dentry_open+0x203/0x290 Sep 16 12:37:21 hydra01 kernel: [2091748.678705] [] ? init_once+0x20/0x20 Sep 16 12:37:21 hydra01 kernel: [2091748.678709] [] vfs_open+0x49/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.678714] [] do_last+0x296/0xea0 Sep 16 12:37:21 hydra01 kernel: [2091748.678719] [] ? inode_permission+0x18/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.678723] [] ? link_path_walk+0x6b/0x880 Sep 16 12:37:21 hydra01 kernel: [2091748.678728] [] path_openat+0xb3/0x4b0 Sep 16 12:37:21 hydra01 kernel: [2091748.678735] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.678741] [] ? unmap_region+0xdb/0x120 Sep 16 12:37:21 hydra01 kernel: [2091748.678746] [] do_filp_open+0x41/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.678751] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:37:21 hydra01 kernel: [2091748.678756] [] do_sys_open+0xf3/0x230 Sep 16 12:37:21 hydra01 kernel: [2091748.678760] [] sys_open+0x21/0x30 Sep 16 12:37:21 hydra01 kernel: [2091748.678766] [] system_call_fastpath+0x1a/0x1f Sep 16 12:37:21 hydra01 kernel: [2091748.678769] Mem-Info: Sep 16 12:37:21 hydra01 kernel: [2091748.678771] Node 0 DMA per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.678776] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678779] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678781] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678784] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678787] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678789] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678792] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678794] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678797] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678799] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678802] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678805] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678807] Node 0 DMA32 per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.678811] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678813] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678816] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678819] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678821] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678824] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678827] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678829] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678832] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678835] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678837] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678840] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678842] Node 0 Normal per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.678845] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678848] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678851] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678853] CPU 3: hi: 186, btch: 31 usd: 30 Sep 16 12:37:21 hydra01 kernel: [2091748.678856] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678859] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678861] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678864] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678866] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678869] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678872] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678874] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678881] active_anon:1283114 inactive_anon:824435 isolated_anon:0 Sep 16 12:37:21 hydra01 kernel: [2091748.678881] active_file:1869136 inactive_file:3467787 isolated_file:0 Sep 16 12:37:21 hydra01 kernel: [2091748.678881] unevictable:0 dirty:254 writeback:0 unstable:0 Sep 16 12:37:21 hydra01 kernel: [2091748.678881] free:80823 slab_reclaimable:375415 slab_unreclaimable:102663 Sep 16 12:37:21 hydra01 kernel: [2091748.678881] mapped:25806 shmem:153 pagetables:9600 bounce:0 Sep 16 12:37:21 hydra01 kernel: [2091748.678881] free_cma:0 Sep 16 12:37:21 hydra01 kernel: [2091748.678888] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:37:21 hydra01 kernel: [2091748.678897] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:37:21 hydra01 kernel: [2091748.678903] Node 0 DMA32 free:137124kB min:6224kB low:7780kB high:9336kB active_anon:430256kB inactive_anon:793020kB active_file:95304kB inactive_file:1387508kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:168152kB slab_unreclaimable:43956kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.678912] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:37:21 hydra01 kernel: [2091748.678917] Node 0 Normal free:170324kB min:61324kB low:76652kB high:91984kB active_anon:4702200kB inactive_anon:2504720kB active_file:7381240kB inactive_file:12483640kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:1016kB writeback:0kB mapped:100452kB shmem:612kB slab_reclaimable:1333508kB slab_unreclaimable:366696kB kernel_stack:8024kB pagetables:36872kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.678926] lowmem_reserve[]: 0 0 0 0 Sep 16 12:37:21 hydra01 kernel: [2091748.678931] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:37:21 hydra01 kernel: [2091748.678953] Node 0 DMA32: 3761*4kB (UEM) 1845*8kB (UEM) 5835*16kB (UEM) 428*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137436kB Sep 16 12:37:21 hydra01 kernel: [2091748.678972] Node 0 Normal: 17006*4kB (UEMR) 12331*8kB (UEMR) 92*16kB (UEMR) 45*32kB (UEMR) 7*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 171312kB Sep 16 12:37:21 hydra01 kernel: [2091748.678992] 5768768 total pagecache pages Sep 16 12:37:21 hydra01 kernel: [2091748.678994] 431664 pages in swap cache Sep 16 12:37:21 hydra01 kernel: [2091748.678997] Swap cache stats: add 20774541, delete 20342877, find 12661890/14316286 Sep 16 12:37:21 hydra01 kernel: [2091748.679000] Free swap = 185326772kB Sep 16 12:37:21 hydra01 kernel: [2091748.679002] Total swap = 187617276kB Sep 16 12:37:21 hydra01 kernel: [2091748.871861] 8388592 pages RAM Sep 16 12:37:21 hydra01 kernel: [2091748.871872] 151696 pages reserved Sep 16 12:37:21 hydra01 kernel: [2091748.871874] 8362512 pages shared Sep 16 12:37:21 hydra01 kernel: [2091748.871877] 3780865 pages non-shared Sep 16 12:37:21 hydra01 kernel: [2091748.971645] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:37:21 hydra01 kernel: [2091748.971650] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:37:21 hydra01 kernel: [2091748.971652] Call Trace: Sep 16 12:37:21 hydra01 kernel: [2091748.971660] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.971665] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:37:21 hydra01 kernel: [2091748.971668] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:37:21 hydra01 kernel: [2091748.971672] [] alloc_pages_current+0xb8/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.971675] [] __get_free_pages+0xe/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.971680] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:37:21 hydra01 kernel: [2091748.971683] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:37:21 hydra01 kernel: [2091748.971686] [] __kmalloc+0x161/0x170 Sep 16 12:37:21 hydra01 kernel: [2091748.971690] [] ? amiga_partition+0x2c4/0x4d0 Sep 16 12:37:21 hydra01 kernel: [2091748.971694] [] ? get_iowait_time+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.971697] [] stat_open+0x56/0xc0 Sep 16 12:37:21 hydra01 kernel: [2091748.971700] [] proc_reg_open+0xa0/0x180 Sep 16 12:37:21 hydra01 kernel: [2091748.971704] [] ? seq_release_private+0x60/0x60 Sep 16 12:37:21 hydra01 kernel: [2091748.971707] [] do_dentry_open+0x203/0x290 Sep 16 12:37:21 hydra01 kernel: [2091748.971710] [] ? init_once+0x20/0x20 Sep 16 12:37:21 hydra01 kernel: [2091748.971712] [] vfs_open+0x49/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.971715] [] do_last+0x296/0xea0 Sep 16 12:37:21 hydra01 kernel: [2091748.971718] [] ? inode_permission+0x18/0x50 Sep 16 12:37:21 hydra01 kernel: [2091748.971720] [] ? link_path_walk+0x6b/0x880 Sep 16 12:37:21 hydra01 kernel: [2091748.971723] [] path_openat+0xb3/0x4b0 Sep 16 12:37:21 hydra01 kernel: [2091748.971726] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.971730] [] ? unmap_region+0xdb/0x120 Sep 16 12:37:21 hydra01 kernel: [2091748.971732] [] do_filp_open+0x41/0xa0 Sep 16 12:37:21 hydra01 kernel: [2091748.971735] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:37:21 hydra01 kernel: [2091748.971738] [] do_sys_open+0xf3/0x230 Sep 16 12:37:21 hydra01 kernel: [2091748.971740] [] sys_open+0x21/0x30 Sep 16 12:37:21 hydra01 kernel: [2091748.971743] [] system_call_fastpath+0x1a/0x1f Sep 16 12:37:21 hydra01 kernel: [2091748.971745] Mem-Info: Sep 16 12:37:21 hydra01 kernel: [2091748.971746] Node 0 DMA per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.971749] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971750] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971752] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971753] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971755] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971756] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971758] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971759] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971760] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971762] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971763] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971765] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971766] Node 0 DMA32 per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.971768] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971769] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971771] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971772] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971773] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971775] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971776] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971778] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971779] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971780] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971782] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971783] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971784] Node 0 Normal per-cpu: Sep 16 12:37:21 hydra01 kernel: [2091748.971786] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971787] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971789] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971790] CPU 3: hi: 186, btch: 31 usd: 30 Sep 16 12:37:21 hydra01 kernel: [2091748.971791] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971793] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971794] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971796] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971797] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971798] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971800] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971801] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971805] active_anon:1283114 inactive_anon:824435 isolated_anon:0 Sep 16 12:37:21 hydra01 kernel: [2091748.971805] active_file:1869136 inactive_file:3467787 isolated_file:0 Sep 16 12:37:21 hydra01 kernel: [2091748.971805] unevictable:0 dirty:254 writeback:0 unstable:0 Sep 16 12:37:21 hydra01 kernel: [2091748.971805] free:80822 slab_reclaimable:375415 slab_unreclaimable:102663 Sep 16 12:37:21 hydra01 kernel: [2091748.971805] mapped:25806 shmem:153 pagetables:9600 bounce:0 Sep 16 12:37:21 hydra01 kernel: [2091748.971805] free_cma:0 Sep 16 12:37:21 hydra01 kernel: [2091748.971808] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:37:21 hydra01 kernel: [2091748.971813] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:37:21 hydra01 kernel: [2091748.971816] Node 0 DMA32 free:137124kB min:6224kB low:7780kB high:9336kB active_anon:430256kB inactive_anon:793020kB active_file:95304kB inactive_file:1387508kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:168152kB slab_unreclaimable:43956kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.971821] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:37:21 hydra01 kernel: [2091748.971823] Node 0 Normal free:170320kB min:61324kB low:76652kB high:91984kB active_anon:4702200kB inactive_anon:2504720kB active_file:7381240kB inactive_file:12483640kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:1016kB writeback:0kB mapped:100452kB shmem:612kB slab_reclaimable:1333508kB slab_unreclaimable:366696kB kernel_stack:8024kB pagetables:36872kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:37:21 hydra01 kernel: [2091748.971828] lowmem_reserve[]: 0 0 0 0 Sep 16 12:37:21 hydra01 kernel: [2091748.971831] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:37:21 hydra01 kernel: [2091748.971842] Node 0 DMA32: 3761*4kB (UEM) 1845*8kB (UEM) 5835*16kB (UEM) 428*32kB (UEMR) 1*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137436kB Sep 16 12:37:21 hydra01 kernel: [2091748.971852] Node 0 Normal: 15563*4kB (UEMR) 4992*8kB (UEMR) 3107*16kB (UEMR) 484*32kB (UEMR) 35*64kB (UEMR) 5*128kB (MR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 171292kB Sep 16 12:37:21 hydra01 kernel: [2091748.971862] 5768768 total pagecache pages Sep 16 12:37:21 hydra01 kernel: [2091748.971864] 431664 pages in swap cache Sep 16 12:37:21 hydra01 kernel: [2091748.971865] Swap cache stats: add 20774541, delete 20342877, find 12661890/14316286 Sep 16 12:37:21 hydra01 kernel: [2091748.971867] Free swap = 185326772kB Sep 16 12:37:21 hydra01 kernel: [2091748.971868] Total swap = 187617276kB Sep 16 12:37:21 hydra01 kernel: [2091749.078584] 8388592 pages RAM Sep 16 12:37:21 hydra01 kernel: [2091749.078592] 151696 pages reserved Sep 16 12:37:21 hydra01 kernel: [2091749.078596] 8362891 pages shared Sep 16 12:37:21 hydra01 kernel: [2091749.078601] 3780323 pages non-shared Sep 16 12:38:39 hydra01 kernel: [2091826.633964] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:38:39 hydra01 kernel: [2091826.633973] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:38:39 hydra01 kernel: [2091826.633976] Call Trace: Sep 16 12:38:39 hydra01 kernel: [2091826.633990] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:38:39 hydra01 kernel: [2091826.633997] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:38:39 hydra01 kernel: [2091826.634004] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:38:39 hydra01 kernel: [2091826.634011] [] alloc_pages_current+0xb8/0x180 Sep 16 12:38:39 hydra01 kernel: [2091826.634017] [] __get_free_pages+0xe/0x50 Sep 16 12:38:39 hydra01 kernel: [2091826.634024] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:38:39 hydra01 kernel: [2091826.634030] [] __kmalloc+0x161/0x170 Sep 16 12:38:39 hydra01 kernel: [2091826.634037] [] ? get_iowait_time+0x60/0x60 Sep 16 12:38:39 hydra01 kernel: [2091826.634042] [] stat_open+0x56/0xc0 Sep 16 12:38:39 hydra01 kernel: [2091826.634048] [] proc_reg_open+0xa0/0x180 Sep 16 12:38:39 hydra01 kernel: [2091826.634055] [] ? seq_release_private+0x60/0x60 Sep 16 12:38:39 hydra01 kernel: [2091826.634060] [] do_dentry_open+0x203/0x290 Sep 16 12:38:39 hydra01 kernel: [2091826.634065] [] ? init_once+0x20/0x20 Sep 16 12:38:39 hydra01 kernel: [2091826.634069] [] vfs_open+0x49/0x50 Sep 16 12:38:39 hydra01 kernel: [2091826.634074] [] do_last+0x296/0xea0 Sep 16 12:38:39 hydra01 kernel: [2091826.634079] [] ? inode_permission+0x18/0x50 Sep 16 12:38:39 hydra01 kernel: [2091826.634083] [] ? link_path_walk+0x6b/0x880 Sep 16 12:38:39 hydra01 kernel: [2091826.634088] [] path_openat+0xb3/0x4b0 Sep 16 12:38:39 hydra01 kernel: [2091826.634094] [] do_filp_open+0x41/0xa0 Sep 16 12:38:39 hydra01 kernel: [2091826.634111] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:38:39 hydra01 kernel: [2091826.634117] [] do_sys_open+0xf3/0x230 Sep 16 12:38:39 hydra01 kernel: [2091826.634120] [] ? sys_select+0xcb/0x110 Sep 16 12:38:39 hydra01 kernel: [2091826.634122] [] sys_open+0x21/0x30 Sep 16 12:38:39 hydra01 kernel: [2091826.634125] [] system_call_fastpath+0x1a/0x1f Sep 16 12:38:39 hydra01 kernel: [2091826.634127] Mem-Info: Sep 16 12:38:39 hydra01 kernel: [2091826.634128] Node 0 DMA per-cpu: Sep 16 12:38:39 hydra01 kernel: [2091826.634131] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634132] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634134] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634135] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634136] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634138] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634139] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634140] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634142] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634143] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634145] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634147] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634148] Node 0 DMA32 per-cpu: Sep 16 12:38:39 hydra01 kernel: [2091826.634150] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634152] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634153] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634154] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634156] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634157] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634158] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634172] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634175] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634177] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634180] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634183] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634185] Node 0 Normal per-cpu: Sep 16 12:38:39 hydra01 kernel: [2091826.634188] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634191] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634194] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634196] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634199] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634202] CPU 5: hi: 186, btch: 31 usd: 175 Sep 16 12:38:39 hydra01 kernel: [2091826.634204] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634207] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634210] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634212] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634215] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634218] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634225] active_anon:1284673 inactive_anon:823349 isolated_anon:0 Sep 16 12:38:39 hydra01 kernel: [2091826.634225] active_file:1869216 inactive_file:3491044 isolated_file:0 Sep 16 12:38:39 hydra01 kernel: [2091826.634225] unevictable:0 dirty:16 writeback:0 unstable:0 Sep 16 12:38:39 hydra01 kernel: [2091826.634225] free:67882 slab_reclaimable:370936 slab_unreclaimable:102248 Sep 16 12:38:39 hydra01 kernel: [2091826.634225] mapped:25733 shmem:151 pagetables:9641 bounce:0 Sep 16 12:38:39 hydra01 kernel: [2091826.634225] free_cma:0 Sep 16 12:38:39 hydra01 kernel: [2091826.634232] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:38:39 hydra01 kernel: [2091826.634240] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:38:39 hydra01 kernel: [2091826.634246] Node 0 DMA32 free:139472kB min:6224kB low:7780kB high:9336kB active_anon:430660kB inactive_anon:793008kB active_file:95328kB inactive_file:1387036kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165908kB slab_unreclaimable:43556kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:39 hydra01 kernel: [2091826.634256] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:38:39 hydra01 kernel: [2091826.634261] Node 0 Normal free:116212kB min:61324kB low:76652kB high:91984kB active_anon:4708032kB inactive_anon:2500388kB active_file:7381536kB inactive_file:12577140kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:64kB writeback:0kB mapped:100160kB shmem:604kB slab_reclaimable:1317836kB slab_unreclaimable:365436kB kernel_stack:8016kB pagetables:37036kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:38:39 hydra01 kernel: [2091826.634271] lowmem_reserve[]: 0 0 0 0 Sep 16 12:38:39 hydra01 kernel: [2091826.634276] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:38:39 hydra01 kernel: [2091826.634307] Node 0 DMA32: 5064*4kB (UEM) 1573*8kB (UEM) 5793*16kB (UEM) 433*32kB (UEMR) 2*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140024kB Sep 16 12:38:39 hydra01 kernel: [2091826.634317] Node 0 Normal: 20154*4kB (UEMR) 2836*8kB (UEMR) 826*16kB (UMR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 118376kB Sep 16 12:38:39 hydra01 kernel: [2091826.634330] 5791986 total pagecache pages Sep 16 12:38:39 hydra01 kernel: [2091826.634331] 431650 pages in swap cache Sep 16 12:38:39 hydra01 kernel: [2091826.634333] Swap cache stats: add 20774725, delete 20343075, find 12661890/14316286 Sep 16 12:38:39 hydra01 kernel: [2091826.634334] Free swap = 185326108kB Sep 16 12:38:39 hydra01 kernel: [2091826.634335] Total swap = 187617276kB Sep 16 12:38:39 hydra01 kernel: [2091826.830891] 8388592 pages RAM Sep 16 12:38:39 hydra01 kernel: [2091826.830894] 151696 pages reserved Sep 16 12:38:39 hydra01 kernel: [2091826.830896] 8669869 pages shared Sep 16 12:38:39 hydra01 kernel: [2091826.830897] 3734070 pages non-shared Sep 16 12:38:51 hydra01 kernel: [2091839.059708] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:38:51 hydra01 kernel: [2091839.059719] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:38:51 hydra01 kernel: [2091839.059724] Call Trace: Sep 16 12:38:51 hydra01 kernel: [2091839.059741] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:38:51 hydra01 kernel: [2091839.059752] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:38:51 hydra01 kernel: [2091839.059761] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:38:51 hydra01 kernel: [2091839.059772] [] alloc_pages_current+0xb8/0x180 Sep 16 12:38:51 hydra01 kernel: [2091839.059780] [] __get_free_pages+0xe/0x50 Sep 16 12:38:51 hydra01 kernel: [2091839.059792] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:38:51 hydra01 kernel: [2091839.059800] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:38:51 hydra01 kernel: [2091839.059809] [] __kmalloc+0x161/0x170 Sep 16 12:38:51 hydra01 kernel: [2091839.059821] [] ? setkey+0x80/0x100 Sep 16 12:38:51 hydra01 kernel: [2091839.059830] [] ? get_iowait_time+0x60/0x60 Sep 16 12:38:51 hydra01 kernel: [2091839.059838] [] stat_open+0x56/0xc0 Sep 16 12:38:51 hydra01 kernel: [2091839.059847] [] proc_reg_open+0xa0/0x180 Sep 16 12:38:51 hydra01 kernel: [2091839.059857] [] ? seq_release_private+0x60/0x60 Sep 16 12:38:51 hydra01 kernel: [2091839.059866] [] do_dentry_open+0x203/0x290 Sep 16 12:38:51 hydra01 kernel: [2091839.059873] [] ? init_once+0x20/0x20 Sep 16 12:38:51 hydra01 kernel: [2091839.059880] [] vfs_open+0x49/0x50 Sep 16 12:38:51 hydra01 kernel: [2091839.059888] [] do_last+0x296/0xea0 Sep 16 12:38:51 hydra01 kernel: [2091839.059895] [] ? inode_permission+0x18/0x50 Sep 16 12:38:51 hydra01 kernel: [2091839.059902] [] ? link_path_walk+0x6b/0x880 Sep 16 12:38:51 hydra01 kernel: [2091839.059909] [] path_openat+0xb3/0x4b0 Sep 16 12:38:51 hydra01 kernel: [2091839.059919] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:38:51 hydra01 kernel: [2091839.059928] [] ? unmap_region+0xdb/0x120 Sep 16 12:38:51 hydra01 kernel: [2091839.059936] [] do_filp_open+0x41/0xa0 Sep 16 12:38:51 hydra01 kernel: [2091839.059945] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:38:51 hydra01 kernel: [2091839.059952] [] do_sys_open+0xf3/0x230 Sep 16 12:38:51 hydra01 kernel: [2091839.059960] [] sys_open+0x21/0x30 Sep 16 12:38:51 hydra01 kernel: [2091839.059967] [] system_call_fastpath+0x1a/0x1f Sep 16 12:38:51 hydra01 kernel: [2091839.059972] Mem-Info: Sep 16 12:38:51 hydra01 kernel: [2091839.059976] Node 0 DMA per-cpu: Sep 16 12:38:51 hydra01 kernel: [2091839.059984] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.059989] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.059994] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.059998] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060003] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060007] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060012] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060016] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060021] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060026] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060030] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060035] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060038] Node 0 DMA32 per-cpu: Sep 16 12:38:51 hydra01 kernel: [2091839.060045] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060050] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060054] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060059] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060063] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060068] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060072] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060077] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060081] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060085] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060090] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060095] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060098] Node 0 Normal per-cpu: Sep 16 12:38:51 hydra01 kernel: [2091839.060104] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060109] CPU 1: hi: 186, btch: 31 usd: 186 Sep 16 12:38:51 hydra01 kernel: [2091839.060113] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060118] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060122] CPU 4: hi: 186, btch: 31 usd: 30 Sep 16 12:38:51 hydra01 kernel: [2091839.060126] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060130] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060135] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060139] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060144] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060149] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060154] CPU 11: hi: 186, btch: 31 usd: 30 Sep 16 12:38:51 hydra01 kernel: [2091839.060167] active_anon:1284687 inactive_anon:823318 isolated_anon:0 Sep 16 12:38:51 hydra01 kernel: [2091839.060167] active_file:1869218 inactive_file:3486206 isolated_file:0 Sep 16 12:38:51 hydra01 kernel: [2091839.060167] unevictable:0 dirty:88 writeback:0 unstable:0 Sep 16 12:38:51 hydra01 kernel: [2091839.060167] free:75449 slab_reclaimable:369317 slab_unreclaimable:102168 Sep 16 12:38:51 hydra01 kernel: [2091839.060167] mapped:25890 shmem:151 pagetables:9660 bounce:0 Sep 16 12:38:51 hydra01 kernel: [2091839.060167] free_cma:0 Sep 16 12:38:51 hydra01 kernel: [2091839.060179] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:38:51 hydra01 kernel: [2091839.060194] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:38:51 hydra01 kernel: [2091839.060203] Node 0 DMA32 free:137920kB min:6224kB low:7780kB high:9336kB active_anon:430388kB inactive_anon:793008kB active_file:95328kB inactive_file:1387696kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165776kB slab_unreclaimable:43508kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:51 hydra01 kernel: [2091839.060220] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:38:51 hydra01 kernel: [2091839.060228] Node 0 Normal free:148324kB min:61324kB low:76652kB high:91984kB active_anon:4708360kB inactive_anon:2500264kB active_file:7381544kB inactive_file:12556744kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:356kB writeback:0kB mapped:100788kB shmem:604kB slab_reclaimable:1311492kB slab_unreclaimable:365164kB kernel_stack:8024kB pagetables:37112kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:38:51 hydra01 kernel: [2091839.060244] lowmem_reserve[]: 0 0 0 0 Sep 16 12:38:51 hydra01 kernel: [2091839.060252] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:38:51 hydra01 kernel: [2091839.060286] Node 0 DMA32: 4748*4kB (UEM) 1573*8kB (UEM) 5799*16kB (UEM) 436*32kB (UEMR) 2*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 138952kB Sep 16 12:38:51 hydra01 kernel: [2091839.060317] Node 0 Normal: 20575*4kB (UEMR) 252*8kB (UEMR) 4007*16kB (UEMR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 150284kB Sep 16 12:38:51 hydra01 kernel: [2091839.060348] 5786872 total pagecache pages Sep 16 12:38:51 hydra01 kernel: [2091839.060352] 431659 pages in swap cache Sep 16 12:38:51 hydra01 kernel: [2091839.060356] Swap cache stats: add 20774746, delete 20343087, find 12661890/14316286 Sep 16 12:38:51 hydra01 kernel: [2091839.060360] Free swap = 185326024kB Sep 16 12:38:51 hydra01 kernel: [2091839.060363] Total swap = 187617276kB Sep 16 12:38:51 hydra01 kernel: [2091839.255129] 8388592 pages RAM Sep 16 12:38:51 hydra01 kernel: [2091839.255134] 151696 pages reserved Sep 16 12:38:51 hydra01 kernel: [2091839.255136] 8417239 pages shared Sep 16 12:38:51 hydra01 kernel: [2091839.255139] 3725529 pages non-shared Sep 16 12:38:52 hydra01 kernel: [2091839.256873] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:38:52 hydra01 kernel: [2091839.256880] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:38:52 hydra01 kernel: [2091839.256883] Call Trace: Sep 16 12:38:52 hydra01 kernel: [2091839.256896] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.256904] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:38:52 hydra01 kernel: [2091839.256910] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:38:52 hydra01 kernel: [2091839.256918] [] alloc_pages_current+0xb8/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.256923] [] __get_free_pages+0xe/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.256930] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:38:52 hydra01 kernel: [2091839.256936] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.256942] [] __kmalloc+0x161/0x170 Sep 16 12:38:52 hydra01 kernel: [2091839.256950] [] ? setkey+0x81/0x100 Sep 16 12:38:52 hydra01 kernel: [2091839.256956] [] ? get_iowait_time+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.256961] [] stat_open+0x56/0xc0 Sep 16 12:38:52 hydra01 kernel: [2091839.256967] [] proc_reg_open+0xa0/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.256975] [] ? seq_release_private+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.256980] [] do_dentry_open+0x203/0x290 Sep 16 12:38:52 hydra01 kernel: [2091839.256985] [] ? init_once+0x20/0x20 Sep 16 12:38:52 hydra01 kernel: [2091839.256989] [] vfs_open+0x49/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.256995] [] do_last+0x296/0xea0 Sep 16 12:38:52 hydra01 kernel: [2091839.256999] [] ? inode_permission+0x18/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.257004] [] ? link_path_walk+0x6b/0x880 Sep 16 12:38:52 hydra01 kernel: [2091839.257009] [] path_openat+0xb3/0x4b0 Sep 16 12:38:52 hydra01 kernel: [2091839.257015] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.257022] [] ? unmap_region+0xdb/0x120 Sep 16 12:38:52 hydra01 kernel: [2091839.257026] [] do_filp_open+0x41/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.257032] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:38:52 hydra01 kernel: [2091839.257037] [] do_sys_open+0xf3/0x230 Sep 16 12:38:52 hydra01 kernel: [2091839.257041] [] sys_open+0x21/0x30 Sep 16 12:38:52 hydra01 kernel: [2091839.257047] [] system_call_fastpath+0x1a/0x1f Sep 16 12:38:52 hydra01 kernel: [2091839.257049] Mem-Info: Sep 16 12:38:52 hydra01 kernel: [2091839.257052] Node 0 DMA per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.257057] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257059] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257062] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257065] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257067] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257070] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257072] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257075] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257077] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257080] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257083] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257085] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257087] Node 0 DMA32 per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.257091] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257094] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257097] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257099] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257102] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257105] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257107] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257110] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257113] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257115] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257118] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257121] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257122] Node 0 Normal per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.257126] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257129] CPU 1: hi: 186, btch: 31 usd: 29 Sep 16 12:38:52 hydra01 kernel: [2091839.257131] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257134] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257137] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257139] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257142] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257144] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257147] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257150] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257152] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257155] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257163] active_anon:1284599 inactive_anon:823294 isolated_anon:0 Sep 16 12:38:52 hydra01 kernel: [2091839.257163] active_file:1869205 inactive_file:3481068 isolated_file:0 Sep 16 12:38:52 hydra01 kernel: [2091839.257163] unevictable:0 dirty:57 writeback:0 unstable:0 Sep 16 12:38:52 hydra01 kernel: [2091839.257163] free:81292 slab_reclaimable:369181 slab_unreclaimable:102144 Sep 16 12:38:52 hydra01 kernel: [2091839.257163] mapped:25896 shmem:152 pagetables:9654 bounce:0 Sep 16 12:38:52 hydra01 kernel: [2091839.257163] free_cma:0 Sep 16 12:38:52 hydra01 kernel: [2091839.257169] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:38:52 hydra01 kernel: [2091839.257178] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:38:52 hydra01 kernel: [2091839.257184] Node 0 DMA32 free:138592kB min:6224kB low:7780kB high:9336kB active_anon:430384kB inactive_anon:793000kB active_file:95276kB inactive_file:1387772kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165684kB slab_unreclaimable:43572kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.257193] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:38:52 hydra01 kernel: [2091839.257199] Node 0 Normal free:170732kB min:61324kB low:76652kB high:91984kB active_anon:4708012kB inactive_anon:2500176kB active_file:7381544kB inactive_file:12536500kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:228kB writeback:0kB mapped:100812kB shmem:608kB slab_reclaimable:1311040kB slab_unreclaimable:365004kB kernel_stack:8048kB pagetables:37088kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.257208] lowmem_reserve[]: 0 0 0 0 Sep 16 12:38:52 hydra01 kernel: [2091839.257213] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:38:52 hydra01 kernel: [2091839.257234] Node 0 DMA32: 4761*4kB (UEM) 1573*8kB (UEM) 5799*16kB (UEM) 436*32kB (UEMR) 2*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139004kB Sep 16 12:38:52 hydra01 kernel: [2091839.257253] Node 0 Normal: 21431*4kB (UEMR) 421*8kB (UEMR) 4514*16kB (UEMR) 270*32kB (MR) 17*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 172324kB Sep 16 12:38:52 hydra01 kernel: [2091839.257273] 5782109 total pagecache pages Sep 16 12:38:52 hydra01 kernel: [2091839.257275] 431659 pages in swap cache Sep 16 12:38:52 hydra01 kernel: [2091839.257279] Swap cache stats: add 20774746, delete 20343087, find 12661890/14316286 Sep 16 12:38:52 hydra01 kernel: [2091839.257281] Free swap = 185326024kB Sep 16 12:38:52 hydra01 kernel: [2091839.257283] Total swap = 187617276kB Sep 16 12:38:52 hydra01 kernel: [2091839.452131] 8388592 pages RAM Sep 16 12:38:52 hydra01 kernel: [2091839.452156] 151696 pages reserved Sep 16 12:38:52 hydra01 kernel: [2091839.452161] 8417508 pages shared Sep 16 12:38:52 hydra01 kernel: [2091839.452167] 3725512 pages non-shared Sep 16 12:38:52 hydra01 kernel: [2091839.453312] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:38:52 hydra01 kernel: [2091839.453316] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:38:52 hydra01 kernel: [2091839.453318] Call Trace: Sep 16 12:38:52 hydra01 kernel: [2091839.453328] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.453333] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:38:52 hydra01 kernel: [2091839.453337] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:38:52 hydra01 kernel: [2091839.453342] [] alloc_pages_current+0xb8/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.453346] [] __get_free_pages+0xe/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.453352] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:38:52 hydra01 kernel: [2091839.453356] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.453360] [] __kmalloc+0x161/0x170 Sep 16 12:38:52 hydra01 kernel: [2091839.453365] [] ? setkey+0x80/0x100 Sep 16 12:38:52 hydra01 kernel: [2091839.453370] [] ? get_iowait_time+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.453374] [] stat_open+0x56/0xc0 Sep 16 12:38:52 hydra01 kernel: [2091839.453378] [] proc_reg_open+0xa0/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.453383] [] ? seq_release_private+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.453387] [] do_dentry_open+0x203/0x290 Sep 16 12:38:52 hydra01 kernel: [2091839.453391] [] ? init_once+0x20/0x20 Sep 16 12:38:52 hydra01 kernel: [2091839.453394] [] vfs_open+0x49/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.453399] [] do_last+0x296/0xea0 Sep 16 12:38:52 hydra01 kernel: [2091839.453402] [] ? inode_permission+0x18/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.453406] [] ? link_path_walk+0x6b/0x880 Sep 16 12:38:52 hydra01 kernel: [2091839.453409] [] path_openat+0xb3/0x4b0 Sep 16 12:38:52 hydra01 kernel: [2091839.453414] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.453419] [] ? unmap_region+0xdb/0x120 Sep 16 12:38:52 hydra01 kernel: [2091839.453423] [] do_filp_open+0x41/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.453427] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:38:52 hydra01 kernel: [2091839.453431] [] do_sys_open+0xf3/0x230 Sep 16 12:38:52 hydra01 kernel: [2091839.453435] [] sys_open+0x21/0x30 Sep 16 12:38:52 hydra01 kernel: [2091839.453439] [] system_call_fastpath+0x1a/0x1f Sep 16 12:38:52 hydra01 kernel: [2091839.453441] Mem-Info: Sep 16 12:38:52 hydra01 kernel: [2091839.453443] Node 0 DMA per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.453447] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453449] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453451] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453453] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453455] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453457] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453459] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453461] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453463] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453466] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453468] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453470] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453472] Node 0 DMA32 per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.453475] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453477] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453479] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453481] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453483] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453485] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453488] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453490] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453492] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453494] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453496] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453499] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453500] Node 0 Normal per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.453503] CPU 0: hi: 186, btch: 31 usd: 30 Sep 16 12:38:52 hydra01 kernel: [2091839.453506] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453508] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453510] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453512] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453514] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453516] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453518] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453520] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453522] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453524] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453527] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453533] active_anon:1284599 inactive_anon:823294 isolated_anon:0 Sep 16 12:38:52 hydra01 kernel: [2091839.453533] active_file:1869205 inactive_file:3481068 isolated_file:0 Sep 16 12:38:52 hydra01 kernel: [2091839.453533] unevictable:0 dirty:57 writeback:0 unstable:0 Sep 16 12:38:52 hydra01 kernel: [2091839.453533] free:81257 slab_reclaimable:369181 slab_unreclaimable:102144 Sep 16 12:38:52 hydra01 kernel: [2091839.453533] mapped:25896 shmem:152 pagetables:9654 bounce:0 Sep 16 12:38:52 hydra01 kernel: [2091839.453533] free_cma:0 Sep 16 12:38:52 hydra01 kernel: [2091839.453538] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:38:52 hydra01 kernel: [2091839.453545] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:38:52 hydra01 kernel: [2091839.453550] Node 0 DMA32 free:138452kB min:6224kB low:7780kB high:9336kB active_anon:430384kB inactive_anon:793000kB active_file:95276kB inactive_file:1387772kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165684kB slab_unreclaimable:43572kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.453558] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:38:52 hydra01 kernel: [2091839.453562] Node 0 Normal free:170732kB min:61324kB low:76652kB high:91984kB active_anon:4708012kB inactive_anon:2500176kB active_file:7381544kB inactive_file:12536500kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:228kB writeback:0kB mapped:100812kB shmem:608kB slab_reclaimable:1311040kB slab_unreclaimable:365004kB kernel_stack:8048kB pagetables:37088kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.453569] lowmem_reserve[]: 0 0 0 0 Sep 16 12:38:52 hydra01 kernel: [2091839.453572] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:38:52 hydra01 kernel: [2091839.453583] Node 0 DMA32: 4761*4kB (UEM) 1573*8kB (UEM) 5799*16kB (UEM) 435*32kB (UEMR) 2*64kB (U) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 138972kB Sep 16 12:38:52 hydra01 kernel: [2091839.453593] Node 0 Normal: 21425*4kB (UEMR) 421*8kB (UEMR) 4514*16kB (UEMR) 271*32kB (UMR) 17*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 172332kB Sep 16 12:38:52 hydra01 kernel: [2091839.453604] 5782109 total pagecache pages Sep 16 12:38:52 hydra01 kernel: [2091839.453605] 431659 pages in swap cache Sep 16 12:38:52 hydra01 kernel: [2091839.453607] Swap cache stats: add 20774746, delete 20343087, find 12661890/14316286 Sep 16 12:38:52 hydra01 kernel: [2091839.453608] Free swap = 185326024kB Sep 16 12:38:52 hydra01 kernel: [2091839.453609] Total swap = 187617276kB Sep 16 12:38:52 hydra01 kernel: [2091839.563710] 8388592 pages RAM Sep 16 12:38:52 hydra01 kernel: [2091839.563716] 151696 pages reserved Sep 16 12:38:52 hydra01 kernel: [2091839.563721] 8417976 pages shared Sep 16 12:38:52 hydra01 kernel: [2091839.563726] 3725533 pages non-shared Sep 16 12:38:52 hydra01 kernel: [2091839.564556] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:38:52 hydra01 kernel: [2091839.564559] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:38:52 hydra01 kernel: [2091839.564560] Call Trace: Sep 16 12:38:52 hydra01 kernel: [2091839.564568] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.564572] [] ? drain_local_pages+0x16/0x20 Sep 16 12:38:52 hydra01 kernel: [2091839.564577] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:38:52 hydra01 kernel: [2091839.564580] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:38:52 hydra01 kernel: [2091839.564584] [] alloc_pages_current+0xb8/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.564587] [] __get_free_pages+0xe/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.564591] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:38:52 hydra01 kernel: [2091839.564594] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.564597] [] __kmalloc+0x161/0x170 Sep 16 12:38:52 hydra01 kernel: [2091839.564601] [] ? amiga_partition+0x2c4/0x4d0 Sep 16 12:38:52 hydra01 kernel: [2091839.564605] [] ? get_iowait_time+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.564607] [] stat_open+0x56/0xc0 Sep 16 12:38:52 hydra01 kernel: [2091839.564611] [] proc_reg_open+0xa0/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.564615] [] ? seq_release_private+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.564618] [] do_dentry_open+0x203/0x290 Sep 16 12:38:52 hydra01 kernel: [2091839.564621] [] ? init_once+0x20/0x20 Sep 16 12:38:52 hydra01 kernel: [2091839.564623] [] vfs_open+0x49/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.564626] [] do_last+0x296/0xea0 Sep 16 12:38:52 hydra01 kernel: [2091839.564628] [] ? inode_permission+0x18/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.564631] [] ? link_path_walk+0x6b/0x880 Sep 16 12:38:52 hydra01 kernel: [2091839.564633] [] path_openat+0xb3/0x4b0 Sep 16 12:38:52 hydra01 kernel: [2091839.564637] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.564640] [] ? unmap_region+0xdb/0x120 Sep 16 12:38:52 hydra01 kernel: [2091839.564643] [] do_filp_open+0x41/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.564645] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:38:52 hydra01 kernel: [2091839.564648] [] do_sys_open+0xf3/0x230 Sep 16 12:38:52 hydra01 kernel: [2091839.564650] [] sys_open+0x21/0x30 Sep 16 12:38:52 hydra01 kernel: [2091839.564654] [] system_call_fastpath+0x1a/0x1f Sep 16 12:38:52 hydra01 kernel: [2091839.564655] Mem-Info: Sep 16 12:38:52 hydra01 kernel: [2091839.564657] Node 0 DMA per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.564659] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564661] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564662] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564664] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564665] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564666] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564668] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564669] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564670] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564672] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564673] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564674] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564675] Node 0 DMA32 per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.564677] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564679] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564680] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564682] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564683] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564684] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564686] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564687] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564688] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564690] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564691] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564693] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564694] Node 0 Normal per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.564695] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564697] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564698] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564699] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564701] CPU 4: hi: 186, btch: 31 usd: 24 Sep 16 12:38:52 hydra01 kernel: [2091839.564702] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564704] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564705] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564706] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564708] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564709] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564710] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564714] active_anon:1284599 inactive_anon:823294 isolated_anon:0 Sep 16 12:38:52 hydra01 kernel: [2091839.564714] active_file:1869205 inactive_file:3481360 isolated_file:0 Sep 16 12:38:52 hydra01 kernel: [2091839.564714] unevictable:0 dirty:166 writeback:0 unstable:0 Sep 16 12:38:52 hydra01 kernel: [2091839.564714] free:81062 slab_reclaimable:369181 slab_unreclaimable:102144 Sep 16 12:38:52 hydra01 kernel: [2091839.564714] mapped:25896 shmem:152 pagetables:9654 bounce:0 Sep 16 12:38:52 hydra01 kernel: [2091839.564714] free_cma:0 Sep 16 12:38:52 hydra01 kernel: [2091839.564717] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:38:52 hydra01 kernel: [2091839.564722] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:38:52 hydra01 kernel: [2091839.564725] Node 0 DMA32 free:138488kB min:6224kB low:7780kB high:9336kB active_anon:430384kB inactive_anon:793000kB active_file:95276kB inactive_file:1387772kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165684kB slab_unreclaimable:43572kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.564730] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:38:52 hydra01 kernel: [2091839.564733] Node 0 Normal free:169916kB min:61324kB low:76652kB high:91984kB active_anon:4708012kB inactive_anon:2500176kB active_file:7381544kB inactive_file:12537668kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:664kB writeback:0kB mapped:100812kB shmem:608kB slab_reclaimable:1311040kB slab_unreclaimable:365004kB kernel_stack:8048kB pagetables:37088kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.564738] lowmem_reserve[]: 0 0 0 0 Sep 16 12:38:52 hydra01 kernel: [2091839.564741] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:38:52 hydra01 kernel: [2091839.564752] Node 0 DMA32: 4761*4kB (UEM) 1573*8kB (UEM) 5799*16kB (UEM) 433*32kB (UEMR) 2*64kB (U) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139036kB Sep 16 12:38:52 hydra01 kernel: [2091839.564762] Node 0 Normal: 20994*4kB (UEMR) 422*8kB (UEMR) 4514*16kB (UEMR) 269*32kB (UMR) 17*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170552kB Sep 16 12:38:52 hydra01 kernel: [2091839.564773] 5782436 total pagecache pages Sep 16 12:38:52 hydra01 kernel: [2091839.564774] 431659 pages in swap cache Sep 16 12:38:52 hydra01 kernel: [2091839.564776] Swap cache stats: add 20774746, delete 20343087, find 12661890/14316286 Sep 16 12:38:52 hydra01 kernel: [2091839.564777] Free swap = 185326024kB Sep 16 12:38:52 hydra01 kernel: [2091839.564778] Total swap = 187617276kB Sep 16 12:38:52 hydra01 kernel: [2091839.672294] 8388592 pages RAM Sep 16 12:38:52 hydra01 kernel: [2091839.672300] 151696 pages reserved Sep 16 12:38:52 hydra01 kernel: [2091839.672305] 8417384 pages shared Sep 16 12:38:52 hydra01 kernel: [2091839.672311] 3725487 pages non-shared Sep 16 12:38:52 hydra01 kernel: [2091839.673218] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:38:52 hydra01 kernel: [2091839.673222] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:38:52 hydra01 kernel: [2091839.673223] Call Trace: Sep 16 12:38:52 hydra01 kernel: [2091839.673231] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.673235] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:38:52 hydra01 kernel: [2091839.673238] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:38:52 hydra01 kernel: [2091839.673243] [] alloc_pages_current+0xb8/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.673246] [] __get_free_pages+0xe/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.673250] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:38:52 hydra01 kernel: [2091839.673253] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:38:52 hydra01 kernel: [2091839.673256] [] __kmalloc+0x161/0x170 Sep 16 12:38:52 hydra01 kernel: [2091839.673260] [] ? amiga_partition+0x2c4/0x4d0 Sep 16 12:38:52 hydra01 kernel: [2091839.673264] [] ? get_iowait_time+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.673266] [] stat_open+0x56/0xc0 Sep 16 12:38:52 hydra01 kernel: [2091839.673270] [] proc_reg_open+0xa0/0x180 Sep 16 12:38:52 hydra01 kernel: [2091839.673274] [] ? seq_release_private+0x60/0x60 Sep 16 12:38:52 hydra01 kernel: [2091839.673277] [] do_dentry_open+0x203/0x290 Sep 16 12:38:52 hydra01 kernel: [2091839.673280] [] ? init_once+0x20/0x20 Sep 16 12:38:52 hydra01 kernel: [2091839.673282] [] vfs_open+0x49/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.673285] [] do_last+0x296/0xea0 Sep 16 12:38:52 hydra01 kernel: [2091839.673287] [] ? inode_permission+0x18/0x50 Sep 16 12:38:52 hydra01 kernel: [2091839.673289] [] ? link_path_walk+0x6b/0x880 Sep 16 12:38:52 hydra01 kernel: [2091839.673292] [] path_openat+0xb3/0x4b0 Sep 16 12:38:52 hydra01 kernel: [2091839.673296] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.673299] [] ? unmap_region+0xdb/0x120 Sep 16 12:38:52 hydra01 kernel: [2091839.673302] [] do_filp_open+0x41/0xa0 Sep 16 12:38:52 hydra01 kernel: [2091839.673305] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:38:52 hydra01 kernel: [2091839.673307] [] do_sys_open+0xf3/0x230 Sep 16 12:38:52 hydra01 kernel: [2091839.673310] [] sys_open+0x21/0x30 Sep 16 12:38:52 hydra01 kernel: [2091839.673313] [] system_call_fastpath+0x1a/0x1f Sep 16 12:38:52 hydra01 kernel: [2091839.673315] Mem-Info: Sep 16 12:38:52 hydra01 kernel: [2091839.673316] Node 0 DMA per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.673319] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673320] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673321] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673323] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673324] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673325] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673327] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673328] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673329] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673331] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673332] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673334] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673335] Node 0 DMA32 per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.673337] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673338] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673340] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673341] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673342] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673344] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673345] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673346] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673348] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673349] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673351] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673352] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673353] Node 0 Normal per-cpu: Sep 16 12:38:52 hydra01 kernel: [2091839.673355] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673356] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673357] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673359] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673360] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673362] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673363] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673364] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673366] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673367] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673368] CPU 10: hi: 186, btch: 31 usd: 29 Sep 16 12:38:52 hydra01 kernel: [2091839.673370] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673374] active_anon:1284599 inactive_anon:823294 isolated_anon:0 Sep 16 12:38:52 hydra01 kernel: [2091839.673374] active_file:1869205 inactive_file:3481433 isolated_file:0 Sep 16 12:38:52 hydra01 kernel: [2091839.673374] unevictable:0 dirty:0 writeback:0 unstable:0 Sep 16 12:38:52 hydra01 kernel: [2091839.673374] free:81135 slab_reclaimable:369181 slab_unreclaimable:102144 Sep 16 12:38:52 hydra01 kernel: [2091839.673374] mapped:25896 shmem:152 pagetables:9654 bounce:0 Sep 16 12:38:52 hydra01 kernel: [2091839.673374] free_cma:0 Sep 16 12:38:52 hydra01 kernel: [2091839.673377] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:38:52 hydra01 kernel: [2091839.673382] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:38:52 hydra01 kernel: [2091839.673385] Node 0 DMA32 free:138488kB min:6224kB low:7780kB high:9336kB active_anon:430384kB inactive_anon:793000kB active_file:95276kB inactive_file:1387772kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165684kB slab_unreclaimable:43572kB kernel_stack:784kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.673390] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:38:52 hydra01 kernel: [2091839.673392] Node 0 Normal free:170208kB min:61324kB low:76652kB high:91984kB active_anon:4708012kB inactive_anon:2500176kB active_file:7381544kB inactive_file:12537960kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:0kB mapped:100812kB shmem:608kB slab_reclaimable:1311040kB slab_unreclaimable:365004kB kernel_stack:8048kB pagetables:37088kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:38:52 hydra01 kernel: [2091839.673397] lowmem_reserve[]: 0 0 0 0 Sep 16 12:38:52 hydra01 kernel: [2091839.673400] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:38:52 hydra01 kernel: [2091839.673411] Node 0 DMA32: 4761*4kB (UEM) 1573*8kB (UEM) 5799*16kB (UEM) 434*32kB (UEMR) 2*64kB (U) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139068kB Sep 16 12:38:52 hydra01 kernel: [2091839.673421] Node 0 Normal: 21017*4kB (UEMR) 426*8kB (UEMR) 4514*16kB (UEMR) 268*32kB (UMR) 17*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170644kB Sep 16 12:38:52 hydra01 kernel: [2091839.673432] 5782436 total pagecache pages Sep 16 12:38:52 hydra01 kernel: [2091839.673433] 431659 pages in swap cache Sep 16 12:38:52 hydra01 kernel: [2091839.673435] Swap cache stats: add 20774746, delete 20343087, find 12661890/14316286 Sep 16 12:38:52 hydra01 kernel: [2091839.673436] Free swap = 185326024kB Sep 16 12:38:52 hydra01 kernel: [2091839.673437] Total swap = 187617276kB Sep 16 12:38:52 hydra01 kernel: [2091839.780357] 8388592 pages RAM Sep 16 12:38:52 hydra01 kernel: [2091839.780361] 151696 pages reserved Sep 16 12:38:52 hydra01 kernel: [2091839.780362] 8417892 pages shared Sep 16 12:38:52 hydra01 kernel: [2091839.780363] 3725479 pages non-shared Sep 16 12:39:14 hydra01 kernel: [2091861.286032] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:39:14 hydra01 kernel: [2091861.286040] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:39:14 hydra01 kernel: [2091861.286044] Call Trace: Sep 16 12:39:14 hydra01 kernel: [2091861.286057] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:39:14 hydra01 kernel: [2091861.286076] [] ? drain_local_pages+0x16/0x20 Sep 16 12:39:14 hydra01 kernel: [2091861.286082] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:39:14 hydra01 kernel: [2091861.286086] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:39:14 hydra01 kernel: [2091861.286092] [] alloc_pages_current+0xb8/0x180 Sep 16 12:39:14 hydra01 kernel: [2091861.286096] [] __get_free_pages+0xe/0x50 Sep 16 12:39:14 hydra01 kernel: [2091861.286101] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:39:14 hydra01 kernel: [2091861.286105] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:39:14 hydra01 kernel: [2091861.286110] [] __kmalloc+0x161/0x170 Sep 16 12:39:14 hydra01 kernel: [2091861.286115] [] ? get_iowait_time+0x60/0x60 Sep 16 12:39:14 hydra01 kernel: [2091861.286118] [] stat_open+0x56/0xc0 Sep 16 12:39:14 hydra01 kernel: [2091861.286123] [] proc_reg_open+0xa0/0x180 Sep 16 12:39:14 hydra01 kernel: [2091861.286128] [] ? seq_release_private+0x60/0x60 Sep 16 12:39:14 hydra01 kernel: [2091861.286132] [] do_dentry_open+0x203/0x290 Sep 16 12:39:14 hydra01 kernel: [2091861.286136] [] ? init_once+0x20/0x20 Sep 16 12:39:14 hydra01 kernel: [2091861.286139] [] vfs_open+0x49/0x50 Sep 16 12:39:14 hydra01 kernel: [2091861.286143] [] do_last+0x296/0xea0 Sep 16 12:39:14 hydra01 kernel: [2091861.286147] [] ? inode_permission+0x18/0x50 Sep 16 12:39:14 hydra01 kernel: [2091861.286150] [] ? link_path_walk+0x6b/0x880 Sep 16 12:39:14 hydra01 kernel: [2091861.286154] [] path_openat+0xb3/0x4b0 Sep 16 12:39:14 hydra01 kernel: [2091861.286158] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:39:14 hydra01 kernel: [2091861.286163] [] ? unmap_region+0xdb/0x120 Sep 16 12:39:14 hydra01 kernel: [2091861.286166] [] do_filp_open+0x41/0xa0 Sep 16 12:39:14 hydra01 kernel: [2091861.286170] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:39:14 hydra01 kernel: [2091861.286173] [] do_sys_open+0xf3/0x230 Sep 16 12:39:14 hydra01 kernel: [2091861.286176] [] sys_open+0x21/0x30 Sep 16 12:39:14 hydra01 kernel: [2091861.286180] [] system_call_fastpath+0x1a/0x1f Sep 16 12:39:14 hydra01 kernel: [2091861.286182] Mem-Info: Sep 16 12:39:14 hydra01 kernel: [2091861.286184] Node 0 DMA per-cpu: Sep 16 12:39:14 hydra01 kernel: [2091861.286187] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286189] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286191] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286193] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286195] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286197] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286199] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286201] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286203] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286204] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286206] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286208] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286210] Node 0 DMA32 per-cpu: Sep 16 12:39:14 hydra01 kernel: [2091861.286212] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286214] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286216] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286218] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286220] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286222] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286224] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286239] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286242] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286245] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286247] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286250] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286252] Node 0 Normal per-cpu: Sep 16 12:39:14 hydra01 kernel: [2091861.286255] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286258] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286261] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286264] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286266] CPU 4: hi: 186, btch: 31 usd: 161 Sep 16 12:39:14 hydra01 kernel: [2091861.286269] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286272] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286274] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286277] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286280] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286282] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286285] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286292] active_anon:1285243 inactive_anon:823158 isolated_anon:0 Sep 16 12:39:14 hydra01 kernel: [2091861.286292] active_file:1869252 inactive_file:3487509 isolated_file:0 Sep 16 12:39:14 hydra01 kernel: [2091861.286292] unevictable:0 dirty:136 writeback:981 unstable:0 Sep 16 12:39:14 hydra01 kernel: [2091861.286292] free:76596 slab_reclaimable:368313 slab_unreclaimable:102045 Sep 16 12:39:14 hydra01 kernel: [2091861.286292] mapped:25941 shmem:151 pagetables:9620 bounce:0 Sep 16 12:39:14 hydra01 kernel: [2091861.286292] free_cma:0 Sep 16 12:39:14 hydra01 kernel: [2091861.286299] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:39:14 hydra01 kernel: [2091861.286308] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:39:14 hydra01 kernel: [2091861.286314] Node 0 DMA32 free:139432kB min:6224kB low:7780kB high:9336kB active_anon:430492kB inactive_anon:793008kB active_file:95336kB inactive_file:1387016kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165264kB slab_unreclaimable:43636kB kernel_stack:816kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:39:14 hydra01 kernel: [2091861.286324] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:39:14 hydra01 kernel: [2091861.286329] Node 0 Normal free:151400kB min:61324kB low:76652kB high:91984kB active_anon:4710480kB inactive_anon:2499624kB active_file:7381672kB inactive_file:12563020kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:544kB writeback:3924kB mapped:100992kB shmem:604kB slab_reclaimable:1307988kB slab_unreclaimable:364544kB kernel_stack:8016kB pagetables:36952kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:39:14 hydra01 kernel: [2091861.286338] lowmem_reserve[]: 0 0 0 0 Sep 16 12:39:14 hydra01 kernel: [2091861.286343] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:39:14 hydra01 kernel: [2091861.286365] Node 0 DMA32: 5140*4kB (UEM) 1567*8kB (UEM) 5795*16kB (UEM) 431*32kB (UEMR) 3*64kB (UM) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140440kB Sep 16 12:39:14 hydra01 kernel: [2091861.286393] Node 0 Normal: 30027*4kB (UEMR) 2842*8kB (UEMR) 366*16kB (UEMR) 88*32kB (UEMR) 8*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 153308kB Sep 16 12:39:14 hydra01 kernel: [2091861.286410] 5788621 total pagecache pages Sep 16 12:39:14 hydra01 kernel: [2091861.286412] 431649 pages in swap cache Sep 16 12:39:14 hydra01 kernel: [2091861.286414] Swap cache stats: add 20774760, delete 20343111, find 12661890/14316286 Sep 16 12:39:14 hydra01 kernel: [2091861.286416] Free swap = 185325968kB Sep 16 12:39:14 hydra01 kernel: [2091861.286417] Total swap = 187617276kB Sep 16 12:39:14 hydra01 kernel: [2091861.485868] 8388592 pages RAM Sep 16 12:39:14 hydra01 kernel: [2091861.485886] 151696 pages reserved Sep 16 12:39:14 hydra01 kernel: [2091861.485898] 8434816 pages shared Sep 16 12:39:14 hydra01 kernel: [2091861.485910] 3707973 pages non-shared Sep 16 12:39:14 hydra01 kernel: [2091861.633527] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:39:14 hydra01 kernel: [2091861.633536] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:39:14 hydra01 kernel: [2091861.633539] Call Trace: Sep 16 12:39:14 hydra01 kernel: [2091861.633552] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:39:14 hydra01 kernel: [2091861.633559] [] ? drain_local_pages+0x16/0x20 Sep 16 12:39:14 hydra01 kernel: [2091861.633567] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:39:14 hydra01 kernel: [2091861.633573] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:39:14 hydra01 kernel: [2091861.633581] [] alloc_pages_current+0xb8/0x180 Sep 16 12:39:14 hydra01 kernel: [2091861.633586] [] __get_free_pages+0xe/0x50 Sep 16 12:39:14 hydra01 kernel: [2091861.633593] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:39:14 hydra01 kernel: [2091861.633599] [] __kmalloc+0x161/0x170 Sep 16 12:39:14 hydra01 kernel: [2091861.633605] [] ? get_iowait_time+0x60/0x60 Sep 16 12:39:14 hydra01 kernel: [2091861.633610] [] stat_open+0x56/0xc0 Sep 16 12:39:14 hydra01 kernel: [2091861.633616] [] proc_reg_open+0xa0/0x180 Sep 16 12:39:14 hydra01 kernel: [2091861.633623] [] ? seq_release_private+0x60/0x60 Sep 16 12:39:14 hydra01 kernel: [2091861.633629] [] do_dentry_open+0x203/0x290 Sep 16 12:39:14 hydra01 kernel: [2091861.633634] [] ? init_once+0x20/0x20 Sep 16 12:39:14 hydra01 kernel: [2091861.633638] [] vfs_open+0x49/0x50 Sep 16 12:39:14 hydra01 kernel: [2091861.633643] [] do_last+0x296/0xea0 Sep 16 12:39:14 hydra01 kernel: [2091861.633648] [] ? inode_permission+0x18/0x50 Sep 16 12:39:14 hydra01 kernel: [2091861.633652] [] ? link_path_walk+0x6b/0x880 Sep 16 12:39:14 hydra01 kernel: [2091861.633657] [] path_openat+0xb3/0x4b0 Sep 16 12:39:14 hydra01 kernel: [2091861.633663] [] do_filp_open+0x41/0xa0 Sep 16 12:39:14 hydra01 kernel: [2091861.633668] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:39:14 hydra01 kernel: [2091861.633673] [] do_sys_open+0xf3/0x230 Sep 16 12:39:14 hydra01 kernel: [2091861.633678] [] ? sys_select+0xcb/0x110 Sep 16 12:39:14 hydra01 kernel: [2091861.633683] [] sys_open+0x21/0x30 Sep 16 12:39:14 hydra01 kernel: [2091861.633689] [] system_call_fastpath+0x1a/0x1f Sep 16 12:39:14 hydra01 kernel: [2091861.633691] Mem-Info: Sep 16 12:39:14 hydra01 kernel: [2091861.633694] Node 0 DMA per-cpu: Sep 16 12:39:14 hydra01 kernel: [2091861.633699] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633701] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633704] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633707] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633709] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633712] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633714] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633717] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633720] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633722] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633725] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633728] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633730] Node 0 DMA32 per-cpu: Sep 16 12:39:14 hydra01 kernel: [2091861.633733] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633736] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633739] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633742] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633744] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633747] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633749] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633752] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633755] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633757] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633760] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633763] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633764] Node 0 Normal per-cpu: Sep 16 12:39:14 hydra01 kernel: [2091861.633768] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633770] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633773] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633776] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633778] CPU 4: hi: 186, btch: 31 usd: 164 Sep 16 12:39:14 hydra01 kernel: [2091861.633781] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633784] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633786] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633789] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633792] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633794] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633797] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633805] active_anon:1285243 inactive_anon:823158 isolated_anon:0 Sep 16 12:39:14 hydra01 kernel: [2091861.633805] active_file:1869252 inactive_file:3484770 isolated_file:0 Sep 16 12:39:14 hydra01 kernel: [2091861.633805] unevictable:0 dirty:27 writeback:0 unstable:0 Sep 16 12:39:14 hydra01 kernel: [2091861.633805] free:79600 slab_reclaimable:368313 slab_unreclaimable:102045 Sep 16 12:39:14 hydra01 kernel: [2091861.633805] mapped:25941 shmem:151 pagetables:9620 bounce:0 Sep 16 12:39:14 hydra01 kernel: [2091861.633805] free_cma:0 Sep 16 12:39:14 hydra01 kernel: [2091861.633811] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:39:14 hydra01 kernel: [2091861.633820] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:39:14 hydra01 kernel: [2091861.633826] Node 0 DMA32 free:139564kB min:6224kB low:7780kB high:9336kB active_anon:430492kB inactive_anon:793008kB active_file:95336kB inactive_file:1387016kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2772kB shmem:0kB slab_reclaimable:165264kB slab_unreclaimable:43636kB kernel_stack:816kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:39:14 hydra01 kernel: [2091861.633835] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:39:14 hydra01 kernel: [2091861.633841] Node 0 Normal free:163284kB min:61324kB low:76652kB high:91984kB active_anon:4710480kB inactive_anon:2499624kB active_file:7381672kB inactive_file:12551680kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:108kB writeback:0kB mapped:100992kB shmem:604kB slab_reclaimable:1307988kB slab_unreclaimable:364544kB kernel_stack:8016kB pagetables:36952kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:39:14 hydra01 kernel: [2091861.633850] lowmem_reserve[]: 0 0 0 0 Sep 16 12:39:14 hydra01 kernel: [2091861.633855] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:39:14 hydra01 kernel: [2091861.633876] Node 0 DMA32: 5150*4kB (UEM) 1567*8kB (UEM) 5795*16kB (UEM) 432*32kB (UEMR) 3*64kB (UM) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140512kB Sep 16 12:39:14 hydra01 kernel: [2091861.633896] Node 0 Normal: 28279*4kB (UEMR) 2981*8kB (UEMR) 1450*16kB (UEMR) 91*32kB (UEMR) 8*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 164868kB Sep 16 12:39:14 hydra01 kernel: [2091861.633921] 5785569 total pagecache pages Sep 16 12:39:14 hydra01 kernel: [2091861.633924] 431649 pages in swap cache Sep 16 12:39:14 hydra01 kernel: [2091861.633927] Swap cache stats: add 20774760, delete 20343111, find 12661890/14316286 Sep 16 12:39:14 hydra01 kernel: [2091861.633929] Free swap = 185325968kB Sep 16 12:39:14 hydra01 kernel: [2091861.633931] Total swap = 187617276kB Sep 16 12:39:14 hydra01 kernel: [2091861.829168] 8388592 pages RAM Sep 16 12:39:14 hydra01 kernel: [2091861.829171] 151696 pages reserved Sep 16 12:39:14 hydra01 kernel: [2091861.829173] 8434443 pages shared Sep 16 12:39:14 hydra01 kernel: [2091861.829174] 3707443 pages non-shared Sep 16 12:39:44 hydra01 kernel: [2091891.634303] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:39:44 hydra01 kernel: [2091891.634308] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:39:44 hydra01 kernel: [2091891.634309] Call Trace: Sep 16 12:39:44 hydra01 kernel: [2091891.634319] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:39:44 hydra01 kernel: [2091891.634323] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:39:44 hydra01 kernel: [2091891.634327] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:39:44 hydra01 kernel: [2091891.634331] [] alloc_pages_current+0xb8/0x180 Sep 16 12:39:44 hydra01 kernel: [2091891.634334] [] __get_free_pages+0xe/0x50 Sep 16 12:39:44 hydra01 kernel: [2091891.634339] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:39:44 hydra01 kernel: [2091891.634342] [] __kmalloc+0x161/0x170 Sep 16 12:39:44 hydra01 kernel: [2091891.634346] [] ? get_iowait_time+0x60/0x60 Sep 16 12:39:44 hydra01 kernel: [2091891.634348] [] stat_open+0x56/0xc0 Sep 16 12:39:44 hydra01 kernel: [2091891.634352] [] proc_reg_open+0xa0/0x180 Sep 16 12:39:44 hydra01 kernel: [2091891.634356] [] ? seq_release_private+0x60/0x60 Sep 16 12:39:44 hydra01 kernel: [2091891.634359] [] do_dentry_open+0x203/0x290 Sep 16 12:39:44 hydra01 kernel: [2091891.634362] [] ? init_once+0x20/0x20 Sep 16 12:39:44 hydra01 kernel: [2091891.634364] [] vfs_open+0x49/0x50 Sep 16 12:39:44 hydra01 kernel: [2091891.634367] [] do_last+0x296/0xea0 Sep 16 12:39:44 hydra01 kernel: [2091891.634370] [] ? inode_permission+0x18/0x50 Sep 16 12:39:44 hydra01 kernel: [2091891.634373] [] ? link_path_walk+0x6b/0x880 Sep 16 12:39:44 hydra01 kernel: [2091891.634375] [] path_openat+0xb3/0x4b0 Sep 16 12:39:44 hydra01 kernel: [2091891.634378] [] do_filp_open+0x41/0xa0 Sep 16 12:39:44 hydra01 kernel: [2091891.634381] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:39:44 hydra01 kernel: [2091891.634384] [] do_sys_open+0xf3/0x230 Sep 16 12:39:44 hydra01 kernel: [2091891.634387] [] ? sys_select+0xcb/0x110 Sep 16 12:39:44 hydra01 kernel: [2091891.634389] [] sys_open+0x21/0x30 Sep 16 12:39:44 hydra01 kernel: [2091891.634392] [] system_call_fastpath+0x1a/0x1f Sep 16 12:39:44 hydra01 kernel: [2091891.634394] Mem-Info: Sep 16 12:39:44 hydra01 kernel: [2091891.634395] Node 0 DMA per-cpu: Sep 16 12:39:44 hydra01 kernel: [2091891.634398] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634399] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634401] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634402] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634404] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634405] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634406] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634408] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634409] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634410] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634412] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634413] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634414] Node 0 DMA32 per-cpu: Sep 16 12:39:44 hydra01 kernel: [2091891.634416] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634418] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634419] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634420] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634422] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634423] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634425] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634426] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634427] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634429] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634430] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634432] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634433] Node 0 Normal per-cpu: Sep 16 12:39:44 hydra01 kernel: [2091891.634434] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634436] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634437] CPU 2: hi: 186, btch: 31 usd: 175 Sep 16 12:39:44 hydra01 kernel: [2091891.634439] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634440] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634441] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634443] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634444] CPU 7: hi: 186, btch: 31 usd: 25 Sep 16 12:39:44 hydra01 kernel: [2091891.634446] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634447] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634448] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634450] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634454] active_anon:1285599 inactive_anon:822962 isolated_anon:0 Sep 16 12:39:44 hydra01 kernel: [2091891.634454] active_file:1833899 inactive_file:3539999 isolated_file:0 Sep 16 12:39:44 hydra01 kernel: [2091891.634454] unevictable:0 dirty:38 writeback:218 unstable:0 Sep 16 12:39:44 hydra01 kernel: [2091891.634454] free:61152 slab_reclaimable:367518 slab_unreclaimable:101915 Sep 16 12:39:44 hydra01 kernel: [2091891.634454] mapped:26082 shmem:151 pagetables:9642 bounce:0 Sep 16 12:39:44 hydra01 kernel: [2091891.634454] free_cma:0 Sep 16 12:39:44 hydra01 kernel: [2091891.634459] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:39:44 hydra01 kernel: [2091891.634466] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:39:44 hydra01 kernel: [2091891.634471] Node 0 DMA32 free:141592kB min:6224kB low:7780kB high:9336kB active_anon:430308kB inactive_anon:793008kB active_file:95336kB inactive_file:1383044kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2788kB shmem:0kB slab_reclaimable:164664kB slab_unreclaimable:43524kB kernel_stack:792kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:39:44 hydra01 kernel: [2091891.634479] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:39:44 hydra01 kernel: [2091891.634483] Node 0 Normal free:87464kB min:61324kB low:76652kB high:91984kB active_anon:4712088kB inactive_anon:2498840kB active_file:7240260kB inactive_file:12776952kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:588kB writeback:872kB mapped:101540kB shmem:604kB slab_reclaimable:1305408kB slab_unreclaimable:364136kB kernel_stack:8024kB pagetables:37040kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:20 all_unreclaimable? no Sep 16 12:39:44 hydra01 kernel: [2091891.634490] lowmem_reserve[]: 0 0 0 0 Sep 16 12:39:44 hydra01 kernel: [2091891.634494] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:39:44 hydra01 kernel: [2091891.634508] Node 0 DMA32: 6320*4kB (UEM) 1573*8kB (UEM) 5802*16kB (UEM) 349*32kB (UMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 142376kB Sep 16 12:39:44 hydra01 kernel: [2091891.634522] Node 0 Normal: 19859*4kB (UEMR) 169*8kB (UEMR) 262*16kB (UEMR) 73*32kB (UEMR) 7*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 89044kB Sep 16 12:39:44 hydra01 kernel: [2091891.634534] 5805688 total pagecache pages Sep 16 12:39:44 hydra01 kernel: [2091891.634535] 431644 pages in swap cache Sep 16 12:39:44 hydra01 kernel: [2091891.634537] Swap cache stats: add 20774782, delete 20343138, find 12661890/14316286 Sep 16 12:39:44 hydra01 kernel: [2091891.634538] Free swap = 185325884kB Sep 16 12:39:44 hydra01 kernel: [2091891.634539] Total swap = 187617276kB Sep 16 12:39:44 hydra01 kernel: [2091891.809674] 8388592 pages RAM Sep 16 12:39:44 hydra01 kernel: [2091891.809677] 151696 pages reserved Sep 16 12:39:44 hydra01 kernel: [2091891.809679] 8967801 pages shared Sep 16 12:39:44 hydra01 kernel: [2091891.809681] 3696915 pages non-shared Sep 16 12:40:44 hydra01 kernel: [2091951.633115] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:40:44 hydra01 kernel: [2091951.633124] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:40:44 hydra01 kernel: [2091951.633127] Call Trace: Sep 16 12:40:44 hydra01 kernel: [2091951.633140] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:40:44 hydra01 kernel: [2091951.633175] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:40:44 hydra01 kernel: [2091951.633182] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:40:44 hydra01 kernel: [2091951.633190] [] alloc_pages_current+0xb8/0x180 Sep 16 12:40:44 hydra01 kernel: [2091951.633195] [] __get_free_pages+0xe/0x50 Sep 16 12:40:44 hydra01 kernel: [2091951.633216] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:40:44 hydra01 kernel: [2091951.633227] [] __kmalloc+0x161/0x170 Sep 16 12:40:44 hydra01 kernel: [2091951.633234] [] ? get_iowait_time+0x60/0x60 Sep 16 12:40:44 hydra01 kernel: [2091951.633238] [] stat_open+0x56/0xc0 Sep 16 12:40:44 hydra01 kernel: [2091951.633244] [] proc_reg_open+0xa0/0x180 Sep 16 12:40:44 hydra01 kernel: [2091951.633251] [] ? seq_release_private+0x60/0x60 Sep 16 12:40:44 hydra01 kernel: [2091951.633257] [] do_dentry_open+0x203/0x290 Sep 16 12:40:44 hydra01 kernel: [2091951.633262] [] ? init_once+0x20/0x20 Sep 16 12:40:44 hydra01 kernel: [2091951.633266] [] vfs_open+0x49/0x50 Sep 16 12:40:44 hydra01 kernel: [2091951.633271] [] do_last+0x296/0xea0 Sep 16 12:40:44 hydra01 kernel: [2091951.633276] [] ? inode_permission+0x18/0x50 Sep 16 12:40:44 hydra01 kernel: [2091951.633280] [] ? link_path_walk+0x6b/0x880 Sep 16 12:40:44 hydra01 kernel: [2091951.633285] [] path_openat+0xb3/0x4b0 Sep 16 12:40:44 hydra01 kernel: [2091951.633290] [] do_filp_open+0x41/0xa0 Sep 16 12:40:44 hydra01 kernel: [2091951.633306] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:40:44 hydra01 kernel: [2091951.633321] [] do_sys_open+0xf3/0x230 Sep 16 12:40:44 hydra01 kernel: [2091951.633326] [] ? sys_select+0xcb/0x110 Sep 16 12:40:44 hydra01 kernel: [2091951.633331] [] sys_open+0x21/0x30 Sep 16 12:40:44 hydra01 kernel: [2091951.633336] [] system_call_fastpath+0x1a/0x1f Sep 16 12:40:44 hydra01 kernel: [2091951.633339] Mem-Info: Sep 16 12:40:44 hydra01 kernel: [2091951.633341] Node 0 DMA per-cpu: Sep 16 12:40:44 hydra01 kernel: [2091951.633346] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633349] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633352] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633354] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633361] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633371] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633380] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633383] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633386] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633388] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633391] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633394] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633396] Node 0 DMA32 per-cpu: Sep 16 12:40:44 hydra01 kernel: [2091951.633399] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633402] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633405] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633415] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633426] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633432] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633434] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633437] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633440] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633442] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633445] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633448] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633449] Node 0 Normal per-cpu: Sep 16 12:40:44 hydra01 kernel: [2091951.633453] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633460] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633471] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633478] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633481] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633483] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633486] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633489] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633491] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633494] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633497] CPU 10: hi: 186, btch: 31 usd: 29 Sep 16 12:40:44 hydra01 kernel: [2091951.633504] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633519] active_anon:1286480 inactive_anon:822523 isolated_anon:0 Sep 16 12:40:44 hydra01 kernel: [2091951.633519] active_file:1833941 inactive_file:3526193 isolated_file:0 Sep 16 12:40:44 hydra01 kernel: [2091951.633519] unevictable:0 dirty:122 writeback:0 unstable:0 Sep 16 12:40:44 hydra01 kernel: [2091951.633519] free:82863 slab_reclaimable:366370 slab_unreclaimable:101477 Sep 16 12:40:44 hydra01 kernel: [2091951.633519] mapped:26232 shmem:151 pagetables:9570 bounce:0 Sep 16 12:40:44 hydra01 kernel: [2091951.633519] free_cma:0 Sep 16 12:40:44 hydra01 kernel: [2091951.633534] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:40:44 hydra01 kernel: [2091951.633543] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:40:44 hydra01 kernel: [2091951.633549] Node 0 DMA32 free:140036kB min:6224kB low:7780kB high:9336kB active_anon:430556kB inactive_anon:793008kB active_file:95340kB inactive_file:1388328kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3312kB shmem:0kB slab_reclaimable:164612kB slab_unreclaimable:43284kB kernel_stack:864kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:40:44 hydra01 kernel: [2091951.633558] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:40:44 hydra01 kernel: [2091951.633568] Node 0 Normal free:175572kB min:61324kB low:76652kB high:91984kB active_anon:4715364kB inactive_anon:2497084kB active_file:7240424kB inactive_file:12716444kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:480kB writeback:0kB mapped:101616kB shmem:604kB slab_reclaimable:1300868kB slab_unreclaimable:362624kB kernel_stack:7944kB pagetables:36752kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:40:44 hydra01 kernel: [2091951.633582] lowmem_reserve[]: 0 0 0 0 Sep 16 12:40:44 hydra01 kernel: [2091951.633587] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:40:44 hydra01 kernel: [2091951.633613] Node 0 DMA32: 5597*4kB (UEM) 1569*8kB (UEM) 5703*16kB (UEM) 432*32kB (UEMR) 3*64kB (U) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140844kB Sep 16 12:40:44 hydra01 kernel: [2091951.633633] Node 0 Normal: 28089*4kB (UEMR) 6187*8kB (UEMR) 556*16kB (UEMR) 106*32kB (UEMR) 20*64kB (UEMR) 3*128kB (UR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 176828kB Sep 16 12:40:44 hydra01 kernel: [2091951.633653] 5792023 total pagecache pages Sep 16 12:40:44 hydra01 kernel: [2091951.633656] 431642 pages in swap cache Sep 16 12:40:44 hydra01 kernel: [2091951.633659] Swap cache stats: add 20774814, delete 20343172, find 12661891/14316287 Sep 16 12:40:44 hydra01 kernel: [2091951.633661] Free swap = 185325768kB Sep 16 12:40:44 hydra01 kernel: [2091951.633663] Total swap = 187617276kB Sep 16 12:40:44 hydra01 kernel: [2091951.827268] 8388592 pages RAM Sep 16 12:40:44 hydra01 kernel: [2091951.827272] 151696 pages reserved Sep 16 12:40:44 hydra01 kernel: [2091951.827273] 8473196 pages shared Sep 16 12:40:44 hydra01 kernel: [2091951.827274] 3667350 pages non-shared Sep 16 12:41:14 hydra01 kernel: [2091981.226793] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:41:14 hydra01 kernel: [2091981.226799] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:41:14 hydra01 kernel: [2091981.226802] Call Trace: Sep 16 12:41:14 hydra01 kernel: [2091981.226813] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:41:14 hydra01 kernel: [2091981.226819] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:41:14 hydra01 kernel: [2091981.226824] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:41:14 hydra01 kernel: [2091981.226830] [] alloc_pages_current+0xb8/0x180 Sep 16 12:41:14 hydra01 kernel: [2091981.226834] [] __get_free_pages+0xe/0x50 Sep 16 12:41:14 hydra01 kernel: [2091981.226841] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:41:14 hydra01 kernel: [2091981.226848] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:41:14 hydra01 kernel: [2091981.226851] [] __kmalloc+0x161/0x170 Sep 16 12:41:14 hydra01 kernel: [2091981.226855] [] ? get_iowait_time+0x60/0x60 Sep 16 12:41:14 hydra01 kernel: [2091981.226858] [] stat_open+0x56/0xc0 Sep 16 12:41:14 hydra01 kernel: [2091981.226861] [] proc_reg_open+0xa0/0x180 Sep 16 12:41:14 hydra01 kernel: [2091981.226866] [] ? seq_release_private+0x60/0x60 Sep 16 12:41:14 hydra01 kernel: [2091981.226869] [] do_dentry_open+0x203/0x290 Sep 16 12:41:14 hydra01 kernel: [2091981.226871] [] ? init_once+0x20/0x20 Sep 16 12:41:14 hydra01 kernel: [2091981.226874] [] vfs_open+0x49/0x50 Sep 16 12:41:14 hydra01 kernel: [2091981.226877] [] do_last+0x296/0xea0 Sep 16 12:41:14 hydra01 kernel: [2091981.226879] [] ? inode_permission+0x18/0x50 Sep 16 12:41:14 hydra01 kernel: [2091981.226882] [] ? link_path_walk+0x6b/0x880 Sep 16 12:41:14 hydra01 kernel: [2091981.226884] [] path_openat+0xb3/0x4b0 Sep 16 12:41:14 hydra01 kernel: [2091981.226888] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:41:14 hydra01 kernel: [2091981.226891] [] ? unmap_region+0xdb/0x120 Sep 16 12:41:14 hydra01 kernel: [2091981.226893] [] do_filp_open+0x41/0xa0 Sep 16 12:41:14 hydra01 kernel: [2091981.226896] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:41:14 hydra01 kernel: [2091981.226899] [] do_sys_open+0xf3/0x230 Sep 16 12:41:14 hydra01 kernel: [2091981.226901] [] sys_open+0x21/0x30 Sep 16 12:41:14 hydra01 kernel: [2091981.226904] [] system_call_fastpath+0x1a/0x1f Sep 16 12:41:14 hydra01 kernel: [2091981.226906] Mem-Info: Sep 16 12:41:14 hydra01 kernel: [2091981.226907] Node 0 DMA per-cpu: Sep 16 12:41:14 hydra01 kernel: [2091981.226910] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226912] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226913] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226914] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226916] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226917] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226919] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226920] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226922] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226923] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226925] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226926] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226927] Node 0 DMA32 per-cpu: Sep 16 12:41:14 hydra01 kernel: [2091981.226929] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226931] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226932] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226933] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226935] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226936] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226938] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226939] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226940] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226942] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226943] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226944] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226945] Node 0 Normal per-cpu: Sep 16 12:41:14 hydra01 kernel: [2091981.226947] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226949] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226950] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226951] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226953] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226954] CPU 5: hi: 186, btch: 31 usd: 160 Sep 16 12:41:14 hydra01 kernel: [2091981.226956] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226957] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226958] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226960] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226962] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226964] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.226970] active_anon:1287705 inactive_anon:822354 isolated_anon:0 Sep 16 12:41:14 hydra01 kernel: [2091981.226970] active_file:1833953 inactive_file:3530677 isolated_file:0 Sep 16 12:41:14 hydra01 kernel: [2091981.226970] unevictable:0 dirty:0 writeback:43 unstable:0 Sep 16 12:41:14 hydra01 kernel: [2091981.226970] free:81176 slab_reclaimable:365661 slab_unreclaimable:101324 Sep 16 12:41:14 hydra01 kernel: [2091981.226970] mapped:25954 shmem:151 pagetables:9681 bounce:0 Sep 16 12:41:14 hydra01 kernel: [2091981.226970] free_cma:0 Sep 16 12:41:14 hydra01 kernel: [2091981.226975] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:41:14 hydra01 kernel: [2091981.226982] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:41:14 hydra01 kernel: [2091981.226987] Node 0 DMA32 free:139632kB min:6224kB low:7780kB high:9336kB active_anon:434212kB inactive_anon:793008kB active_file:95340kB inactive_file:1385240kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2900kB shmem:0kB slab_reclaimable:164856kB slab_unreclaimable:43076kB kernel_stack:864kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:41:14 hydra01 kernel: [2091981.226994] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:41:14 hydra01 kernel: [2091981.226998] Node 0 Normal free:169228kB min:61324kB low:76652kB high:91984kB active_anon:4716608kB inactive_anon:2496408kB active_file:7240472kB inactive_file:12737468kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:172kB mapped:100916kB shmem:604kB slab_reclaimable:1297788kB slab_unreclaimable:362220kB kernel_stack:7952kB pagetables:37196kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:41:14 hydra01 kernel: [2091981.227006] lowmem_reserve[]: 0 0 0 0 Sep 16 12:41:14 hydra01 kernel: [2091981.227013] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:41:14 hydra01 kernel: [2091981.227025] Node 0 DMA32: 5623*4kB (UEM) 1571*8kB (UEM) 5677*16kB (UEM) 433*32kB (UEMR) 4*64kB (UM) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140516kB Sep 16 12:41:14 hydra01 kernel: [2091981.227035] Node 0 Normal: 30699*4kB (UEMR) 1257*8kB (UEMR) 1026*16kB (UEMR) 570*32kB (UEMR) 32*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170836kB Sep 16 12:41:14 hydra01 kernel: [2091981.227045] 5796380 total pagecache pages Sep 16 12:41:14 hydra01 kernel: [2091981.227047] 431641 pages in swap cache Sep 16 12:41:14 hydra01 kernel: [2091981.227049] Swap cache stats: add 20774842, delete 20343201, find 12661892/14316288 Sep 16 12:41:14 hydra01 kernel: [2091981.227050] Free swap = 185325664kB Sep 16 12:41:14 hydra01 kernel: [2091981.227051] Total swap = 187617276kB Sep 16 12:41:14 hydra01 kernel: [2091981.336826] 8388592 pages RAM Sep 16 12:41:14 hydra01 kernel: [2091981.336834] 151696 pages reserved Sep 16 12:41:14 hydra01 kernel: [2091981.336845] 8482536 pages shared Sep 16 12:41:14 hydra01 kernel: [2091981.336851] 3659608 pages non-shared Sep 16 12:41:14 hydra01 kernel: [2091981.629741] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:41:14 hydra01 kernel: [2091981.629750] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:41:14 hydra01 kernel: [2091981.629753] Call Trace: Sep 16 12:41:14 hydra01 kernel: [2091981.629766] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:41:14 hydra01 kernel: [2091981.629774] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:41:14 hydra01 kernel: [2091981.629781] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:41:14 hydra01 kernel: [2091981.629788] [] alloc_pages_current+0xb8/0x180 Sep 16 12:41:14 hydra01 kernel: [2091981.629794] [] __get_free_pages+0xe/0x50 Sep 16 12:41:14 hydra01 kernel: [2091981.629801] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:41:14 hydra01 kernel: [2091981.629807] [] __kmalloc+0x161/0x170 Sep 16 12:41:14 hydra01 kernel: [2091981.629813] [] ? get_iowait_time+0x60/0x60 Sep 16 12:41:14 hydra01 kernel: [2091981.629818] [] stat_open+0x56/0xc0 Sep 16 12:41:14 hydra01 kernel: [2091981.629824] [] proc_reg_open+0xa0/0x180 Sep 16 12:41:14 hydra01 kernel: [2091981.629831] [] ? seq_release_private+0x60/0x60 Sep 16 12:41:14 hydra01 kernel: [2091981.629837] [] do_dentry_open+0x203/0x290 Sep 16 12:41:14 hydra01 kernel: [2091981.629842] [] ? init_once+0x20/0x20 Sep 16 12:41:14 hydra01 kernel: [2091981.629846] [] vfs_open+0x49/0x50 Sep 16 12:41:14 hydra01 kernel: [2091981.629851] [] do_last+0x296/0xea0 Sep 16 12:41:14 hydra01 kernel: [2091981.629856] [] ? inode_permission+0x18/0x50 Sep 16 12:41:14 hydra01 kernel: [2091981.629860] [] ? link_path_walk+0x6b/0x880 Sep 16 12:41:14 hydra01 kernel: [2091981.629865] [] path_openat+0xb3/0x4b0 Sep 16 12:41:14 hydra01 kernel: [2091981.629871] [] do_filp_open+0x41/0xa0 Sep 16 12:41:14 hydra01 kernel: [2091981.629876] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:41:14 hydra01 kernel: [2091981.629881] [] do_sys_open+0xf3/0x230 Sep 16 12:41:14 hydra01 kernel: [2091981.629886] [] ? sys_select+0xcb/0x110 Sep 16 12:41:14 hydra01 kernel: [2091981.629891] [] sys_open+0x21/0x30 Sep 16 12:41:14 hydra01 kernel: [2091981.629896] [] system_call_fastpath+0x1a/0x1f Sep 16 12:41:14 hydra01 kernel: [2091981.629899] Mem-Info: Sep 16 12:41:14 hydra01 kernel: [2091981.629901] Node 0 DMA per-cpu: Sep 16 12:41:14 hydra01 kernel: [2091981.629906] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629909] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629911] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629914] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629916] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629919] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629922] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629924] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629927] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629929] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629932] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629935] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629937] Node 0 DMA32 per-cpu: Sep 16 12:41:14 hydra01 kernel: [2091981.629941] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629944] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629946] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629949] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629952] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629954] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629957] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629959] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629962] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629965] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629967] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629970] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629972] Node 0 Normal per-cpu: Sep 16 12:41:14 hydra01 kernel: [2091981.629976] CPU 0: hi: 186, btch: 31 usd: 159 Sep 16 12:41:14 hydra01 kernel: [2091981.629978] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629981] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629983] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629986] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629989] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629991] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629994] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629997] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.629999] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.630002] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.630005] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:41:14 hydra01 kernel: [2091981.630012] active_anon:1287705 inactive_anon:822354 isolated_anon:0 Sep 16 12:41:14 hydra01 kernel: [2091981.630012] active_file:1833953 inactive_file:3534780 isolated_file:0 Sep 16 12:41:14 hydra01 kernel: [2091981.630012] unevictable:0 dirty:0 writeback:0 unstable:0 Sep 16 12:41:14 hydra01 kernel: [2091981.630012] free:77169 slab_reclaimable:365661 slab_unreclaimable:101324 Sep 16 12:41:14 hydra01 kernel: [2091981.630012] mapped:25954 shmem:151 pagetables:9681 bounce:0 Sep 16 12:41:14 hydra01 kernel: [2091981.630012] free_cma:0 Sep 16 12:41:14 hydra01 kernel: [2091981.630018] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:41:14 hydra01 kernel: [2091981.630027] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:41:14 hydra01 kernel: [2091981.630033] Node 0 DMA32 free:139632kB min:6224kB low:7780kB high:9336kB active_anon:434212kB inactive_anon:793008kB active_file:95340kB inactive_file:1385240kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2900kB shmem:0kB slab_reclaimable:164856kB slab_unreclaimable:43076kB kernel_stack:864kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:41:14 hydra01 kernel: [2091981.630043] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:41:14 hydra01 kernel: [2091981.630048] Node 0 Normal free:153200kB min:61324kB low:76652kB high:91984kB active_anon:4716608kB inactive_anon:2496408kB active_file:7240472kB inactive_file:12753880kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:0kB mapped:100916kB shmem:604kB slab_reclaimable:1297788kB slab_unreclaimable:362220kB kernel_stack:7952kB pagetables:37196kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:41:14 hydra01 kernel: [2091981.630057] lowmem_reserve[]: 0 0 0 0 Sep 16 12:41:14 hydra01 kernel: [2091981.630062] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:41:14 hydra01 kernel: [2091981.630084] Node 0 DMA32: 5623*4kB (UEM) 1571*8kB (UEM) 5677*16kB (UEM) 435*32kB (UEMR) 4*64kB (UM) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140580kB Sep 16 12:41:14 hydra01 kernel: [2091981.630103] Node 0 Normal: 26688*4kB (UEMR) 1281*8kB (UEMR) 1027*16kB (UEMR) 568*32kB (UEMR) 32*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 154936kB Sep 16 12:41:14 hydra01 kernel: [2091981.630123] 5800522 total pagecache pages Sep 16 12:41:14 hydra01 kernel: [2091981.630125] 431641 pages in swap cache Sep 16 12:41:14 hydra01 kernel: [2091981.630128] Swap cache stats: add 20774842, delete 20343201, find 12661892/14316288 Sep 16 12:41:14 hydra01 kernel: [2091981.630131] Free swap = 185325664kB Sep 16 12:41:14 hydra01 kernel: [2091981.630133] Total swap = 187617276kB Sep 16 12:41:14 hydra01 kernel: [2091981.825613] 8388592 pages RAM Sep 16 12:41:14 hydra01 kernel: [2091981.825615] 151696 pages reserved Sep 16 12:41:14 hydra01 kernel: [2091981.825617] 8483946 pages shared Sep 16 12:41:14 hydra01 kernel: [2091981.825618] 3658347 pages non-shared Sep 16 12:41:29 hydra01 kernel: [2091996.628702] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:41:29 hydra01 kernel: [2091996.628711] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:41:29 hydra01 kernel: [2091996.628714] Call Trace: Sep 16 12:41:29 hydra01 kernel: [2091996.628727] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:41:29 hydra01 kernel: [2091996.628734] [] ? drain_local_pages+0x16/0x20 Sep 16 12:41:29 hydra01 kernel: [2091996.628742] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:41:29 hydra01 kernel: [2091996.628748] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:41:29 hydra01 kernel: [2091996.628756] [] alloc_pages_current+0xb8/0x180 Sep 16 12:41:29 hydra01 kernel: [2091996.628761] [] __get_free_pages+0xe/0x50 Sep 16 12:41:29 hydra01 kernel: [2091996.628768] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:41:29 hydra01 kernel: [2091996.628774] [] __kmalloc+0x161/0x170 Sep 16 12:41:29 hydra01 kernel: [2091996.628781] [] ? get_iowait_time+0x60/0x60 Sep 16 12:41:29 hydra01 kernel: [2091996.628785] [] stat_open+0x56/0xc0 Sep 16 12:41:29 hydra01 kernel: [2091996.628792] [] proc_reg_open+0xa0/0x180 Sep 16 12:41:29 hydra01 kernel: [2091996.628799] [] ? seq_release_private+0x60/0x60 Sep 16 12:41:29 hydra01 kernel: [2091996.628804] [] do_dentry_open+0x203/0x290 Sep 16 12:41:29 hydra01 kernel: [2091996.628809] [] ? init_once+0x20/0x20 Sep 16 12:41:29 hydra01 kernel: [2091996.628814] [] vfs_open+0x49/0x50 Sep 16 12:41:29 hydra01 kernel: [2091996.628819] [] do_last+0x296/0xea0 Sep 16 12:41:29 hydra01 kernel: [2091996.628824] [] ? inode_permission+0x18/0x50 Sep 16 12:41:29 hydra01 kernel: [2091996.628828] [] ? link_path_walk+0x6b/0x880 Sep 16 12:41:29 hydra01 kernel: [2091996.628833] [] path_openat+0xb3/0x4b0 Sep 16 12:41:29 hydra01 kernel: [2091996.628838] [] do_filp_open+0x41/0xa0 Sep 16 12:41:29 hydra01 kernel: [2091996.628844] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:41:29 hydra01 kernel: [2091996.628849] [] do_sys_open+0xf3/0x230 Sep 16 12:41:29 hydra01 kernel: [2091996.628854] [] ? sys_select+0xcb/0x110 Sep 16 12:41:29 hydra01 kernel: [2091996.628858] [] sys_open+0x21/0x30 Sep 16 12:41:29 hydra01 kernel: [2091996.628864] [] system_call_fastpath+0x1a/0x1f Sep 16 12:41:29 hydra01 kernel: [2091996.628867] Mem-Info: Sep 16 12:41:29 hydra01 kernel: [2091996.628869] Node 0 DMA per-cpu: Sep 16 12:41:29 hydra01 kernel: [2091996.628874] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628877] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628879] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628882] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628884] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628887] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628890] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628892] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628895] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628897] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628900] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628903] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628905] Node 0 DMA32 per-cpu: Sep 16 12:41:29 hydra01 kernel: [2091996.628909] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628912] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628914] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628917] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628920] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628922] CPU 5: hi: 186, btch: 31 usd: 1 Sep 16 12:41:29 hydra01 kernel: [2091996.628925] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628927] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628930] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628933] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628935] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628938] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628940] Node 0 Normal per-cpu: Sep 16 12:41:29 hydra01 kernel: [2091996.628943] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628946] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628949] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628951] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628954] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628957] CPU 5: hi: 186, btch: 31 usd: 178 Sep 16 12:41:29 hydra01 kernel: [2091996.628959] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628962] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628964] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628967] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628970] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628972] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:41:29 hydra01 kernel: [2091996.628980] active_anon:1287765 inactive_anon:822240 isolated_anon:0 Sep 16 12:41:29 hydra01 kernel: [2091996.628980] active_file:1834008 inactive_file:3532200 isolated_file:0 Sep 16 12:41:29 hydra01 kernel: [2091996.628980] unevictable:0 dirty:88 writeback:0 unstable:0 Sep 16 12:41:29 hydra01 kernel: [2091996.628980] free:80730 slab_reclaimable:365312 slab_unreclaimable:101276 Sep 16 12:41:29 hydra01 kernel: [2091996.628980] mapped:25749 shmem:151 pagetables:9572 bounce:0 Sep 16 12:41:29 hydra01 kernel: [2091996.628980] free_cma:0 Sep 16 12:41:29 hydra01 kernel: [2091996.628986] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:41:29 hydra01 kernel: [2091996.628995] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:41:29 hydra01 kernel: [2091996.629002] Node 0 DMA32 free:135876kB min:6224kB low:7780kB high:9336kB active_anon:434696kB inactive_anon:793016kB active_file:95348kB inactive_file:1384520kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2948kB shmem:16kB slab_reclaimable:164872kB slab_unreclaimable:43124kB kernel_stack:864kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:41:29 hydra01 kernel: [2091996.629011] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:41:29 hydra01 kernel: [2091996.629016] Node 0 Normal free:171200kB min:61324kB low:76652kB high:91984kB active_anon:4716364kB inactive_anon:2495944kB active_file:7240684kB inactive_file:12744280kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:352kB writeback:0kB mapped:100048kB shmem:588kB slab_reclaimable:1296376kB slab_unreclaimable:361980kB kernel_stack:7952kB pagetables:36760kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:41:29 hydra01 kernel: [2091996.629026] lowmem_reserve[]: 0 0 0 0 Sep 16 12:41:29 hydra01 kernel: [2091996.629030] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:41:29 hydra01 kernel: [2091996.629052] Node 0 DMA32: 2057*4kB (UEM) 3420*8kB (UEM) 5683*16kB (UEM) 310*32kB (UMR) 0*64kB 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 136948kB Sep 16 12:41:29 hydra01 kernel: [2091996.629070] Node 0 Normal: 32929*4kB (UEMR) 4374*8kB (UEMR) 162*16kB (UEMR) 31*32kB (UEMR) 11*64kB (EMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 172276kB Sep 16 12:41:29 hydra01 kernel: [2091996.629090] 5797925 total pagecache pages Sep 16 12:41:29 hydra01 kernel: [2091996.629093] 431637 pages in swap cache Sep 16 12:41:29 hydra01 kernel: [2091996.629096] Swap cache stats: add 20774844, delete 20343207, find 12661892/14316288 Sep 16 12:41:29 hydra01 kernel: [2091996.629098] Free swap = 185325664kB Sep 16 12:41:29 hydra01 kernel: [2091996.629100] Total swap = 187617276kB Sep 16 12:41:29 hydra01 kernel: [2091996.822541] 8388592 pages RAM Sep 16 12:41:29 hydra01 kernel: [2091996.822551] 151696 pages reserved Sep 16 12:41:29 hydra01 kernel: [2091996.822559] 8751780 pages shared Sep 16 12:41:29 hydra01 kernel: [2091996.822564] 3653379 pages non-shared Sep 16 12:42:09 hydra01 kernel: [2092036.632060] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:42:09 hydra01 kernel: [2092036.632069] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:42:09 hydra01 kernel: [2092036.632072] Call Trace: Sep 16 12:42:09 hydra01 kernel: [2092036.632086] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:42:09 hydra01 kernel: [2092036.632094] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:42:09 hydra01 kernel: [2092036.632100] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:42:09 hydra01 kernel: [2092036.632107] [] alloc_pages_current+0xb8/0x180 Sep 16 12:42:09 hydra01 kernel: [2092036.632113] [] __get_free_pages+0xe/0x50 Sep 16 12:42:09 hydra01 kernel: [2092036.632121] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:42:09 hydra01 kernel: [2092036.632126] [] __kmalloc+0x161/0x170 Sep 16 12:42:09 hydra01 kernel: [2092036.632133] [] ? get_iowait_time+0x60/0x60 Sep 16 12:42:09 hydra01 kernel: [2092036.632138] [] stat_open+0x56/0xc0 Sep 16 12:42:09 hydra01 kernel: [2092036.632144] [] proc_reg_open+0xa0/0x180 Sep 16 12:42:09 hydra01 kernel: [2092036.632152] [] ? seq_release_private+0x60/0x60 Sep 16 12:42:09 hydra01 kernel: [2092036.632157] [] do_dentry_open+0x203/0x290 Sep 16 12:42:09 hydra01 kernel: [2092036.632162] [] ? init_once+0x20/0x20 Sep 16 12:42:09 hydra01 kernel: [2092036.632166] [] vfs_open+0x49/0x50 Sep 16 12:42:09 hydra01 kernel: [2092036.632172] [] do_last+0x296/0xea0 Sep 16 12:42:09 hydra01 kernel: [2092036.632177] [] ? inode_permission+0x18/0x50 Sep 16 12:42:09 hydra01 kernel: [2092036.632181] [] ? link_path_walk+0x6b/0x880 Sep 16 12:42:09 hydra01 kernel: [2092036.632186] [] path_openat+0xb3/0x4b0 Sep 16 12:42:09 hydra01 kernel: [2092036.632191] [] do_filp_open+0x41/0xa0 Sep 16 12:42:09 hydra01 kernel: [2092036.632197] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:42:09 hydra01 kernel: [2092036.632201] [] do_sys_open+0xf3/0x230 Sep 16 12:42:09 hydra01 kernel: [2092036.632207] [] ? sys_select+0xcb/0x110 Sep 16 12:42:09 hydra01 kernel: [2092036.632211] [] sys_open+0x21/0x30 Sep 16 12:42:09 hydra01 kernel: [2092036.632216] [] system_call_fastpath+0x1a/0x1f Sep 16 12:42:09 hydra01 kernel: [2092036.632219] Mem-Info: Sep 16 12:42:09 hydra01 kernel: [2092036.632222] Node 0 DMA per-cpu: Sep 16 12:42:09 hydra01 kernel: [2092036.632226] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632229] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632232] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632235] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632238] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632240] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632243] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632246] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632248] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632251] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632254] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632257] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632259] Node 0 DMA32 per-cpu: Sep 16 12:42:09 hydra01 kernel: [2092036.632263] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632265] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632268] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632271] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632273] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632276] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632278] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632281] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632284] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632286] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632289] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632292] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632293] Node 0 Normal per-cpu: Sep 16 12:42:09 hydra01 kernel: [2092036.632297] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632300] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632302] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632305] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632307] CPU 4: hi: 186, btch: 31 usd: 167 Sep 16 12:42:09 hydra01 kernel: [2092036.632310] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632313] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632315] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632318] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632321] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632323] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632326] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632334] active_anon:1289086 inactive_anon:822733 isolated_anon:0 Sep 16 12:42:09 hydra01 kernel: [2092036.632334] active_file:1834054 inactive_file:3537227 isolated_file:0 Sep 16 12:42:09 hydra01 kernel: [2092036.632334] unevictable:0 dirty:60 writeback:25 unstable:0 Sep 16 12:42:09 hydra01 kernel: [2092036.632334] free:80507 slab_reclaimable:364014 slab_unreclaimable:100960 Sep 16 12:42:09 hydra01 kernel: [2092036.632334] mapped:26083 shmem:151 pagetables:9682 bounce:0 Sep 16 12:42:09 hydra01 kernel: [2092036.632334] free_cma:0 Sep 16 12:42:09 hydra01 kernel: [2092036.632340] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:42:09 hydra01 kernel: [2092036.632349] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:42:09 hydra01 kernel: [2092036.632355] Node 0 DMA32 free:136956kB min:6224kB low:7780kB high:9336kB active_anon:434408kB inactive_anon:793008kB active_file:95352kB inactive_file:1388792kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:16kB writeback:4kB mapped:3044kB shmem:0kB slab_reclaimable:164996kB slab_unreclaimable:42932kB kernel_stack:904kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:42:09 hydra01 kernel: [2092036.632365] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:42:09 hydra01 kernel: [2092036.632370] Node 0 Normal free:169228kB min:61324kB low:76652kB high:91984kB active_anon:4721936kB inactive_anon:2497924kB active_file:7240864kB inactive_file:12760116kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:224kB writeback:96kB mapped:101288kB shmem:604kB slab_reclaimable:1291060kB slab_unreclaimable:360908kB kernel_stack:7952kB pagetables:37200kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:42:09 hydra01 kernel: [2092036.632379] lowmem_reserve[]: 0 0 0 0 Sep 16 12:42:09 hydra01 kernel: [2092036.632384] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:42:09 hydra01 kernel: [2092036.632405] Node 0 DMA32: 1585*4kB (UEM) 3212*8kB (UEM) 5668*16kB (UEM) 432*32kB (UEMR) 6*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137444kB Sep 16 12:42:09 hydra01 kernel: [2092036.632425] Node 0 Normal: 25443*4kB (UEMR) 7779*8kB (UEMR) 148*16kB (UEMR) 60*32kB (UEMR) 9*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170148kB Sep 16 12:42:09 hydra01 kernel: [2092036.632445] 5803663 total pagecache pages Sep 16 12:42:09 hydra01 kernel: [2092036.632447] 432202 pages in swap cache Sep 16 12:42:09 hydra01 kernel: [2092036.632450] Swap cache stats: add 20775851, delete 20343649, find 12663001/14317761 Sep 16 12:42:09 hydra01 kernel: [2092036.632452] Free swap = 185326452kB Sep 16 12:42:09 hydra01 kernel: [2092036.632454] Total swap = 187617276kB Sep 16 12:42:09 hydra01 kernel: [2092036.827342] 8388592 pages RAM Sep 16 12:42:09 hydra01 kernel: [2092036.827346] 151696 pages reserved Sep 16 12:42:09 hydra01 kernel: [2092036.827349] 8487059 pages shared Sep 16 12:42:09 hydra01 kernel: [2092036.827351] 3654091 pages non-shared Sep 16 12:42:44 hydra01 kernel: [2092071.617901] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:42:44 hydra01 kernel: [2092071.617910] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:42:44 hydra01 kernel: [2092071.617913] Call Trace: Sep 16 12:42:44 hydra01 kernel: [2092071.617926] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:42:44 hydra01 kernel: [2092071.617934] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:42:44 hydra01 kernel: [2092071.617940] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:42:44 hydra01 kernel: [2092071.617948] [] alloc_pages_current+0xb8/0x180 Sep 16 12:42:44 hydra01 kernel: [2092071.617953] [] __get_free_pages+0xe/0x50 Sep 16 12:42:44 hydra01 kernel: [2092071.617960] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:42:44 hydra01 kernel: [2092071.617966] [] __kmalloc+0x161/0x170 Sep 16 12:42:44 hydra01 kernel: [2092071.617973] [] ? get_iowait_time+0x60/0x60 Sep 16 12:42:44 hydra01 kernel: [2092071.617977] [] stat_open+0x56/0xc0 Sep 16 12:42:44 hydra01 kernel: [2092071.617984] [] proc_reg_open+0xa0/0x180 Sep 16 12:42:44 hydra01 kernel: [2092071.617991] [] ? seq_release_private+0x60/0x60 Sep 16 12:42:44 hydra01 kernel: [2092071.617996] [] do_dentry_open+0x203/0x290 Sep 16 12:42:44 hydra01 kernel: [2092071.618002] [] ? init_once+0x20/0x20 Sep 16 12:42:44 hydra01 kernel: [2092071.618006] [] vfs_open+0x49/0x50 Sep 16 12:42:44 hydra01 kernel: [2092071.618011] [] do_last+0x296/0xea0 Sep 16 12:42:44 hydra01 kernel: [2092071.618016] [] ? inode_permission+0x18/0x50 Sep 16 12:42:44 hydra01 kernel: [2092071.618020] [] ? link_path_walk+0x6b/0x880 Sep 16 12:42:44 hydra01 kernel: [2092071.618025] [] path_openat+0xb3/0x4b0 Sep 16 12:42:44 hydra01 kernel: [2092071.618030] [] do_filp_open+0x41/0xa0 Sep 16 12:42:44 hydra01 kernel: [2092071.618036] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:42:44 hydra01 kernel: [2092071.618040] [] do_sys_open+0xf3/0x230 Sep 16 12:42:44 hydra01 kernel: [2092071.618048] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:42:44 hydra01 kernel: [2092071.618053] [] sys_open+0x21/0x30 Sep 16 12:42:44 hydra01 kernel: [2092071.618058] [] system_call_fastpath+0x1a/0x1f Sep 16 12:42:44 hydra01 kernel: [2092071.618061] Mem-Info: Sep 16 12:42:44 hydra01 kernel: [2092071.618063] Node 0 DMA per-cpu: Sep 16 12:42:44 hydra01 kernel: [2092071.618068] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618070] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618073] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618076] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618078] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618081] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618083] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618086] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618089] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618091] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618094] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618097] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618098] Node 0 DMA32 per-cpu: Sep 16 12:42:44 hydra01 kernel: [2092071.618102] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618105] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618108] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618111] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618113] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618116] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618118] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618121] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618124] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618126] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618129] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618132] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618133] Node 0 Normal per-cpu: Sep 16 12:42:44 hydra01 kernel: [2092071.618137] CPU 0: hi: 186, btch: 31 usd: 173 Sep 16 12:42:44 hydra01 kernel: [2092071.618140] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618142] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618145] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618148] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618150] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618153] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618155] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618158] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618161] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618163] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618166] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618174] active_anon:1286493 inactive_anon:822526 isolated_anon:0 Sep 16 12:42:44 hydra01 kernel: [2092071.618174] active_file:1832607 inactive_file:3544163 isolated_file:0 Sep 16 12:42:44 hydra01 kernel: [2092071.618174] unevictable:0 dirty:116 writeback:0 unstable:0 Sep 16 12:42:44 hydra01 kernel: [2092071.618174] free:81652 slab_reclaimable:363010 slab_unreclaimable:100940 Sep 16 12:42:44 hydra01 kernel: [2092071.618174] mapped:25883 shmem:151 pagetables:9677 bounce:0 Sep 16 12:42:44 hydra01 kernel: [2092071.618174] free_cma:0 Sep 16 12:42:44 hydra01 kernel: [2092071.618180] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:42:44 hydra01 kernel: [2092071.618189] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:42:44 hydra01 kernel: [2092071.618195] Node 0 DMA32 free:143980kB min:6224kB low:7780kB high:9336kB active_anon:434972kB inactive_anon:793008kB active_file:95488kB inactive_file:1380572kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:52kB writeback:0kB mapped:3368kB shmem:0kB slab_reclaimable:164764kB slab_unreclaimable:43092kB kernel_stack:904kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:42:44 hydra01 kernel: [2092071.618205] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:42:44 hydra01 kernel: [2092071.618210] Node 0 Normal free:167076kB min:61324kB low:76652kB high:91984kB active_anon:4711000kB inactive_anon:2497096kB active_file:7234940kB inactive_file:12796080kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:412kB writeback:0kB mapped:100164kB shmem:604kB slab_reclaimable:1287276kB slab_unreclaimable:360668kB kernel_stack:7952kB pagetables:37180kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:42:44 hydra01 kernel: [2092071.618220] lowmem_reserve[]: 0 0 0 0 Sep 16 12:42:44 hydra01 kernel: [2092071.618225] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:42:44 hydra01 kernel: [2092071.618246] Node 0 DMA32: 1507*4kB (UEM) 4167*8kB (UEM) 5664*16kB (UEM) 443*32kB (UEMR) 4*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 144932kB Sep 16 12:42:44 hydra01 kernel: [2092071.618265] Node 0 Normal: 33303*4kB (UEMR) 2767*8kB (UEMR) 663*16kB (UEMR) 24*32kB (UEMR) 9*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 168580kB Sep 16 12:42:44 hydra01 kernel: [2092071.618289] 5808990 total pagecache pages Sep 16 12:42:44 hydra01 kernel: [2092071.618291] 432153 pages in swap cache Sep 16 12:42:44 hydra01 kernel: [2092071.618325] Swap cache stats: add 20775923, delete 20343770, find 12663098/14317866 Sep 16 12:42:44 hydra01 kernel: [2092071.618328] Free swap = 185326808kB Sep 16 12:42:44 hydra01 kernel: [2092071.618330] Total swap = 187617276kB Sep 16 12:42:44 hydra01 kernel: [2092071.813310] 8388592 pages RAM Sep 16 12:42:44 hydra01 kernel: [2092071.813315] 151696 pages reserved Sep 16 12:42:44 hydra01 kernel: [2092071.813317] 8497926 pages shared Sep 16 12:42:44 hydra01 kernel: [2092071.813319] 3642514 pages non-shared Sep 16 12:42:52 hydra01 kernel: [2092079.839475] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:42:52 hydra01 kernel: [2092079.839484] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:42:52 hydra01 kernel: [2092079.839487] Call Trace: Sep 16 12:42:52 hydra01 kernel: [2092079.839500] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:42:52 hydra01 kernel: [2092079.839508] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:42:52 hydra01 kernel: [2092079.839514] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:42:52 hydra01 kernel: [2092079.839522] [] alloc_pages_current+0xb8/0x180 Sep 16 12:42:52 hydra01 kernel: [2092079.839527] [] __get_free_pages+0xe/0x50 Sep 16 12:42:52 hydra01 kernel: [2092079.839534] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:42:52 hydra01 kernel: [2092079.839540] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:42:52 hydra01 kernel: [2092079.839546] [] __kmalloc+0x161/0x170 Sep 16 12:42:52 hydra01 kernel: [2092079.839554] [] ? setkey+0x80/0x100 Sep 16 12:42:52 hydra01 kernel: [2092079.839560] [] ? get_iowait_time+0x60/0x60 Sep 16 12:42:52 hydra01 kernel: [2092079.839565] [] stat_open+0x56/0xc0 Sep 16 12:42:52 hydra01 kernel: [2092079.839571] [] proc_reg_open+0xa0/0x180 Sep 16 12:42:52 hydra01 kernel: [2092079.839578] [] ? seq_release_private+0x60/0x60 Sep 16 12:42:52 hydra01 kernel: [2092079.839584] [] do_dentry_open+0x203/0x290 Sep 16 12:42:52 hydra01 kernel: [2092079.839589] [] ? init_once+0x20/0x20 Sep 16 12:42:52 hydra01 kernel: [2092079.839593] [] vfs_open+0x49/0x50 Sep 16 12:42:52 hydra01 kernel: [2092079.839598] [] do_last+0x296/0xea0 Sep 16 12:42:52 hydra01 kernel: [2092079.839603] [] ? inode_permission+0x18/0x50 Sep 16 12:42:52 hydra01 kernel: [2092079.839607] [] ? link_path_walk+0x6b/0x880 Sep 16 12:42:52 hydra01 kernel: [2092079.839612] [] path_openat+0xb3/0x4b0 Sep 16 12:42:52 hydra01 kernel: [2092079.839619] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:42:52 hydra01 kernel: [2092079.839625] [] ? unmap_region+0xdb/0x120 Sep 16 12:42:52 hydra01 kernel: [2092079.839630] [] do_filp_open+0x41/0xa0 Sep 16 12:42:52 hydra01 kernel: [2092079.839635] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:42:52 hydra01 kernel: [2092079.839640] [] do_sys_open+0xf3/0x230 Sep 16 12:42:52 hydra01 kernel: [2092079.839645] [] sys_open+0x21/0x30 Sep 16 12:42:52 hydra01 kernel: [2092079.839650] [] system_call_fastpath+0x1a/0x1f Sep 16 12:42:52 hydra01 kernel: [2092079.839653] Mem-Info: Sep 16 12:42:52 hydra01 kernel: [2092079.839655] Node 0 DMA per-cpu: Sep 16 12:42:52 hydra01 kernel: [2092079.839660] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839663] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839666] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839669] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839671] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839674] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839676] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839679] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839682] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839684] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839687] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839689] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839691] Node 0 DMA32 per-cpu: Sep 16 12:42:52 hydra01 kernel: [2092079.839696] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839698] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839701] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839703] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839706] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839709] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839711] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839714] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839717] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839719] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839722] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839725] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839727] Node 0 Normal per-cpu: Sep 16 12:42:52 hydra01 kernel: [2092079.839730] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839733] CPU 1: hi: 186, btch: 31 usd: 167 Sep 16 12:42:52 hydra01 kernel: [2092079.839736] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839738] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839741] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839743] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839746] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839749] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839751] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839754] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839757] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839760] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839767] active_anon:1286426 inactive_anon:822495 isolated_anon:0 Sep 16 12:42:52 hydra01 kernel: [2092079.839767] active_file:1832621 inactive_file:3563527 isolated_file:0 Sep 16 12:42:52 hydra01 kernel: [2092079.839767] unevictable:0 dirty:116 writeback:0 unstable:0 Sep 16 12:42:52 hydra01 kernel: [2092079.839767] free:62788 slab_reclaimable:362848 slab_unreclaimable:100937 Sep 16 12:42:52 hydra01 kernel: [2092079.839767] mapped:26134 shmem:151 pagetables:9677 bounce:0 Sep 16 12:42:52 hydra01 kernel: [2092079.839767] free_cma:0 Sep 16 12:42:52 hydra01 kernel: [2092079.839773] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:42:52 hydra01 kernel: [2092079.839782] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:42:52 hydra01 kernel: [2092079.839788] Node 0 DMA32 free:137284kB min:6224kB low:7780kB high:9336kB active_anon:434808kB inactive_anon:793008kB active_file:95496kB inactive_file:1387540kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:36kB writeback:0kB mapped:3348kB shmem:0kB slab_reclaimable:164704kB slab_unreclaimable:43064kB kernel_stack:952kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:42:52 hydra01 kernel: [2092079.839798] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:42:52 hydra01 kernel: [2092079.839803] Node 0 Normal free:98316kB min:61324kB low:76652kB high:91984kB active_anon:4710896kB inactive_anon:2496972kB active_file:7234988kB inactive_file:12866184kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:428kB writeback:0kB mapped:101188kB shmem:604kB slab_reclaimable:1286688kB slab_unreclaimable:360684kB kernel_stack:7904kB pagetables:37180kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:42:52 hydra01 kernel: [2092079.839812] lowmem_reserve[]: 0 0 0 0 Sep 16 12:42:52 hydra01 kernel: [2092079.839817] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:42:52 hydra01 kernel: [2092079.839839] Node 0 DMA32: 1003*4kB (UEM) 3521*8kB (UEM) 5664*16kB (UEM) 442*32kB (UEMR) 4*64kB (UMR) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137844kB Sep 16 12:42:52 hydra01 kernel: [2092079.839859] Node 0 Normal: 21089*4kB (UEMR) 1588*8kB (UEMR) 16*16kB (UEMR) 28*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 100132kB Sep 16 12:42:52 hydra01 kernel: [2092079.839879] 5828358 total pagecache pages Sep 16 12:42:52 hydra01 kernel: [2092079.839881] 432155 pages in swap cache Sep 16 12:42:52 hydra01 kernel: [2092079.839884] Swap cache stats: add 20775938, delete 20343783, find 12663106/14317875 Sep 16 12:42:52 hydra01 kernel: [2092079.839886] Free swap = 185326812kB Sep 16 12:42:52 hydra01 kernel: [2092079.839888] Total swap = 187617276kB Sep 16 12:42:52 hydra01 kernel: [2092080.034499] 8388592 pages RAM Sep 16 12:42:52 hydra01 kernel: [2092080.034504] 151696 pages reserved Sep 16 12:42:52 hydra01 kernel: [2092080.034507] 8501619 pages shared Sep 16 12:42:52 hydra01 kernel: [2092080.034509] 3639317 pages non-shared Sep 16 12:43:04 hydra01 kernel: [2092091.135680] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:43:04 hydra01 kernel: [2092091.135689] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:43:04 hydra01 kernel: [2092091.135692] Call Trace: Sep 16 12:43:04 hydra01 kernel: [2092091.135705] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:43:04 hydra01 kernel: [2092091.135712] [] ? drain_local_pages+0x16/0x20 Sep 16 12:43:04 hydra01 kernel: [2092091.135720] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:43:04 hydra01 kernel: [2092091.135726] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:43:04 hydra01 kernel: [2092091.135733] [] alloc_pages_current+0xb8/0x180 Sep 16 12:43:04 hydra01 kernel: [2092091.135739] [] __get_free_pages+0xe/0x50 Sep 16 12:43:04 hydra01 kernel: [2092091.135746] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:43:04 hydra01 kernel: [2092091.135752] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:43:04 hydra01 kernel: [2092091.135757] [] __kmalloc+0x161/0x170 Sep 16 12:43:04 hydra01 kernel: [2092091.135764] [] ? get_iowait_time+0x60/0x60 Sep 16 12:43:04 hydra01 kernel: [2092091.135769] [] stat_open+0x56/0xc0 Sep 16 12:43:04 hydra01 kernel: [2092091.135775] [] proc_reg_open+0xa0/0x180 Sep 16 12:43:04 hydra01 kernel: [2092091.135782] [] ? seq_release_private+0x60/0x60 Sep 16 12:43:04 hydra01 kernel: [2092091.135787] [] do_dentry_open+0x203/0x290 Sep 16 12:43:04 hydra01 kernel: [2092091.135792] [] ? init_once+0x20/0x20 Sep 16 12:43:04 hydra01 kernel: [2092091.135796] [] vfs_open+0x49/0x50 Sep 16 12:43:04 hydra01 kernel: [2092091.135802] [] do_last+0x296/0xea0 Sep 16 12:43:04 hydra01 kernel: [2092091.135806] [] ? inode_permission+0x18/0x50 Sep 16 12:43:04 hydra01 kernel: [2092091.135811] [] ? link_path_walk+0x6b/0x880 Sep 16 12:43:04 hydra01 kernel: [2092091.135816] [] path_openat+0xb3/0x4b0 Sep 16 12:43:04 hydra01 kernel: [2092091.135822] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:43:04 hydra01 kernel: [2092091.135828] [] ? unmap_region+0xdb/0x120 Sep 16 12:43:04 hydra01 kernel: [2092091.135833] [] do_filp_open+0x41/0xa0 Sep 16 12:43:04 hydra01 kernel: [2092091.135838] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:43:04 hydra01 kernel: [2092091.135843] [] do_sys_open+0xf3/0x230 Sep 16 12:43:04 hydra01 kernel: [2092091.135848] [] sys_open+0x21/0x30 Sep 16 12:43:04 hydra01 kernel: [2092091.135854] [] system_call_fastpath+0x1a/0x1f Sep 16 12:43:04 hydra01 kernel: [2092091.135856] Mem-Info: Sep 16 12:43:04 hydra01 kernel: [2092091.135859] Node 0 DMA per-cpu: Sep 16 12:43:04 hydra01 kernel: [2092091.135864] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135866] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135869] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135872] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135874] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135877] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135879] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135882] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135885] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135887] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135890] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135893] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135895] Node 0 DMA32 per-cpu: Sep 16 12:43:04 hydra01 kernel: [2092091.135899] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135901] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135904] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135907] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135909] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135912] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135914] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135917] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135920] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135922] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135925] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135928] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135930] Node 0 Normal per-cpu: Sep 16 12:43:04 hydra01 kernel: [2092091.135933] CPU 0: hi: 186, btch: 31 usd: 164 Sep 16 12:43:04 hydra01 kernel: [2092091.135936] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135939] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135941] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135944] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135947] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135949] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135952] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135954] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135957] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135960] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135962] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.135970] active_anon:1286383 inactive_anon:822399 isolated_anon:0 Sep 16 12:43:04 hydra01 kernel: [2092091.135970] active_file:1832641 inactive_file:3550218 isolated_file:0 Sep 16 12:43:04 hydra01 kernel: [2092091.135970] unevictable:0 dirty:131 writeback:0 unstable:0 Sep 16 12:43:04 hydra01 kernel: [2092091.135970] free:77519 slab_reclaimable:362661 slab_unreclaimable:100906 Sep 16 12:43:04 hydra01 kernel: [2092091.135970] mapped:26022 shmem:151 pagetables:9553 bounce:0 Sep 16 12:43:04 hydra01 kernel: [2092091.135970] free_cma:0 Sep 16 12:43:04 hydra01 kernel: [2092091.135976] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:43:04 hydra01 kernel: [2092091.135985] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:43:04 hydra01 kernel: [2092091.135991] Node 0 DMA32 free:136528kB min:6224kB low:7780kB high:9336kB active_anon:434596kB inactive_anon:793008kB active_file:95496kB inactive_file:1388600kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:16kB writeback:0kB mapped:3348kB shmem:0kB slab_reclaimable:164664kB slab_unreclaimable:42920kB kernel_stack:952kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:04 hydra01 kernel: [2092091.136001] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:43:04 hydra01 kernel: [2092091.136006] Node 0 Normal free:157704kB min:61324kB low:76652kB high:91984kB active_anon:4710936kB inactive_anon:2496588kB active_file:7235068kB inactive_file:12812272kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:508kB writeback:0kB mapped:100740kB shmem:604kB slab_reclaimable:1285980kB slab_unreclaimable:360704kB kernel_stack:7904kB pagetables:36684kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:04 hydra01 kernel: [2092091.136015] lowmem_reserve[]: 0 0 0 0 Sep 16 12:43:04 hydra01 kernel: [2092091.136020] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:43:04 hydra01 kernel: [2092091.136042] Node 0 DMA32: 1163*4kB (UEM) 3312*8kB (UEM) 5663*16kB (UEM) 455*32kB (UEMR) 4*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137084kB Sep 16 12:43:04 hydra01 kernel: [2092091.136061] Node 0 Normal: 35779*4kB (UEMR) 1336*8kB (UEMR) 201*16kB (UEMR) 36*32kB (UEMR) 8*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 159964kB Sep 16 12:43:04 hydra01 kernel: [2092091.136081] 5815054 total pagecache pages Sep 16 12:43:04 hydra01 kernel: [2092091.136083] 432150 pages in swap cache Sep 16 12:43:04 hydra01 kernel: [2092091.136087] Swap cache stats: add 20775943, delete 20343793, find 12663108/14317877 Sep 16 12:43:04 hydra01 kernel: [2092091.136089] Free swap = 185326800kB Sep 16 12:43:04 hydra01 kernel: [2092091.136091] Total swap = 187617276kB Sep 16 12:43:04 hydra01 kernel: [2092091.325847] 8388592 pages RAM Sep 16 12:43:04 hydra01 kernel: [2092091.325859] 151696 pages reserved Sep 16 12:43:04 hydra01 kernel: [2092091.325869] 8508645 pages shared Sep 16 12:43:04 hydra01 kernel: [2092091.325878] 3637777 pages non-shared Sep 16 12:43:04 hydra01 kernel: [2092091.615127] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:43:04 hydra01 kernel: [2092091.615136] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:43:04 hydra01 kernel: [2092091.615138] Call Trace: Sep 16 12:43:04 hydra01 kernel: [2092091.615152] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:43:04 hydra01 kernel: [2092091.615159] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:43:04 hydra01 kernel: [2092091.615165] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:43:04 hydra01 kernel: [2092091.615173] [] alloc_pages_current+0xb8/0x180 Sep 16 12:43:04 hydra01 kernel: [2092091.615178] [] __get_free_pages+0xe/0x50 Sep 16 12:43:04 hydra01 kernel: [2092091.615186] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:43:04 hydra01 kernel: [2092091.615191] [] __kmalloc+0x161/0x170 Sep 16 12:43:04 hydra01 kernel: [2092091.615198] [] ? get_iowait_time+0x60/0x60 Sep 16 12:43:04 hydra01 kernel: [2092091.615202] [] stat_open+0x56/0xc0 Sep 16 12:43:04 hydra01 kernel: [2092091.615209] [] proc_reg_open+0xa0/0x180 Sep 16 12:43:04 hydra01 kernel: [2092091.615215] [] ? seq_release_private+0x60/0x60 Sep 16 12:43:04 hydra01 kernel: [2092091.615221] [] do_dentry_open+0x203/0x290 Sep 16 12:43:04 hydra01 kernel: [2092091.615226] [] ? init_once+0x20/0x20 Sep 16 12:43:04 hydra01 kernel: [2092091.615230] [] vfs_open+0x49/0x50 Sep 16 12:43:04 hydra01 kernel: [2092091.615235] [] do_last+0x296/0xea0 Sep 16 12:43:04 hydra01 kernel: [2092091.615240] [] ? inode_permission+0x18/0x50 Sep 16 12:43:04 hydra01 kernel: [2092091.615244] [] ? link_path_walk+0x6b/0x880 Sep 16 12:43:04 hydra01 kernel: [2092091.615249] [] path_openat+0xb3/0x4b0 Sep 16 12:43:04 hydra01 kernel: [2092091.615255] [] do_filp_open+0x41/0xa0 Sep 16 12:43:04 hydra01 kernel: [2092091.615260] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:43:04 hydra01 kernel: [2092091.615265] [] do_sys_open+0xf3/0x230 Sep 16 12:43:04 hydra01 kernel: [2092091.615270] [] ? sys_select+0xcb/0x110 Sep 16 12:43:04 hydra01 kernel: [2092091.615275] [] sys_open+0x21/0x30 Sep 16 12:43:04 hydra01 kernel: [2092091.615283] [] system_call_fastpath+0x1a/0x1f Sep 16 12:43:04 hydra01 kernel: [2092091.615287] Mem-Info: Sep 16 12:43:04 hydra01 kernel: [2092091.615290] Node 0 DMA per-cpu: Sep 16 12:43:04 hydra01 kernel: [2092091.615296] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615301] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615305] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615308] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615312] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615316] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615320] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615325] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615329] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615333] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615338] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615343] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615346] Node 0 DMA32 per-cpu: Sep 16 12:43:04 hydra01 kernel: [2092091.615352] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615356] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615360] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615365] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615368] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615371] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615375] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615379] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615383] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615387] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615391] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615396] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615399] Node 0 Normal per-cpu: Sep 16 12:43:04 hydra01 kernel: [2092091.615405] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615409] CPU 1: hi: 186, btch: 31 usd: 166 Sep 16 12:43:04 hydra01 kernel: [2092091.615413] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615418] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615422] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615426] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615430] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615434] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615438] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615442] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615446] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615450] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615458] active_anon:1286383 inactive_anon:822399 isolated_anon:0 Sep 16 12:43:04 hydra01 kernel: [2092091.615458] active_file:1832641 inactive_file:3552057 isolated_file:0 Sep 16 12:43:04 hydra01 kernel: [2092091.615458] unevictable:0 dirty:131 writeback:0 unstable:0 Sep 16 12:43:04 hydra01 kernel: [2092091.615458] free:76001 slab_reclaimable:362661 slab_unreclaimable:100906 Sep 16 12:43:04 hydra01 kernel: [2092091.615458] mapped:26022 shmem:151 pagetables:9553 bounce:0 Sep 16 12:43:04 hydra01 kernel: [2092091.615458] free_cma:0 Sep 16 12:43:04 hydra01 kernel: [2092091.615464] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:43:04 hydra01 kernel: [2092091.615474] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:43:04 hydra01 kernel: [2092091.615480] Node 0 DMA32 free:136528kB min:6224kB low:7780kB high:9336kB active_anon:434596kB inactive_anon:793008kB active_file:95496kB inactive_file:1388600kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:16kB writeback:0kB mapped:3348kB shmem:0kB slab_reclaimable:164664kB slab_unreclaimable:42920kB kernel_stack:952kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:04 hydra01 kernel: [2092091.615490] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:43:04 hydra01 kernel: [2092091.615495] Node 0 Normal free:151924kB min:61324kB low:76652kB high:91984kB active_anon:4710936kB inactive_anon:2496588kB active_file:7235068kB inactive_file:12819256kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:508kB writeback:0kB mapped:100740kB shmem:604kB slab_reclaimable:1285980kB slab_unreclaimable:360704kB kernel_stack:7904kB pagetables:36684kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:43:04 hydra01 kernel: [2092091.615505] lowmem_reserve[]: 0 0 0 0 Sep 16 12:43:04 hydra01 kernel: [2092091.615509] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:43:04 hydra01 kernel: [2092091.615531] Node 0 DMA32: 1163*4kB (UEM) 3312*8kB (UEM) 5663*16kB (UEM) 455*32kB (UEMR) 4*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137084kB Sep 16 12:43:04 hydra01 kernel: [2092091.615550] Node 0 Normal: 34148*4kB (UEMR) 1371*8kB (UEMR) 204*16kB (UEMR) 36*32kB (UEMR) 8*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 153768kB Sep 16 12:43:04 hydra01 kernel: [2092091.615570] 5816798 total pagecache pages Sep 16 12:43:04 hydra01 kernel: [2092091.615572] 432150 pages in swap cache Sep 16 12:43:04 hydra01 kernel: [2092091.615575] Swap cache stats: add 20775943, delete 20343793, find 12663108/14317877 Sep 16 12:43:04 hydra01 kernel: [2092091.615578] Free swap = 185326800kB Sep 16 12:43:04 hydra01 kernel: [2092091.615581] Total swap = 187617276kB Sep 16 12:43:04 hydra01 kernel: [2092091.811040] 8388592 pages RAM Sep 16 12:43:04 hydra01 kernel: [2092091.811054] 151696 pages reserved Sep 16 12:43:04 hydra01 kernel: [2092091.811064] 8502676 pages shared Sep 16 12:43:04 hydra01 kernel: [2092091.811074] 3637624 pages non-shared Sep 16 12:43:19 hydra01 kernel: [2092106.717518] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:43:19 hydra01 kernel: [2092106.717524] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:43:19 hydra01 kernel: [2092106.717525] Call Trace: Sep 16 12:43:19 hydra01 kernel: [2092106.717535] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:43:19 hydra01 kernel: [2092106.717539] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:43:19 hydra01 kernel: [2092106.717543] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:43:19 hydra01 kernel: [2092106.717547] [] alloc_pages_current+0xb8/0x180 Sep 16 12:43:19 hydra01 kernel: [2092106.717550] [] __get_free_pages+0xe/0x50 Sep 16 12:43:19 hydra01 kernel: [2092106.717554] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:43:19 hydra01 kernel: [2092106.717557] [] __kmalloc+0x161/0x170 Sep 16 12:43:19 hydra01 kernel: [2092106.717561] [] ? get_iowait_time+0x60/0x60 Sep 16 12:43:19 hydra01 kernel: [2092106.717563] [] stat_open+0x56/0xc0 Sep 16 12:43:19 hydra01 kernel: [2092106.717567] [] proc_reg_open+0xa0/0x180 Sep 16 12:43:19 hydra01 kernel: [2092106.717571] [] ? seq_release_private+0x60/0x60 Sep 16 12:43:19 hydra01 kernel: [2092106.717574] [] do_dentry_open+0x203/0x290 Sep 16 12:43:19 hydra01 kernel: [2092106.717577] [] ? init_once+0x20/0x20 Sep 16 12:43:19 hydra01 kernel: [2092106.717579] [] vfs_open+0x49/0x50 Sep 16 12:43:19 hydra01 kernel: [2092106.717582] [] do_last+0x296/0xea0 Sep 16 12:43:19 hydra01 kernel: [2092106.717585] [] ? inode_permission+0x18/0x50 Sep 16 12:43:19 hydra01 kernel: [2092106.717587] [] ? link_path_walk+0x6b/0x880 Sep 16 12:43:19 hydra01 kernel: [2092106.717590] [] path_openat+0xb3/0x4b0 Sep 16 12:43:19 hydra01 kernel: [2092106.717592] [] do_filp_open+0x41/0xa0 Sep 16 12:43:19 hydra01 kernel: [2092106.717595] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:43:19 hydra01 kernel: [2092106.717598] [] do_sys_open+0xf3/0x230 Sep 16 12:43:19 hydra01 kernel: [2092106.717601] [] ? sys_select+0xcb/0x110 Sep 16 12:43:19 hydra01 kernel: [2092106.717603] [] sys_open+0x21/0x30 Sep 16 12:43:19 hydra01 kernel: [2092106.717606] [] system_call_fastpath+0x1a/0x1f Sep 16 12:43:19 hydra01 kernel: [2092106.717607] Mem-Info: Sep 16 12:43:19 hydra01 kernel: [2092106.717609] Node 0 DMA per-cpu: Sep 16 12:43:19 hydra01 kernel: [2092106.717611] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717613] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717614] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717616] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717617] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717619] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717620] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717621] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717623] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717624] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717626] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717627] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717628] Node 0 DMA32 per-cpu: Sep 16 12:43:19 hydra01 kernel: [2092106.717630] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717632] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717633] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717635] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717636] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717637] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717639] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717640] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717642] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717643] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717644] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717646] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717647] Node 0 Normal per-cpu: Sep 16 12:43:19 hydra01 kernel: [2092106.717649] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717650] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717651] CPU 2: hi: 186, btch: 31 usd: 30 Sep 16 12:43:19 hydra01 kernel: [2092106.717653] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717654] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717656] CPU 5: hi: 186, btch: 31 usd: 155 Sep 16 12:43:19 hydra01 kernel: [2092106.717657] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717658] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717660] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717661] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717662] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717664] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717668] active_anon:1286619 inactive_anon:822320 isolated_anon:0 Sep 16 12:43:19 hydra01 kernel: [2092106.717668] active_file:1832654 inactive_file:3549478 isolated_file:0 Sep 16 12:43:19 hydra01 kernel: [2092106.717668] unevictable:0 dirty:5 writeback:0 unstable:0 Sep 16 12:43:19 hydra01 kernel: [2092106.717668] free:80405 slab_reclaimable:362032 slab_unreclaimable:100916 Sep 16 12:43:19 hydra01 kernel: [2092106.717668] mapped:26023 shmem:151 pagetables:9568 bounce:0 Sep 16 12:43:19 hydra01 kernel: [2092106.717668] free_cma:0 Sep 16 12:43:19 hydra01 kernel: [2092106.717671] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:43:19 hydra01 kernel: [2092106.717676] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:43:19 hydra01 kernel: [2092106.717679] Node 0 DMA32 free:136336kB min:6224kB low:7780kB high:9336kB active_anon:434536kB inactive_anon:793008kB active_file:95496kB inactive_file:1388700kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:104kB writeback:0kB mapped:3348kB shmem:0kB slab_reclaimable:164748kB slab_unreclaimable:43080kB kernel_stack:952kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:19 hydra01 kernel: [2092106.717684] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:43:19 hydra01 kernel: [2092106.717687] Node 0 Normal free:169440kB min:61324kB low:76652kB high:91984kB active_anon:4711940kB inactive_anon:2496272kB active_file:7235120kB inactive_file:12809212kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:0kB mapped:100744kB shmem:604kB slab_reclaimable:1283380kB slab_unreclaimable:360584kB kernel_stack:7904kB pagetables:36744kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:19 hydra01 kernel: [2092106.717692] lowmem_reserve[]: 0 0 0 0 Sep 16 12:43:19 hydra01 kernel: [2092106.717694] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:43:19 hydra01 kernel: [2092106.717706] Node 0 DMA32: 946*4kB (UEM) 3418*8kB (UEM) 5661*16kB (UEM) 446*32kB (UEMR) 4*64kB (UR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 136744kB Sep 16 12:43:19 hydra01 kernel: [2092106.717716] Node 0 Normal: 24318*4kB (UEMR) 3151*8kB (UEMR) 2077*16kB (UEMR) 338*32kB (UEMR) 35*64kB (UEMR) 3*128kB (MR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170176kB Sep 16 12:43:19 hydra01 kernel: [2092106.717726] 5814530 total pagecache pages Sep 16 12:43:19 hydra01 kernel: [2092106.717727] 432147 pages in swap cache Sep 16 12:43:19 hydra01 kernel: [2092106.717729] Swap cache stats: add 20775961, delete 20343814, find 12663108/14317877 Sep 16 12:43:19 hydra01 kernel: [2092106.717730] Free swap = 185326732kB Sep 16 12:43:19 hydra01 kernel: [2092106.717732] Total swap = 187617276kB Sep 16 12:43:19 hydra01 kernel: [2092106.824070] 8388592 pages RAM Sep 16 12:43:19 hydra01 kernel: [2092106.824078] 151696 pages reserved Sep 16 12:43:19 hydra01 kernel: [2092106.824083] 8508431 pages shared Sep 16 12:43:19 hydra01 kernel: [2092106.824088] 3633043 pages non-shared Sep 16 12:43:24 hydra01 kernel: [2092111.122890] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:43:24 hydra01 kernel: [2092111.122896] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:43:24 hydra01 kernel: [2092111.122897] Call Trace: Sep 16 12:43:24 hydra01 kernel: [2092111.122906] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:43:24 hydra01 kernel: [2092111.122911] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:43:24 hydra01 kernel: [2092111.122914] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:43:24 hydra01 kernel: [2092111.122919] [] alloc_pages_current+0xb8/0x180 Sep 16 12:43:24 hydra01 kernel: [2092111.122921] [] __get_free_pages+0xe/0x50 Sep 16 12:43:24 hydra01 kernel: [2092111.122926] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:43:24 hydra01 kernel: [2092111.122929] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:43:24 hydra01 kernel: [2092111.122932] [] __kmalloc+0x161/0x170 Sep 16 12:43:24 hydra01 kernel: [2092111.122936] [] ? get_iowait_time+0x60/0x60 Sep 16 12:43:24 hydra01 kernel: [2092111.122939] [] stat_open+0x56/0xc0 Sep 16 12:43:24 hydra01 kernel: [2092111.122942] [] proc_reg_open+0xa0/0x180 Sep 16 12:43:24 hydra01 kernel: [2092111.122947] [] ? seq_release_private+0x60/0x60 Sep 16 12:43:24 hydra01 kernel: [2092111.122950] [] do_dentry_open+0x203/0x290 Sep 16 12:43:24 hydra01 kernel: [2092111.122953] [] ? init_once+0x20/0x20 Sep 16 12:43:24 hydra01 kernel: [2092111.122955] [] vfs_open+0x49/0x50 Sep 16 12:43:24 hydra01 kernel: [2092111.122958] [] do_last+0x296/0xea0 Sep 16 12:43:24 hydra01 kernel: [2092111.122961] [] ? inode_permission+0x18/0x50 Sep 16 12:43:24 hydra01 kernel: [2092111.122963] [] ? link_path_walk+0x6b/0x880 Sep 16 12:43:24 hydra01 kernel: [2092111.122966] [] path_openat+0xb3/0x4b0 Sep 16 12:43:24 hydra01 kernel: [2092111.122969] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:43:24 hydra01 kernel: [2092111.122973] [] ? unmap_region+0xdb/0x120 Sep 16 12:43:24 hydra01 kernel: [2092111.122975] [] do_filp_open+0x41/0xa0 Sep 16 12:43:24 hydra01 kernel: [2092111.122978] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:43:24 hydra01 kernel: [2092111.122981] [] do_sys_open+0xf3/0x230 Sep 16 12:43:24 hydra01 kernel: [2092111.122983] [] sys_open+0x21/0x30 Sep 16 12:43:24 hydra01 kernel: [2092111.122986] [] system_call_fastpath+0x1a/0x1f Sep 16 12:43:24 hydra01 kernel: [2092111.122988] Mem-Info: Sep 16 12:43:24 hydra01 kernel: [2092111.122989] Node 0 DMA per-cpu: Sep 16 12:43:24 hydra01 kernel: [2092111.122992] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.122993] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.122995] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.122996] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.122997] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.122999] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123000] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123001] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123003] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123004] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123006] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123007] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123008] Node 0 DMA32 per-cpu: Sep 16 12:43:24 hydra01 kernel: [2092111.123010] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123012] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123013] CPU 2: hi: 186, btch: 31 usd: 166 Sep 16 12:43:24 hydra01 kernel: [2092111.123014] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123016] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123017] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123019] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123020] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123021] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123023] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123024] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123026] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123027] Node 0 Normal per-cpu: Sep 16 12:43:24 hydra01 kernel: [2092111.123028] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123030] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123031] CPU 2: hi: 186, btch: 31 usd: 182 Sep 16 12:43:24 hydra01 kernel: [2092111.123032] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123034] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123035] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123037] CPU 6: hi: 186, btch: 31 usd: 44 Sep 16 12:43:24 hydra01 kernel: [2092111.123038] CPU 7: hi: 186, btch: 31 usd: 98 Sep 16 12:43:24 hydra01 kernel: [2092111.123039] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123041] CPU 9: hi: 186, btch: 31 usd: 45 Sep 16 12:43:24 hydra01 kernel: [2092111.123042] CPU 10: hi: 186, btch: 31 usd: 14 Sep 16 12:43:24 hydra01 kernel: [2092111.123044] CPU 11: hi: 186, btch: 31 usd: 51 Sep 16 12:43:24 hydra01 kernel: [2092111.123048] active_anon:1286502 inactive_anon:822279 isolated_anon:0 Sep 16 12:43:24 hydra01 kernel: [2092111.123048] active_file:1832709 inactive_file:3567256 isolated_file:0 Sep 16 12:43:24 hydra01 kernel: [2092111.123048] unevictable:0 dirty:35 writeback:1090 unstable:0 Sep 16 12:43:24 hydra01 kernel: [2092111.123048] free:61174 slab_reclaimable:361876 slab_unreclaimable:100920 Sep 16 12:43:24 hydra01 kernel: [2092111.123048] mapped:26111 shmem:151 pagetables:9677 bounce:0 Sep 16 12:43:24 hydra01 kernel: [2092111.123048] free_cma:0 Sep 16 12:43:24 hydra01 kernel: [2092111.123051] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:43:24 hydra01 kernel: [2092111.123056] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:43:24 hydra01 kernel: [2092111.123059] Node 0 DMA32 free:133256kB min:6224kB low:7780kB high:9336kB active_anon:434512kB inactive_anon:793008kB active_file:95504kB inactive_file:1387008kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3348kB shmem:0kB slab_reclaimable:164696kB slab_unreclaimable:43176kB kernel_stack:952kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:24 hydra01 kernel: [2092111.123064] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:43:24 hydra01 kernel: [2092111.123067] Node 0 Normal free:95888kB min:61324kB low:76652kB high:91984kB active_anon:4711496kB inactive_anon:2496108kB active_file:7235332kB inactive_file:12882016kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:140kB writeback:4360kB mapped:101096kB shmem:604kB slab_reclaimable:1282808kB slab_unreclaimable:360504kB kernel_stack:7904kB pagetables:37180kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:24 hydra01 kernel: [2092111.123072] lowmem_reserve[]: 0 0 0 0 Sep 16 12:43:24 hydra01 kernel: [2092111.123074] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:43:24 hydra01 kernel: [2092111.123086] Node 0 DMA32: 956*4kB (UEM) 3538*8kB (UEM) 5661*16kB (UEM) 321*32kB (UMR) 1*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 133552kB Sep 16 12:43:24 hydra01 kernel: [2092111.123098] Node 0 Normal: 21875*4kB (UEMR) 608*8kB (UEMR) 56*16kB (UMR) 18*32kB (MR) 7*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 95564kB Sep 16 12:43:24 hydra01 kernel: [2092111.123114] 5832293 total pagecache pages Sep 16 12:43:24 hydra01 kernel: [2092111.123116] 432146 pages in swap cache Sep 16 12:43:24 hydra01 kernel: [2092111.123119] Swap cache stats: add 20775962, delete 20343816, find 12663108/14317877 Sep 16 12:43:24 hydra01 kernel: [2092111.123121] Free swap = 185326728kB Sep 16 12:43:24 hydra01 kernel: [2092111.123122] Total swap = 187617276kB Sep 16 12:43:24 hydra01 kernel: [2092111.232400] 8388592 pages RAM Sep 16 12:43:24 hydra01 kernel: [2092111.232402] 151696 pages reserved Sep 16 12:43:24 hydra01 kernel: [2092111.232404] 8507077 pages shared Sep 16 12:43:24 hydra01 kernel: [2092111.232405] 3634999 pages non-shared Sep 16 12:43:29 hydra01 kernel: [2092116.614140] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:43:29 hydra01 kernel: [2092116.614149] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:43:29 hydra01 kernel: [2092116.614152] Call Trace: Sep 16 12:43:29 hydra01 kernel: [2092116.614166] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:43:29 hydra01 kernel: [2092116.614172] [] ? drain_local_pages+0x16/0x20 Sep 16 12:43:29 hydra01 kernel: [2092116.614180] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:43:29 hydra01 kernel: [2092116.614186] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:43:29 hydra01 kernel: [2092116.614194] [] alloc_pages_current+0xb8/0x180 Sep 16 12:43:29 hydra01 kernel: [2092116.614199] [] __get_free_pages+0xe/0x50 Sep 16 12:43:29 hydra01 kernel: [2092116.614206] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:43:29 hydra01 kernel: [2092116.614212] [] __kmalloc+0x161/0x170 Sep 16 12:43:29 hydra01 kernel: [2092116.614218] [] ? get_iowait_time+0x60/0x60 Sep 16 12:43:29 hydra01 kernel: [2092116.614223] [] stat_open+0x56/0xc0 Sep 16 12:43:29 hydra01 kernel: [2092116.614229] [] proc_reg_open+0xa0/0x180 Sep 16 12:43:29 hydra01 kernel: [2092116.614236] [] ? seq_release_private+0x60/0x60 Sep 16 12:43:29 hydra01 kernel: [2092116.614241] [] do_dentry_open+0x203/0x290 Sep 16 12:43:29 hydra01 kernel: [2092116.614246] [] ? init_once+0x20/0x20 Sep 16 12:43:29 hydra01 kernel: [2092116.614251] [] vfs_open+0x49/0x50 Sep 16 12:43:29 hydra01 kernel: [2092116.614256] [] do_last+0x296/0xea0 Sep 16 12:43:29 hydra01 kernel: [2092116.614260] [] ? inode_permission+0x18/0x50 Sep 16 12:43:29 hydra01 kernel: [2092116.614265] [] ? link_path_walk+0x6b/0x880 Sep 16 12:43:29 hydra01 kernel: [2092116.614270] [] path_openat+0xb3/0x4b0 Sep 16 12:43:29 hydra01 kernel: [2092116.614275] [] do_filp_open+0x41/0xa0 Sep 16 12:43:29 hydra01 kernel: [2092116.614281] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:43:29 hydra01 kernel: [2092116.614285] [] do_sys_open+0xf3/0x230 Sep 16 12:43:29 hydra01 kernel: [2092116.614291] [] ? sys_select+0xcb/0x110 Sep 16 12:43:29 hydra01 kernel: [2092116.614295] [] sys_open+0x21/0x30 Sep 16 12:43:29 hydra01 kernel: [2092116.614301] [] system_call_fastpath+0x1a/0x1f Sep 16 12:43:29 hydra01 kernel: [2092116.614304] Mem-Info: Sep 16 12:43:29 hydra01 kernel: [2092116.614306] Node 0 DMA per-cpu: Sep 16 12:43:29 hydra01 kernel: [2092116.614311] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614314] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614317] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614319] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614322] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614324] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614327] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614330] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614332] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614335] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614338] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614340] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614342] Node 0 DMA32 per-cpu: Sep 16 12:43:29 hydra01 kernel: [2092116.614346] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614349] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614352] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614354] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614357] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614360] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614362] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614365] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614367] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614370] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614373] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614375] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614377] Node 0 Normal per-cpu: Sep 16 12:43:29 hydra01 kernel: [2092116.614381] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614383] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614386] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614389] CPU 3: hi: 186, btch: 31 usd: 177 Sep 16 12:43:29 hydra01 kernel: [2092116.614391] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614394] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614397] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614399] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614402] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614405] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614407] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614410] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614417] active_anon:1286641 inactive_anon:822269 isolated_anon:0 Sep 16 12:43:29 hydra01 kernel: [2092116.614417] active_file:1832712 inactive_file:3571892 isolated_file:0 Sep 16 12:43:29 hydra01 kernel: [2092116.614417] unevictable:0 dirty:37 writeback:0 unstable:0 Sep 16 12:43:29 hydra01 kernel: [2092116.614417] free:58207 slab_reclaimable:361912 slab_unreclaimable:100896 Sep 16 12:43:29 hydra01 kernel: [2092116.614417] mapped:26157 shmem:151 pagetables:9568 bounce:0 Sep 16 12:43:29 hydra01 kernel: [2092116.614417] free_cma:0 Sep 16 12:43:29 hydra01 kernel: [2092116.614424] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:43:29 hydra01 kernel: [2092116.614433] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:43:29 hydra01 kernel: [2092116.614439] Node 0 DMA32 free:132032kB min:6224kB low:7780kB high:9336kB active_anon:434476kB inactive_anon:793008kB active_file:95504kB inactive_file:1392548kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3348kB shmem:0kB slab_reclaimable:164820kB slab_unreclaimable:43080kB kernel_stack:952kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:43:29 hydra01 kernel: [2092116.614448] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:43:29 hydra01 kernel: [2092116.614453] Node 0 Normal free:84952kB min:61324kB low:76652kB high:91984kB active_anon:4712088kB inactive_anon:2496068kB active_file:7235344kB inactive_file:12895020kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:148kB writeback:0kB mapped:101280kB shmem:604kB slab_reclaimable:1282828kB slab_unreclaimable:360504kB kernel_stack:7904kB pagetables:36744kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:83 all_unreclaimable? no Sep 16 12:43:29 hydra01 kernel: [2092116.614463] lowmem_reserve[]: 0 0 0 0 Sep 16 12:43:29 hydra01 kernel: [2092116.614468] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:43:29 hydra01 kernel: [2092116.614489] Node 0 DMA32: 843*4kB (UEM) 2938*8kB (UEM) 5660*16kB (UEM) 451*32kB (UEMR) 3*64kB (UMR) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 132700kB Sep 16 12:43:29 hydra01 kernel: [2092116.614509] Node 0 Normal: 20951*4kB (UEMR) 34*8kB (UMR) 8*16kB (MR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 86060kB Sep 16 12:43:29 hydra01 kernel: [2092116.614530] 5836850 total pagecache pages Sep 16 12:43:29 hydra01 kernel: [2092116.614532] 432146 pages in swap cache Sep 16 12:43:29 hydra01 kernel: [2092116.614535] Swap cache stats: add 20775962, delete 20343816, find 12663108/14317877 Sep 16 12:43:29 hydra01 kernel: [2092116.614538] Free swap = 185326728kB Sep 16 12:43:29 hydra01 kernel: [2092116.614540] Total swap = 187617276kB Sep 16 12:43:29 hydra01 kernel: [2092116.808641] 8388592 pages RAM Sep 16 12:43:29 hydra01 kernel: [2092116.808656] 151696 pages reserved Sep 16 12:43:29 hydra01 kernel: [2092116.808659] 8510979 pages shared Sep 16 12:43:29 hydra01 kernel: [2092116.808662] 3632816 pages non-shared Sep 16 12:44:24 hydra01 kernel: [2092171.602156] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:44:24 hydra01 kernel: [2092171.602165] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:44:24 hydra01 kernel: [2092171.602168] Call Trace: Sep 16 12:44:24 hydra01 kernel: [2092171.602182] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:44:24 hydra01 kernel: [2092171.602190] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:44:24 hydra01 kernel: [2092171.602196] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:44:24 hydra01 kernel: [2092171.602204] [] alloc_pages_current+0xb8/0x180 Sep 16 12:44:24 hydra01 kernel: [2092171.602209] [] __get_free_pages+0xe/0x50 Sep 16 12:44:24 hydra01 kernel: [2092171.602216] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:44:24 hydra01 kernel: [2092171.602222] [] __kmalloc+0x161/0x170 Sep 16 12:44:24 hydra01 kernel: [2092171.602228] [] ? get_iowait_time+0x60/0x60 Sep 16 12:44:24 hydra01 kernel: [2092171.602233] [] stat_open+0x56/0xc0 Sep 16 12:44:24 hydra01 kernel: [2092171.602239] [] proc_reg_open+0xa0/0x180 Sep 16 12:44:24 hydra01 kernel: [2092171.602246] [] ? seq_release_private+0x60/0x60 Sep 16 12:44:24 hydra01 kernel: [2092171.602252] [] do_dentry_open+0x203/0x290 Sep 16 12:44:24 hydra01 kernel: [2092171.602257] [] ? init_once+0x20/0x20 Sep 16 12:44:24 hydra01 kernel: [2092171.602261] [] vfs_open+0x49/0x50 Sep 16 12:44:24 hydra01 kernel: [2092171.602266] [] do_last+0x296/0xea0 Sep 16 12:44:24 hydra01 kernel: [2092171.602271] [] ? inode_permission+0x18/0x50 Sep 16 12:44:24 hydra01 kernel: [2092171.602275] [] ? link_path_walk+0x6b/0x880 Sep 16 12:44:24 hydra01 kernel: [2092171.602282] [] ? touch_atime+0x81/0x160 Sep 16 12:44:24 hydra01 kernel: [2092171.602287] [] path_openat+0xb3/0x4b0 Sep 16 12:44:24 hydra01 kernel: [2092171.602292] [] do_filp_open+0x41/0xa0 Sep 16 12:44:24 hydra01 kernel: [2092171.602297] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:44:24 hydra01 kernel: [2092171.602302] [] do_sys_open+0xf3/0x230 Sep 16 12:44:24 hydra01 kernel: [2092171.602307] [] ? sys_select+0xcb/0x110 Sep 16 12:44:24 hydra01 kernel: [2092171.602312] [] sys_open+0x21/0x30 Sep 16 12:44:24 hydra01 kernel: [2092171.602317] [] system_call_fastpath+0x1a/0x1f Sep 16 12:44:24 hydra01 kernel: [2092171.602320] Mem-Info: Sep 16 12:44:24 hydra01 kernel: [2092171.602322] Node 0 DMA per-cpu: Sep 16 12:44:24 hydra01 kernel: [2092171.602327] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602330] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602332] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602335] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602337] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602340] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602342] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602345] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602348] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602350] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602374] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602377] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602379] Node 0 DMA32 per-cpu: Sep 16 12:44:24 hydra01 kernel: [2092171.602383] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602386] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602388] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602391] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602393] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602396] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602399] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602401] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602404] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602406] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602409] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602412] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602414] Node 0 Normal per-cpu: Sep 16 12:44:24 hydra01 kernel: [2092171.602417] CPU 0: hi: 186, btch: 31 usd: 186 Sep 16 12:44:24 hydra01 kernel: [2092171.602420] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602423] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602426] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602428] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602431] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602433] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602436] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602439] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602441] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602444] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602447] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602455] active_anon:1287299 inactive_anon:821534 isolated_anon:0 Sep 16 12:44:24 hydra01 kernel: [2092171.602455] active_file:1832737 inactive_file:3567788 isolated_file:0 Sep 16 12:44:24 hydra01 kernel: [2092171.602455] unevictable:0 dirty:1234 writeback:0 unstable:0 Sep 16 12:44:24 hydra01 kernel: [2092171.602455] free:68764 slab_reclaimable:360921 slab_unreclaimable:100758 Sep 16 12:44:24 hydra01 kernel: [2092171.602455] mapped:26062 shmem:151 pagetables:9663 bounce:0 Sep 16 12:44:24 hydra01 kernel: [2092171.602455] free_cma:0 Sep 16 12:44:24 hydra01 kernel: [2092171.602461] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:44:24 hydra01 kernel: [2092171.602471] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:44:24 hydra01 kernel: [2092171.602477] Node 0 DMA32 free:138592kB min:6224kB low:7780kB high:9336kB active_anon:435848kB inactive_anon:792996kB active_file:95576kB inactive_file:1381740kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:3052kB shmem:0kB slab_reclaimable:164816kB slab_unreclaimable:42996kB kernel_stack:1008kB pagetables:1520kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:44:24 hydra01 kernel: [2092171.602486] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:44:24 hydra01 kernel: [2092171.602492] Node 0 Normal free:120912kB min:61324kB low:76652kB high:91984kB active_anon:4713348kB inactive_anon:2493140kB active_file:7235372kB inactive_file:12889028kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:4936kB writeback:0kB mapped:101196kB shmem:604kB slab_reclaimable:1278868kB slab_unreclaimable:360036kB kernel_stack:7864kB pagetables:37132kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:44:24 hydra01 kernel: [2092171.602501] lowmem_reserve[]: 0 0 0 0 Sep 16 12:44:24 hydra01 kernel: [2092171.602506] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:44:24 hydra01 kernel: [2092171.602527] Node 0 DMA32: 1463*4kB (UM) 3782*8kB (UEM) 5666*16kB (UEM) 379*32kB (UMR) 1*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139468kB Sep 16 12:44:24 hydra01 kernel: [2092171.602556] Node 0 Normal: 26568*4kB (UEMR) 580*8kB (UEMR) 520*16kB (UEMR) 42*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 122240kB Sep 16 12:44:24 hydra01 kernel: [2092171.602576] 5832623 total pagecache pages Sep 16 12:44:24 hydra01 kernel: [2092171.602578] 432086 pages in swap cache Sep 16 12:44:24 hydra01 kernel: [2092171.602581] Swap cache stats: add 20776433, delete 20344347, find 12663160/14317941 Sep 16 12:44:24 hydra01 kernel: [2092171.602584] Free swap = 185325252kB Sep 16 12:44:24 hydra01 kernel: [2092171.602586] Total swap = 187617276kB Sep 16 12:44:24 hydra01 kernel: [2092171.789453] 8388592 pages RAM Sep 16 12:44:24 hydra01 kernel: [2092171.789463] 151696 pages reserved Sep 16 12:44:24 hydra01 kernel: [2092171.789465] 8559480 pages shared Sep 16 12:44:24 hydra01 kernel: [2092171.789467] 3581434 pages non-shared Sep 16 12:44:34 hydra01 kernel: [2092181.075762] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:44:34 hydra01 kernel: [2092181.075770] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:44:34 hydra01 kernel: [2092181.075773] Call Trace: Sep 16 12:44:34 hydra01 kernel: [2092181.075787] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:44:34 hydra01 kernel: [2092181.075795] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:44:34 hydra01 kernel: [2092181.075801] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:44:34 hydra01 kernel: [2092181.075808] [] alloc_pages_current+0xb8/0x180 Sep 16 12:44:34 hydra01 kernel: [2092181.075814] [] __get_free_pages+0xe/0x50 Sep 16 12:44:34 hydra01 kernel: [2092181.075821] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:44:34 hydra01 kernel: [2092181.075827] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:44:34 hydra01 kernel: [2092181.075833] [] __kmalloc+0x161/0x170 Sep 16 12:44:34 hydra01 kernel: [2092181.075839] [] ? get_iowait_time+0x60/0x60 Sep 16 12:44:34 hydra01 kernel: [2092181.075844] [] stat_open+0x56/0xc0 Sep 16 12:44:34 hydra01 kernel: [2092181.075850] [] proc_reg_open+0xa0/0x180 Sep 16 12:44:34 hydra01 kernel: [2092181.075857] [] ? seq_release_private+0x60/0x60 Sep 16 12:44:34 hydra01 kernel: [2092181.075863] [] do_dentry_open+0x203/0x290 Sep 16 12:44:34 hydra01 kernel: [2092181.075868] [] ? init_once+0x20/0x20 Sep 16 12:44:34 hydra01 kernel: [2092181.075872] [] vfs_open+0x49/0x50 Sep 16 12:44:34 hydra01 kernel: [2092181.075877] [] do_last+0x296/0xea0 Sep 16 12:44:34 hydra01 kernel: [2092181.075882] [] ? inode_permission+0x18/0x50 Sep 16 12:44:34 hydra01 kernel: [2092181.075886] [] ? link_path_walk+0x6b/0x880 Sep 16 12:44:34 hydra01 kernel: [2092181.075891] [] path_openat+0xb3/0x4b0 Sep 16 12:44:34 hydra01 kernel: [2092181.075897] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:44:34 hydra01 kernel: [2092181.075903] [] ? unmap_region+0xdb/0x120 Sep 16 12:44:34 hydra01 kernel: [2092181.075908] [] do_filp_open+0x41/0xa0 Sep 16 12:44:34 hydra01 kernel: [2092181.075913] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:44:34 hydra01 kernel: [2092181.075918] [] do_sys_open+0xf3/0x230 Sep 16 12:44:34 hydra01 kernel: [2092181.075922] [] sys_open+0x21/0x30 Sep 16 12:44:34 hydra01 kernel: [2092181.075928] [] system_call_fastpath+0x1a/0x1f Sep 16 12:44:34 hydra01 kernel: [2092181.075930] Mem-Info: Sep 16 12:44:34 hydra01 kernel: [2092181.075933] Node 0 DMA per-cpu: Sep 16 12:44:34 hydra01 kernel: [2092181.075938] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075940] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075943] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075945] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075948] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075951] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075953] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075956] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075958] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075961] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075964] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075966] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075968] Node 0 DMA32 per-cpu: Sep 16 12:44:34 hydra01 kernel: [2092181.075972] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075975] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075978] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075980] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075983] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075986] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075988] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075991] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075993] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075996] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.075999] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076001] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076003] Node 0 Normal per-cpu: Sep 16 12:44:34 hydra01 kernel: [2092181.076007] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076010] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076012] CPU 2: hi: 186, btch: 31 usd: 170 Sep 16 12:44:34 hydra01 kernel: [2092181.076015] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076018] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076020] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076023] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076025] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076028] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076031] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076033] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076036] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076043] active_anon:1287064 inactive_anon:821465 isolated_anon:0 Sep 16 12:44:34 hydra01 kernel: [2092181.076043] active_file:1832742 inactive_file:3576559 isolated_file:0 Sep 16 12:44:34 hydra01 kernel: [2092181.076043] unevictable:0 dirty:97 writeback:0 unstable:0 Sep 16 12:44:34 hydra01 kernel: [2092181.076043] free:61106 slab_reclaimable:361084 slab_unreclaimable:100730 Sep 16 12:44:34 hydra01 kernel: [2092181.076043] mapped:25627 shmem:151 pagetables:9609 bounce:0 Sep 16 12:44:34 hydra01 kernel: [2092181.076043] free_cma:0 Sep 16 12:44:34 hydra01 kernel: [2092181.076050] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:44:34 hydra01 kernel: [2092181.076059] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:44:34 hydra01 kernel: [2092181.076065] Node 0 DMA32 free:138532kB min:6224kB low:7780kB high:9336kB active_anon:435568kB inactive_anon:792996kB active_file:95576kB inactive_file:1384892kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:16kB writeback:0kB mapped:2944kB shmem:0kB slab_reclaimable:164920kB slab_unreclaimable:42884kB kernel_stack:1056kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:44:34 hydra01 kernel: [2092181.076074] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:44:34 hydra01 kernel: [2092181.076079] Node 0 Normal free:90048kB min:61324kB low:76652kB high:91984kB active_anon:4712688kB inactive_anon:2492864kB active_file:7235392kB inactive_file:12921344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:372kB writeback:0kB mapped:99564kB shmem:604kB slab_reclaimable:1279416kB slab_unreclaimable:360036kB kernel_stack:7800kB pagetables:36908kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:68 all_unreclaimable? no Sep 16 12:44:34 hydra01 kernel: [2092181.076089] lowmem_reserve[]: 0 0 0 0 Sep 16 12:44:34 hydra01 kernel: [2092181.076094] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:44:34 hydra01 kernel: [2092181.076115] Node 0 DMA32: 1091*4kB (UEM) 3576*8kB (UEM) 5667*16kB (UEM) 460*32kB (UEMR) 5*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139196kB Sep 16 12:44:34 hydra01 kernel: [2092181.076135] Node 0 Normal: 21337*4kB (UEMR) 255*8kB (UEMR) 107*16kB (UEMR) 25*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 91564kB Sep 16 12:44:34 hydra01 kernel: [2092181.076154] 5841535 total pagecache pages Sep 16 12:44:34 hydra01 kernel: [2092181.076157] 432081 pages in swap cache Sep 16 12:44:34 hydra01 kernel: [2092181.076160] Swap cache stats: add 20776500, delete 20344419, find 12663160/14317941 Sep 16 12:44:34 hydra01 kernel: [2092181.076162] Free swap = 185324992kB Sep 16 12:44:34 hydra01 kernel: [2092181.076164] Total swap = 187617276kB Sep 16 12:44:34 hydra01 kernel: [2092181.270031] 8388592 pages RAM Sep 16 12:44:34 hydra01 kernel: [2092181.270036] 151696 pages reserved Sep 16 12:44:34 hydra01 kernel: [2092181.270039] 8578164 pages shared Sep 16 12:44:34 hydra01 kernel: [2092181.270041] 3562363 pages non-shared Sep 16 12:44:34 hydra01 kernel: [2092181.605595] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:44:34 hydra01 kernel: [2092181.605604] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:44:34 hydra01 kernel: [2092181.605606] Call Trace: Sep 16 12:44:34 hydra01 kernel: [2092181.605620] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:44:34 hydra01 kernel: [2092181.605627] [] ? drain_local_pages+0x16/0x20 Sep 16 12:44:34 hydra01 kernel: [2092181.605634] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:44:34 hydra01 kernel: [2092181.605640] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:44:34 hydra01 kernel: [2092181.605648] [] alloc_pages_current+0xb8/0x180 Sep 16 12:44:34 hydra01 kernel: [2092181.605653] [] __get_free_pages+0xe/0x50 Sep 16 12:44:34 hydra01 kernel: [2092181.605660] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:44:34 hydra01 kernel: [2092181.605666] [] __kmalloc+0x161/0x170 Sep 16 12:44:34 hydra01 kernel: [2092181.605673] [] ? get_iowait_time+0x60/0x60 Sep 16 12:44:34 hydra01 kernel: [2092181.605677] [] stat_open+0x56/0xc0 Sep 16 12:44:34 hydra01 kernel: [2092181.605684] [] proc_reg_open+0xa0/0x180 Sep 16 12:44:34 hydra01 kernel: [2092181.605691] [] ? seq_release_private+0x60/0x60 Sep 16 12:44:34 hydra01 kernel: [2092181.605696] [] do_dentry_open+0x203/0x290 Sep 16 12:44:34 hydra01 kernel: [2092181.605701] [] ? init_once+0x20/0x20 Sep 16 12:44:34 hydra01 kernel: [2092181.605705] [] vfs_open+0x49/0x50 Sep 16 12:44:34 hydra01 kernel: [2092181.605711] [] do_last+0x296/0xea0 Sep 16 12:44:34 hydra01 kernel: [2092181.605715] [] ? inode_permission+0x18/0x50 Sep 16 12:44:34 hydra01 kernel: [2092181.605720] [] ? link_path_walk+0x6b/0x880 Sep 16 12:44:34 hydra01 kernel: [2092181.605725] [] path_openat+0xb3/0x4b0 Sep 16 12:44:34 hydra01 kernel: [2092181.605730] [] do_filp_open+0x41/0xa0 Sep 16 12:44:34 hydra01 kernel: [2092181.605736] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:44:34 hydra01 kernel: [2092181.605741] [] do_sys_open+0xf3/0x230 Sep 16 12:44:34 hydra01 kernel: [2092181.605749] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:44:34 hydra01 kernel: [2092181.605753] [] sys_open+0x21/0x30 Sep 16 12:44:34 hydra01 kernel: [2092181.605759] [] system_call_fastpath+0x1a/0x1f Sep 16 12:44:34 hydra01 kernel: [2092181.605762] Mem-Info: Sep 16 12:44:34 hydra01 kernel: [2092181.605764] Node 0 DMA per-cpu: Sep 16 12:44:34 hydra01 kernel: [2092181.605769] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605772] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605774] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605777] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605780] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605782] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605785] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605787] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605790] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605793] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605795] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605798] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605800] Node 0 DMA32 per-cpu: Sep 16 12:44:34 hydra01 kernel: [2092181.605804] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605807] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605809] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605812] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605815] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605817] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605820] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605822] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605825] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605828] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605830] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605833] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605835] Node 0 Normal per-cpu: Sep 16 12:44:34 hydra01 kernel: [2092181.605839] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605841] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605844] CPU 2: hi: 186, btch: 31 usd: 162 Sep 16 12:44:34 hydra01 kernel: [2092181.605847] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605849] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605852] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605855] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605857] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605860] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605862] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605865] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605868] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605875] active_anon:1286991 inactive_anon:821465 isolated_anon:0 Sep 16 12:44:34 hydra01 kernel: [2092181.605875] active_file:1832742 inactive_file:3558110 isolated_file:0 Sep 16 12:44:34 hydra01 kernel: [2092181.605875] unevictable:0 dirty:206 writeback:0 unstable:0 Sep 16 12:44:34 hydra01 kernel: [2092181.605875] free:80430 slab_reclaimable:360929 slab_unreclaimable:100730 Sep 16 12:44:34 hydra01 kernel: [2092181.605875] mapped:25627 shmem:151 pagetables:9609 bounce:0 Sep 16 12:44:34 hydra01 kernel: [2092181.605875] free_cma:0 Sep 16 12:44:34 hydra01 kernel: [2092181.605882] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:44:34 hydra01 kernel: [2092181.605890] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:44:34 hydra01 kernel: [2092181.605897] Node 0 DMA32 free:138540kB min:6224kB low:7780kB high:9336kB active_anon:435568kB inactive_anon:792996kB active_file:95576kB inactive_file:1384892kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:16kB writeback:0kB mapped:2944kB shmem:0kB slab_reclaimable:164920kB slab_unreclaimable:42884kB kernel_stack:1056kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:44:34 hydra01 kernel: [2092181.605907] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:44:34 hydra01 kernel: [2092181.605912] Node 0 Normal free:167336kB min:61324kB low:76652kB high:91984kB active_anon:4712396kB inactive_anon:2492864kB active_file:7235392kB inactive_file:12847164kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:808kB writeback:0kB mapped:99564kB shmem:604kB slab_reclaimable:1278796kB slab_unreclaimable:360036kB kernel_stack:7800kB pagetables:36908kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:44:34 hydra01 kernel: [2092181.605921] lowmem_reserve[]: 0 0 0 0 Sep 16 12:44:34 hydra01 kernel: [2092181.605927] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:44:34 hydra01 kernel: [2092181.605948] Node 0 DMA32: 1092*4kB (UEM) 3577*8kB (UEM) 5670*16kB (UEM) 461*32kB (UEMR) 5*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139288kB Sep 16 12:44:34 hydra01 kernel: [2092181.605967] Node 0 Normal: 39982*4kB (UEMR) 520*8kB (UEMR) 112*16kB (UEMR) 52*32kB (UEMR) 8*64kB (UR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 169336kB Sep 16 12:44:34 hydra01 kernel: [2092181.605987] 5822896 total pagecache pages Sep 16 12:44:34 hydra01 kernel: [2092181.605989] 432081 pages in swap cache Sep 16 12:44:34 hydra01 kernel: [2092181.605993] Swap cache stats: add 20776500, delete 20344419, find 12663160/14317941 Sep 16 12:44:34 hydra01 kernel: [2092181.605995] Free swap = 185324992kB Sep 16 12:44:34 hydra01 kernel: [2092181.605997] Total swap = 187617276kB Sep 16 12:44:34 hydra01 kernel: [2092181.799249] 8388592 pages RAM Sep 16 12:44:34 hydra01 kernel: [2092181.799260] 151696 pages reserved Sep 16 12:44:34 hydra01 kernel: [2092181.799270] 8577703 pages shared Sep 16 12:44:34 hydra01 kernel: [2092181.799280] 3562227 pages non-shared Sep 16 12:46:34 hydra01 kernel: [2092300.996984] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:46:34 hydra01 kernel: [2092300.996993] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:46:34 hydra01 kernel: [2092300.996995] Call Trace: Sep 16 12:46:34 hydra01 kernel: [2092300.997009] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:46:34 hydra01 kernel: [2092300.997017] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:46:34 hydra01 kernel: [2092300.997024] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:46:34 hydra01 kernel: [2092300.997031] [] alloc_pages_current+0xb8/0x180 Sep 16 12:46:34 hydra01 kernel: [2092300.997037] [] __get_free_pages+0xe/0x50 Sep 16 12:46:34 hydra01 kernel: [2092300.997044] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:46:34 hydra01 kernel: [2092300.997050] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:46:34 hydra01 kernel: [2092300.997055] [] __kmalloc+0x161/0x170 Sep 16 12:46:34 hydra01 kernel: [2092300.997062] [] ? get_iowait_time+0x60/0x60 Sep 16 12:46:34 hydra01 kernel: [2092300.997066] [] stat_open+0x56/0xc0 Sep 16 12:46:34 hydra01 kernel: [2092300.997073] [] proc_reg_open+0xa0/0x180 Sep 16 12:46:34 hydra01 kernel: [2092300.997080] [] ? seq_release_private+0x60/0x60 Sep 16 12:46:34 hydra01 kernel: [2092300.997085] [] do_dentry_open+0x203/0x290 Sep 16 12:46:34 hydra01 kernel: [2092300.997091] [] ? init_once+0x20/0x20 Sep 16 12:46:34 hydra01 kernel: [2092300.997095] [] vfs_open+0x49/0x50 Sep 16 12:46:34 hydra01 kernel: [2092300.997100] [] do_last+0x296/0xea0 Sep 16 12:46:34 hydra01 kernel: [2092300.997105] [] ? inode_permission+0x18/0x50 Sep 16 12:46:34 hydra01 kernel: [2092300.997109] [] ? link_path_walk+0x6b/0x880 Sep 16 12:46:34 hydra01 kernel: [2092300.997114] [] path_openat+0xb3/0x4b0 Sep 16 12:46:34 hydra01 kernel: [2092300.997120] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:46:34 hydra01 kernel: [2092300.997127] [] ? unmap_region+0xdb/0x120 Sep 16 12:46:34 hydra01 kernel: [2092300.997132] [] do_filp_open+0x41/0xa0 Sep 16 12:46:34 hydra01 kernel: [2092300.997137] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:46:34 hydra01 kernel: [2092300.997142] [] do_sys_open+0xf3/0x230 Sep 16 12:46:34 hydra01 kernel: [2092300.997146] [] sys_open+0x21/0x30 Sep 16 12:46:34 hydra01 kernel: [2092300.997152] [] system_call_fastpath+0x1a/0x1f Sep 16 12:46:34 hydra01 kernel: [2092300.997155] Mem-Info: Sep 16 12:46:34 hydra01 kernel: [2092300.997157] Node 0 DMA per-cpu: Sep 16 12:46:34 hydra01 kernel: [2092300.997162] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997164] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997167] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997170] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997172] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997175] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997177] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997180] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997183] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997185] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997188] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997191] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997193] Node 0 DMA32 per-cpu: Sep 16 12:46:34 hydra01 kernel: [2092300.997197] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997199] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997202] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997204] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997207] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997210] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997212] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997215] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997218] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997220] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997223] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997225] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997227] Node 0 Normal per-cpu: Sep 16 12:46:34 hydra01 kernel: [2092300.997231] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997234] CPU 1: hi: 186, btch: 31 usd: 180 Sep 16 12:46:34 hydra01 kernel: [2092300.997236] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997239] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997242] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997244] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997247] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997250] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997252] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997255] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997258] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997260] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997268] active_anon:1283426 inactive_anon:820232 isolated_anon:0 Sep 16 12:46:34 hydra01 kernel: [2092300.997268] active_file:1832887 inactive_file:3586618 isolated_file:0 Sep 16 12:46:34 hydra01 kernel: [2092300.997268] unevictable:0 dirty:28 writeback:0 unstable:0 Sep 16 12:46:34 hydra01 kernel: [2092300.997268] free:62485 slab_reclaimable:364563 slab_unreclaimable:100627 Sep 16 12:46:34 hydra01 kernel: [2092300.997268] mapped:25810 shmem:151 pagetables:9559 bounce:0 Sep 16 12:46:34 hydra01 kernel: [2092300.997268] free_cma:0 Sep 16 12:46:34 hydra01 kernel: [2092300.997274] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:46:34 hydra01 kernel: [2092300.997283] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:46:34 hydra01 kernel: [2092300.997289] Node 0 DMA32 free:141264kB min:6224kB low:7780kB high:9336kB active_anon:435780kB inactive_anon:792996kB active_file:95648kB inactive_file:1383012kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3200kB shmem:0kB slab_reclaimable:165356kB slab_unreclaimable:42712kB kernel_stack:1072kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:46:34 hydra01 kernel: [2092300.997299] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:46:34 hydra01 kernel: [2092300.997304] Node 0 Normal free:93124kB min:61324kB low:76652kB high:91984kB active_anon:4697924kB inactive_anon:2487932kB active_file:7235900kB inactive_file:12963460kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:104kB writeback:0kB mapped:100040kB shmem:604kB slab_reclaimable:1292896kB slab_unreclaimable:359796kB kernel_stack:7784kB pagetables:36708kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:46:34 hydra01 kernel: [2092300.997314] lowmem_reserve[]: 0 0 0 0 Sep 16 12:46:34 hydra01 kernel: [2092300.997319] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:46:34 hydra01 kernel: [2092300.997340] Node 0 DMA32: 1263*4kB (UEM) 3772*8kB (UEM) 5687*16kB (UEM) 458*32kB (UEMR) 6*64kB (UMR) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141900kB Sep 16 12:46:34 hydra01 kernel: [2092300.997360] Node 0 Normal: 19647*4kB (UEMR) 1038*8kB (UEMR) 22*16kB (UEMR) 60*32kB (EMR) 76*64kB (MR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 95308kB Sep 16 12:46:34 hydra01 kernel: [2092300.997381] 5851410 total pagecache pages Sep 16 12:46:34 hydra01 kernel: [2092300.997383] 431998 pages in swap cache Sep 16 12:46:34 hydra01 kernel: [2092300.997386] Swap cache stats: add 20777353, delete 20345355, find 12663172/14317953 Sep 16 12:46:34 hydra01 kernel: [2092300.997388] Free swap = 185321928kB Sep 16 12:46:34 hydra01 kernel: [2092300.997390] Total swap = 187617276kB Sep 16 12:46:34 hydra01 kernel: [2092301.183140] 8388592 pages RAM Sep 16 12:46:34 hydra01 kernel: [2092301.183145] 151696 pages reserved Sep 16 12:46:34 hydra01 kernel: [2092301.183147] 8900257 pages shared Sep 16 12:46:34 hydra01 kernel: [2092301.183149] 3250473 pages non-shared Sep 16 12:46:34 hydra01 kernel: [2092301.583962] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:46:34 hydra01 kernel: [2092301.583971] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:46:34 hydra01 kernel: [2092301.583973] Call Trace: Sep 16 12:46:34 hydra01 kernel: [2092301.583987] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:46:34 hydra01 kernel: [2092301.583996] [] ? drain_local_pages+0x16/0x20 Sep 16 12:46:34 hydra01 kernel: [2092301.584004] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 12:46:34 hydra01 kernel: [2092301.584010] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:46:34 hydra01 kernel: [2092301.584018] [] alloc_pages_current+0xb8/0x180 Sep 16 12:46:34 hydra01 kernel: [2092301.584023] [] __get_free_pages+0xe/0x50 Sep 16 12:46:34 hydra01 kernel: [2092301.584030] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:46:34 hydra01 kernel: [2092301.584036] [] __kmalloc+0x161/0x170 Sep 16 12:46:34 hydra01 kernel: [2092301.584042] [] ? get_iowait_time+0x60/0x60 Sep 16 12:46:34 hydra01 kernel: [2092301.584047] [] stat_open+0x56/0xc0 Sep 16 12:46:34 hydra01 kernel: [2092301.584053] [] proc_reg_open+0xa0/0x180 Sep 16 12:46:34 hydra01 kernel: [2092301.584060] [] ? seq_release_private+0x60/0x60 Sep 16 12:46:34 hydra01 kernel: [2092301.584065] [] do_dentry_open+0x203/0x290 Sep 16 12:46:34 hydra01 kernel: [2092301.584070] [] ? init_once+0x20/0x20 Sep 16 12:46:34 hydra01 kernel: [2092301.584075] [] vfs_open+0x49/0x50 Sep 16 12:46:34 hydra01 kernel: [2092301.584080] [] do_last+0x296/0xea0 Sep 16 12:46:34 hydra01 kernel: [2092301.584084] [] ? inode_permission+0x18/0x50 Sep 16 12:46:34 hydra01 kernel: [2092301.584089] [] ? link_path_walk+0x6b/0x880 Sep 16 12:46:34 hydra01 kernel: [2092301.584094] [] path_openat+0xb3/0x4b0 Sep 16 12:46:34 hydra01 kernel: [2092301.584099] [] do_filp_open+0x41/0xa0 Sep 16 12:46:34 hydra01 kernel: [2092301.584105] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:46:34 hydra01 kernel: [2092301.584110] [] do_sys_open+0xf3/0x230 Sep 16 12:46:34 hydra01 kernel: [2092301.584118] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 12:46:34 hydra01 kernel: [2092301.584122] [] sys_open+0x21/0x30 Sep 16 12:46:34 hydra01 kernel: [2092301.584128] [] system_call_fastpath+0x1a/0x1f Sep 16 12:46:34 hydra01 kernel: [2092301.584131] Mem-Info: Sep 16 12:46:34 hydra01 kernel: [2092301.584133] Node 0 DMA per-cpu: Sep 16 12:46:34 hydra01 kernel: [2092301.584138] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584141] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584144] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584146] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584149] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584151] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584154] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584157] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584159] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584162] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584164] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584167] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584169] Node 0 DMA32 per-cpu: Sep 16 12:46:34 hydra01 kernel: [2092301.584173] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584176] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584178] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584181] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584183] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584186] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584189] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584191] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584194] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584197] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584199] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584202] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584204] Node 0 Normal per-cpu: Sep 16 12:46:34 hydra01 kernel: [2092301.584208] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584211] CPU 1: hi: 186, btch: 31 usd: 186 Sep 16 12:46:34 hydra01 kernel: [2092301.584213] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584216] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584218] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584221] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584224] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584226] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584229] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584232] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584234] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584237] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584244] active_anon:1283645 inactive_anon:820232 isolated_anon:0 Sep 16 12:46:34 hydra01 kernel: [2092301.584244] active_file:1832887 inactive_file:3581568 isolated_file:0 Sep 16 12:46:34 hydra01 kernel: [2092301.584244] unevictable:0 dirty:0 writeback:0 unstable:0 Sep 16 12:46:34 hydra01 kernel: [2092301.584244] free:67718 slab_reclaimable:364563 slab_unreclaimable:100627 Sep 16 12:46:34 hydra01 kernel: [2092301.584244] mapped:25810 shmem:151 pagetables:9668 bounce:0 Sep 16 12:46:34 hydra01 kernel: [2092301.584244] free_cma:0 Sep 16 12:46:34 hydra01 kernel: [2092301.584251] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:46:34 hydra01 kernel: [2092301.584260] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:46:34 hydra01 kernel: [2092301.584266] Node 0 DMA32 free:141264kB min:6224kB low:7780kB high:9336kB active_anon:435780kB inactive_anon:792996kB active_file:95648kB inactive_file:1383012kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:3200kB shmem:0kB slab_reclaimable:165356kB slab_unreclaimable:42712kB kernel_stack:1072kB pagetables:1528kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:46:34 hydra01 kernel: [2092301.584275] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:46:34 hydra01 kernel: [2092301.584281] Node 0 Normal free:113764kB min:61324kB low:76652kB high:91984kB active_anon:4698800kB inactive_anon:2487932kB active_file:7235900kB inactive_file:12943260kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:0kB mapped:100040kB shmem:604kB slab_reclaimable:1292896kB slab_unreclaimable:359796kB kernel_stack:7784kB pagetables:37144kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:46:34 hydra01 kernel: [2092301.584290] lowmem_reserve[]: 0 0 0 0 Sep 16 12:46:34 hydra01 kernel: [2092301.584295] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:46:34 hydra01 kernel: [2092301.584316] Node 0 DMA32: 1259*4kB (UEM) 3777*8kB (UEM) 5687*16kB (UEM) 459*32kB (UEMR) 6*64kB (UMR) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141956kB Sep 16 12:46:34 hydra01 kernel: [2092301.584336] Node 0 Normal: 22112*4kB (UEMR) 2677*8kB (UEMR) 99*16kB (UEMR) 38*32kB (UEMR) 13*64kB (UMR) 9*128kB (UMR) 6*256kB (UMR) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 116184kB Sep 16 12:46:34 hydra01 kernel: [2092301.584356] 5846396 total pagecache pages Sep 16 12:46:34 hydra01 kernel: [2092301.584359] 431998 pages in swap cache Sep 16 12:46:34 hydra01 kernel: [2092301.584362] Swap cache stats: add 20777353, delete 20345355, find 12663172/14317953 Sep 16 12:46:34 hydra01 kernel: [2092301.584364] Free swap = 185321928kB Sep 16 12:46:34 hydra01 kernel: [2092301.584366] Total swap = 187617276kB Sep 16 12:46:34 hydra01 kernel: [2092301.772571] 8388592 pages RAM Sep 16 12:46:34 hydra01 kernel: [2092301.772575] 151696 pages reserved Sep 16 12:46:34 hydra01 kernel: [2092301.772577] 8896161 pages shared Sep 16 12:46:34 hydra01 kernel: [2092301.772579] 3243039 pages non-shared Sep 16 12:48:14 hydra01 kernel: [2092401.593113] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:48:14 hydra01 kernel: [2092401.593122] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:48:14 hydra01 kernel: [2092401.593124] Call Trace: Sep 16 12:48:14 hydra01 kernel: [2092401.593138] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:48:14 hydra01 kernel: [2092401.593145] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:48:14 hydra01 kernel: [2092401.593152] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:48:14 hydra01 kernel: [2092401.593159] [] alloc_pages_current+0xb8/0x180 Sep 16 12:48:14 hydra01 kernel: [2092401.593164] [] __get_free_pages+0xe/0x50 Sep 16 12:48:14 hydra01 kernel: [2092401.593171] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:48:14 hydra01 kernel: [2092401.593177] [] __kmalloc+0x161/0x170 Sep 16 12:48:14 hydra01 kernel: [2092401.593184] [] ? get_iowait_time+0x60/0x60 Sep 16 12:48:14 hydra01 kernel: [2092401.593188] [] stat_open+0x56/0xc0 Sep 16 12:48:14 hydra01 kernel: [2092401.593195] [] proc_reg_open+0xa0/0x180 Sep 16 12:48:14 hydra01 kernel: [2092401.593202] [] ? seq_release_private+0x60/0x60 Sep 16 12:48:14 hydra01 kernel: [2092401.593207] [] do_dentry_open+0x203/0x290 Sep 16 12:48:14 hydra01 kernel: [2092401.593212] [] ? init_once+0x20/0x20 Sep 16 12:48:14 hydra01 kernel: [2092401.593216] [] vfs_open+0x49/0x50 Sep 16 12:48:14 hydra01 kernel: [2092401.593222] [] do_last+0x296/0xea0 Sep 16 12:48:14 hydra01 kernel: [2092401.593226] [] ? inode_permission+0x18/0x50 Sep 16 12:48:14 hydra01 kernel: [2092401.593231] [] ? link_path_walk+0x6b/0x880 Sep 16 12:48:14 hydra01 kernel: [2092401.593236] [] path_openat+0xb3/0x4b0 Sep 16 12:48:14 hydra01 kernel: [2092401.593241] [] do_filp_open+0x41/0xa0 Sep 16 12:48:14 hydra01 kernel: [2092401.593246] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:48:14 hydra01 kernel: [2092401.593251] [] do_sys_open+0xf3/0x230 Sep 16 12:48:14 hydra01 kernel: [2092401.593257] [] ? sys_select+0xcb/0x110 Sep 16 12:48:14 hydra01 kernel: [2092401.593261] [] sys_open+0x21/0x30 Sep 16 12:48:14 hydra01 kernel: [2092401.593266] [] system_call_fastpath+0x1a/0x1f Sep 16 12:48:14 hydra01 kernel: [2092401.593269] Mem-Info: Sep 16 12:48:14 hydra01 kernel: [2092401.593272] Node 0 DMA per-cpu: Sep 16 12:48:14 hydra01 kernel: [2092401.593276] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593279] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593282] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593284] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593287] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593289] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593292] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593295] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593297] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593300] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593303] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593305] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593307] Node 0 DMA32 per-cpu: Sep 16 12:48:14 hydra01 kernel: [2092401.593311] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593314] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593317] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593319] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593322] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593325] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593327] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593330] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593332] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593335] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593338] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593340] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593342] Node 0 Normal per-cpu: Sep 16 12:48:14 hydra01 kernel: [2092401.593347] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593351] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593356] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593360] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593364] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593368] CPU 5: hi: 186, btch: 31 usd: 172 Sep 16 12:48:14 hydra01 kernel: [2092401.593373] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593377] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593381] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593385] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593390] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593394] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593406] active_anon:1285424 inactive_anon:820961 isolated_anon:0 Sep 16 12:48:14 hydra01 kernel: [2092401.593406] active_file:1831826 inactive_file:3589720 isolated_file:0 Sep 16 12:48:14 hydra01 kernel: [2092401.593406] unevictable:0 dirty:0 writeback:0 unstable:0 Sep 16 12:48:14 hydra01 kernel: [2092401.593406] free:65570 slab_reclaimable:365271 slab_unreclaimable:100630 Sep 16 12:48:14 hydra01 kernel: [2092401.593406] mapped:24424 shmem:151 pagetables:9564 bounce:0 Sep 16 12:48:14 hydra01 kernel: [2092401.593406] free_cma:0 Sep 16 12:48:14 hydra01 kernel: [2092401.593417] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:48:14 hydra01 kernel: [2092401.593432] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:48:14 hydra01 kernel: [2092401.593462] Node 0 DMA32 free:140520kB min:6224kB low:7780kB high:9336kB active_anon:435664kB inactive_anon:792996kB active_file:95700kB inactive_file:1382532kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:2820kB shmem:0kB slab_reclaimable:165172kB slab_unreclaimable:42980kB kernel_stack:1120kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:48:14 hydra01 kernel: [2092401.593477] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:48:14 hydra01 kernel: [2092401.593485] Node 0 Normal free:106208kB min:61324kB low:76652kB high:91984kB active_anon:4706032kB inactive_anon:2490848kB active_file:7231604kB inactive_file:12976348kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:0kB mapped:94876kB shmem:604kB slab_reclaimable:1295912kB slab_unreclaimable:359540kB kernel_stack:7736kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:48:14 hydra01 kernel: [2092401.593509] lowmem_reserve[]: 0 0 0 0 Sep 16 12:48:14 hydra01 kernel: [2092401.593552] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:48:14 hydra01 kernel: [2092401.593636] Node 0 DMA32: 1225*4kB (UEM) 759*8kB (UEM) 7164*16kB (UEM) 467*32kB (UEMR) 4*64kB (UMR) 1*128kB (U) 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141436kB Sep 16 12:48:14 hydra01 kernel: [2092401.593656] Node 0 Normal: 22247*4kB (UEMR) 1649*8kB (UEMR) 110*16kB (UEMR) 56*32kB (UEMR) 15*64kB (UEMR) 5*128kB (MR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 108356kB Sep 16 12:48:14 hydra01 kernel: [2092401.593677] 5852956 total pagecache pages Sep 16 12:48:14 hydra01 kernel: [2092401.593679] 431688 pages in swap cache Sep 16 12:48:14 hydra01 kernel: [2092401.593682] Swap cache stats: add 20778051, delete 20346363, find 12663591/14318438 Sep 16 12:48:14 hydra01 kernel: [2092401.593685] Free swap = 185320788kB Sep 16 12:48:14 hydra01 kernel: [2092401.593687] Total swap = 187617276kB Sep 16 12:48:14 hydra01 kernel: [2092401.778568] 8388592 pages RAM Sep 16 12:48:14 hydra01 kernel: [2092401.778573] 151696 pages reserved Sep 16 12:48:14 hydra01 kernel: [2092401.778576] 9082054 pages shared Sep 16 12:48:14 hydra01 kernel: [2092401.778578] 3055069 pages non-shared Sep 16 12:48:29 hydra01 kernel: [2092416.587658] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:48:29 hydra01 kernel: [2092416.587667] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:48:29 hydra01 kernel: [2092416.587669] Call Trace: Sep 16 12:48:29 hydra01 kernel: [2092416.587683] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:48:29 hydra01 kernel: [2092416.587690] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:48:29 hydra01 kernel: [2092416.587697] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:48:29 hydra01 kernel: [2092416.587704] [] alloc_pages_current+0xb8/0x180 Sep 16 12:48:29 hydra01 kernel: [2092416.587710] [] __get_free_pages+0xe/0x50 Sep 16 12:48:29 hydra01 kernel: [2092416.587717] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:48:29 hydra01 kernel: [2092416.587723] [] __kmalloc+0x161/0x170 Sep 16 12:48:29 hydra01 kernel: [2092416.587729] [] ? get_iowait_time+0x60/0x60 Sep 16 12:48:29 hydra01 kernel: [2092416.587734] [] stat_open+0x56/0xc0 Sep 16 12:48:29 hydra01 kernel: [2092416.587740] [] proc_reg_open+0xa0/0x180 Sep 16 12:48:29 hydra01 kernel: [2092416.587747] [] ? seq_release_private+0x60/0x60 Sep 16 12:48:29 hydra01 kernel: [2092416.587753] [] do_dentry_open+0x203/0x290 Sep 16 12:48:29 hydra01 kernel: [2092416.587759] [] ? init_once+0x20/0x20 Sep 16 12:48:29 hydra01 kernel: [2092416.587763] [] vfs_open+0x49/0x50 Sep 16 12:48:29 hydra01 kernel: [2092416.587769] [] do_last+0x296/0xea0 Sep 16 12:48:29 hydra01 kernel: [2092416.587774] [] ? inode_permission+0x18/0x50 Sep 16 12:48:29 hydra01 kernel: [2092416.587779] [] ? link_path_walk+0x6b/0x880 Sep 16 12:48:29 hydra01 kernel: [2092416.587784] [] path_openat+0xb3/0x4b0 Sep 16 12:48:29 hydra01 kernel: [2092416.587790] [] do_filp_open+0x41/0xa0 Sep 16 12:48:29 hydra01 kernel: [2092416.587795] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:48:29 hydra01 kernel: [2092416.587800] [] do_sys_open+0xf3/0x230 Sep 16 12:48:29 hydra01 kernel: [2092416.587806] [] ? sys_select+0xcb/0x110 Sep 16 12:48:29 hydra01 kernel: [2092416.587810] [] sys_open+0x21/0x30 Sep 16 12:48:29 hydra01 kernel: [2092416.587816] [] system_call_fastpath+0x1a/0x1f Sep 16 12:48:29 hydra01 kernel: [2092416.587818] Mem-Info: Sep 16 12:48:29 hydra01 kernel: [2092416.587821] Node 0 DMA per-cpu: Sep 16 12:48:29 hydra01 kernel: [2092416.587826] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587828] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587831] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587834] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587836] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587839] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587842] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587844] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587847] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587849] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587852] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587855] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587857] Node 0 DMA32 per-cpu: Sep 16 12:48:29 hydra01 kernel: [2092416.587861] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587864] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587866] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587869] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587871] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587874] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587877] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587879] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587882] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587885] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587887] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587890] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587892] Node 0 Normal per-cpu: Sep 16 12:48:29 hydra01 kernel: [2092416.587895] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587898] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587901] CPU 2: hi: 186, btch: 31 usd: 171 Sep 16 12:48:29 hydra01 kernel: [2092416.587904] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587906] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587909] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587911] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587914] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587917] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587919] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587922] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587925] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587932] active_anon:1285279 inactive_anon:820766 isolated_anon:0 Sep 16 12:48:29 hydra01 kernel: [2092416.587932] active_file:1831830 inactive_file:3584417 isolated_file:0 Sep 16 12:48:29 hydra01 kernel: [2092416.587932] unevictable:0 dirty:49 writeback:0 unstable:0 Sep 16 12:48:29 hydra01 kernel: [2092416.587932] free:72739 slab_reclaimable:365104 slab_unreclaimable:100632 Sep 16 12:48:29 hydra01 kernel: [2092416.587932] mapped:24370 shmem:151 pagetables:9679 bounce:0 Sep 16 12:48:29 hydra01 kernel: [2092416.587932] free_cma:0 Sep 16 12:48:29 hydra01 kernel: [2092416.587939] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:48:29 hydra01 kernel: [2092416.587948] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:48:29 hydra01 kernel: [2092416.587954] Node 0 DMA32 free:141184kB min:6224kB low:7780kB high:9336kB active_anon:435856kB inactive_anon:792996kB active_file:95680kB inactive_file:1381408kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:2820kB shmem:0kB slab_reclaimable:165472kB slab_unreclaimable:43028kB kernel_stack:1120kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:48:29 hydra01 kernel: [2092416.587963] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:48:29 hydra01 kernel: [2092416.587968] Node 0 Normal free:133928kB min:61324kB low:76652kB high:91984kB active_anon:4705260kB inactive_anon:2490068kB active_file:7231640kB inactive_file:12956260kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:192kB writeback:0kB mapped:94660kB shmem:604kB slab_reclaimable:1294944kB slab_unreclaimable:359500kB kernel_stack:7728kB pagetables:37184kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:48:29 hydra01 kernel: [2092416.587978] lowmem_reserve[]: 0 0 0 0 Sep 16 12:48:29 hydra01 kernel: [2092416.587983] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:48:29 hydra01 kernel: [2092416.588004] Node 0 DMA32: 1310*4kB (UEM) 781*8kB (UEM) 7157*16kB (UEM) 463*32kB (UEMR) 6*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141712kB Sep 16 12:48:29 hydra01 kernel: [2092416.588024] Node 0 Normal: 19459*4kB (UEMR) 6851*8kB (UEMR) 12*16kB (UEMR) 29*32kB (UEMR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 135428kB Sep 16 12:48:29 hydra01 kernel: [2092416.588044] 5847751 total pagecache pages Sep 16 12:48:29 hydra01 kernel: [2092416.588046] 431566 pages in swap cache Sep 16 12:48:29 hydra01 kernel: [2092416.588049] Swap cache stats: add 20778076, delete 20346510, find 12663591/14318438 Sep 16 12:48:29 hydra01 kernel: [2092416.588051] Free swap = 185320696kB Sep 16 12:48:29 hydra01 kernel: [2092416.588053] Total swap = 187617276kB Sep 16 12:48:29 hydra01 kernel: [2092416.781761] 8388592 pages RAM Sep 16 12:48:29 hydra01 kernel: [2092416.781765] 151696 pages reserved Sep 16 12:48:29 hydra01 kernel: [2092416.781767] 9089166 pages shared Sep 16 12:48:29 hydra01 kernel: [2092416.781770] 3048512 pages non-shared Sep 16 12:48:54 hydra01 kernel: [2092440.905026] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:48:54 hydra01 kernel: [2092440.905031] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:48:54 hydra01 kernel: [2092440.905035] Call Trace: Sep 16 12:48:54 hydra01 kernel: [2092440.905045] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:48:54 hydra01 kernel: [2092440.905049] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:48:54 hydra01 kernel: [2092440.905053] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:48:54 hydra01 kernel: [2092440.905057] [] alloc_pages_current+0xb8/0x180 Sep 16 12:48:54 hydra01 kernel: [2092440.905060] [] __get_free_pages+0xe/0x50 Sep 16 12:48:54 hydra01 kernel: [2092440.905064] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:48:54 hydra01 kernel: [2092440.905068] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:48:54 hydra01 kernel: [2092440.905071] [] __kmalloc+0x161/0x170 Sep 16 12:48:54 hydra01 kernel: [2092440.905075] [] ? get_iowait_time+0x60/0x60 Sep 16 12:48:54 hydra01 kernel: [2092440.905078] [] stat_open+0x56/0xc0 Sep 16 12:48:54 hydra01 kernel: [2092440.905081] [] proc_reg_open+0xa0/0x180 Sep 16 12:48:54 hydra01 kernel: [2092440.905085] [] ? seq_release_private+0x60/0x60 Sep 16 12:48:54 hydra01 kernel: [2092440.905089] [] do_dentry_open+0x203/0x290 Sep 16 12:48:54 hydra01 kernel: [2092440.905091] [] ? init_once+0x20/0x20 Sep 16 12:48:54 hydra01 kernel: [2092440.905108] [] vfs_open+0x49/0x50 Sep 16 12:48:54 hydra01 kernel: [2092440.905113] [] do_last+0x296/0xea0 Sep 16 12:48:54 hydra01 kernel: [2092440.905118] [] ? inode_permission+0x18/0x50 Sep 16 12:48:54 hydra01 kernel: [2092440.905122] [] ? link_path_walk+0x6b/0x880 Sep 16 12:48:54 hydra01 kernel: [2092440.905127] [] path_openat+0xb3/0x4b0 Sep 16 12:48:54 hydra01 kernel: [2092440.905134] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:48:54 hydra01 kernel: [2092440.905143] [] ? unmap_region+0xdb/0x120 Sep 16 12:48:54 hydra01 kernel: [2092440.905150] [] do_filp_open+0x41/0xa0 Sep 16 12:48:54 hydra01 kernel: [2092440.905158] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:48:54 hydra01 kernel: [2092440.905173] [] do_sys_open+0xf3/0x230 Sep 16 12:48:54 hydra01 kernel: [2092440.905176] [] sys_open+0x21/0x30 Sep 16 12:48:54 hydra01 kernel: [2092440.905179] [] system_call_fastpath+0x1a/0x1f Sep 16 12:48:54 hydra01 kernel: [2092440.905180] Mem-Info: Sep 16 12:48:54 hydra01 kernel: [2092440.905182] Node 0 DMA per-cpu: Sep 16 12:48:54 hydra01 kernel: [2092440.905185] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905186] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905187] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905189] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905190] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905192] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905197] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905198] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905199] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905201] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905202] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905203] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905205] Node 0 DMA32 per-cpu: Sep 16 12:48:54 hydra01 kernel: [2092440.905207] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905208] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905209] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905211] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905212] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905213] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905215] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905216] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905218] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905219] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905220] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905235] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905237] Node 0 Normal per-cpu: Sep 16 12:48:54 hydra01 kernel: [2092440.905240] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905243] CPU 1: hi: 186, btch: 31 usd: 185 Sep 16 12:48:54 hydra01 kernel: [2092440.905246] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905248] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905251] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905254] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905256] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905259] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905262] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905264] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905267] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905270] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905277] active_anon:1285140 inactive_anon:820714 isolated_anon:0 Sep 16 12:48:54 hydra01 kernel: [2092440.905277] active_file:1831847 inactive_file:3578334 isolated_file:0 Sep 16 12:48:54 hydra01 kernel: [2092440.905277] unevictable:0 dirty:43 writeback:981 unstable:0 Sep 16 12:48:54 hydra01 kernel: [2092440.905277] free:80901 slab_reclaimable:364358 slab_unreclaimable:100584 Sep 16 12:48:54 hydra01 kernel: [2092440.905277] mapped:24534 shmem:151 pagetables:9564 bounce:0 Sep 16 12:48:54 hydra01 kernel: [2092440.905277] free_cma:0 Sep 16 12:48:54 hydra01 kernel: [2092440.905307] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:48:54 hydra01 kernel: [2092440.905312] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:48:54 hydra01 kernel: [2092440.905315] Node 0 DMA32 free:138748kB min:6224kB low:7780kB high:9336kB active_anon:435684kB inactive_anon:792996kB active_file:95700kB inactive_file:1382268kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:2820kB shmem:0kB slab_reclaimable:165828kB slab_unreclaimable:42932kB kernel_stack:1120kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:48:54 hydra01 kernel: [2092440.905321] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:48:54 hydra01 kernel: [2092440.905326] Node 0 Normal free:169012kB min:61324kB low:76652kB high:91984kB active_anon:4704876kB inactive_anon:2489860kB active_file:7231688kB inactive_file:12931068kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:152kB writeback:3924kB mapped:95316kB shmem:604kB slab_reclaimable:1291604kB slab_unreclaimable:359404kB kernel_stack:7736kB pagetables:36724kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:48:54 hydra01 kernel: [2092440.905334] lowmem_reserve[]: 0 0 0 0 Sep 16 12:48:54 hydra01 kernel: [2092440.905337] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:48:54 hydra01 kernel: [2092440.905348] Node 0 DMA32: 3003*4kB (UEM) 1299*8kB (UEM) 6378*16kB (UEM) 450*32kB (UEMR) 2*64kB (UR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139492kB Sep 16 12:48:54 hydra01 kernel: [2092440.905374] Node 0 Normal: 20300*4kB (UEMR) 10953*8kB (UEMR) 12*16kB (UMR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 170872kB Sep 16 12:48:54 hydra01 kernel: [2092440.905394] 5841479 total pagecache pages Sep 16 12:48:54 hydra01 kernel: [2092440.905396] 431347 pages in swap cache Sep 16 12:48:54 hydra01 kernel: [2092440.905400] Swap cache stats: add 20778142, delete 20346795, find 12663592/14318440 Sep 16 12:48:54 hydra01 kernel: [2092440.905402] Free swap = 185320476kB Sep 16 12:48:54 hydra01 kernel: [2092440.905404] Total swap = 187617276kB Sep 16 12:48:54 hydra01 kernel: [2092441.093077] 8388592 pages RAM Sep 16 12:48:54 hydra01 kernel: [2092441.093131] 151696 pages reserved Sep 16 12:48:54 hydra01 kernel: [2092441.093148] 8831731 pages shared Sep 16 12:48:54 hydra01 kernel: [2092441.093150] 3044092 pages non-shared Sep 16 12:50:04 hydra01 kernel: [2092511.580901] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:50:04 hydra01 kernel: [2092511.580909] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:50:04 hydra01 kernel: [2092511.580912] Call Trace: Sep 16 12:50:04 hydra01 kernel: [2092511.580926] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:50:04 hydra01 kernel: [2092511.580934] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:50:04 hydra01 kernel: [2092511.580940] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:50:04 hydra01 kernel: [2092511.580947] [] alloc_pages_current+0xb8/0x180 Sep 16 12:50:04 hydra01 kernel: [2092511.580953] [] __get_free_pages+0xe/0x50 Sep 16 12:50:04 hydra01 kernel: [2092511.580960] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:50:04 hydra01 kernel: [2092511.580966] [] __kmalloc+0x161/0x170 Sep 16 12:50:04 hydra01 kernel: [2092511.580972] [] ? get_iowait_time+0x60/0x60 Sep 16 12:50:04 hydra01 kernel: [2092511.580977] [] stat_open+0x56/0xc0 Sep 16 12:50:04 hydra01 kernel: [2092511.580983] [] proc_reg_open+0xa0/0x180 Sep 16 12:50:04 hydra01 kernel: [2092511.580990] [] ? seq_release_private+0x60/0x60 Sep 16 12:50:04 hydra01 kernel: [2092511.580995] [] do_dentry_open+0x203/0x290 Sep 16 12:50:04 hydra01 kernel: [2092511.581000] [] ? init_once+0x20/0x20 Sep 16 12:50:04 hydra01 kernel: [2092511.581005] [] vfs_open+0x49/0x50 Sep 16 12:50:04 hydra01 kernel: [2092511.581010] [] do_last+0x296/0xea0 Sep 16 12:50:04 hydra01 kernel: [2092511.581015] [] ? inode_permission+0x18/0x50 Sep 16 12:50:04 hydra01 kernel: [2092511.581019] [] ? link_path_walk+0x6b/0x880 Sep 16 12:50:04 hydra01 kernel: [2092511.581024] [] path_openat+0xb3/0x4b0 Sep 16 12:50:04 hydra01 kernel: [2092511.581029] [] do_filp_open+0x41/0xa0 Sep 16 12:50:04 hydra01 kernel: [2092511.581035] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:50:04 hydra01 kernel: [2092511.581039] [] do_sys_open+0xf3/0x230 Sep 16 12:50:04 hydra01 kernel: [2092511.581045] [] ? sys_select+0xcb/0x110 Sep 16 12:50:04 hydra01 kernel: [2092511.581049] [] sys_open+0x21/0x30 Sep 16 12:50:04 hydra01 kernel: [2092511.581054] [] system_call_fastpath+0x1a/0x1f Sep 16 12:50:04 hydra01 kernel: [2092511.581057] Mem-Info: Sep 16 12:50:04 hydra01 kernel: [2092511.581060] Node 0 DMA per-cpu: Sep 16 12:50:04 hydra01 kernel: [2092511.581065] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581067] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581070] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581072] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581075] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581078] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581080] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581083] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581085] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581088] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581091] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581093] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581095] Node 0 DMA32 per-cpu: Sep 16 12:50:04 hydra01 kernel: [2092511.581099] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581102] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581105] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581108] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581110] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581113] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581115] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581118] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581121] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581123] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581126] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581129] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581131] Node 0 Normal per-cpu: Sep 16 12:50:04 hydra01 kernel: [2092511.581134] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581137] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581139] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581142] CPU 3: hi: 186, btch: 31 usd: 163 Sep 16 12:50:04 hydra01 kernel: [2092511.581145] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581148] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581150] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581153] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581155] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581158] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581175] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581178] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581186] active_anon:1285581 inactive_anon:820461 isolated_anon:0 Sep 16 12:50:04 hydra01 kernel: [2092511.581186] active_file:1831955 inactive_file:3587366 isolated_file:0 Sep 16 12:50:04 hydra01 kernel: [2092511.581186] unevictable:0 dirty:92 writeback:0 unstable:0 Sep 16 12:50:04 hydra01 kernel: [2092511.581186] free:79296 slab_reclaimable:362082 slab_unreclaimable:100471 Sep 16 12:50:04 hydra01 kernel: [2092511.581186] mapped:24582 shmem:151 pagetables:9525 bounce:0 Sep 16 12:50:04 hydra01 kernel: [2092511.581186] free_cma:0 Sep 16 12:50:04 hydra01 kernel: [2092511.581192] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:50:04 hydra01 kernel: [2092511.581202] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:50:04 hydra01 kernel: [2092511.581208] Node 0 DMA32 free:136648kB min:6224kB low:7780kB high:9336kB active_anon:441240kB inactive_anon:792996kB active_file:95816kB inactive_file:1376820kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:36kB writeback:0kB mapped:2920kB shmem:0kB slab_reclaimable:167800kB slab_unreclaimable:42724kB kernel_stack:1128kB pagetables:1396kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:50:04 hydra01 kernel: [2092511.581218] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:50:04 hydra01 kernel: [2092511.581223] Node 0 Normal free:164692kB min:61324kB low:76652kB high:91984kB active_anon:4701084kB inactive_anon:2488848kB active_file:7232004kB inactive_file:12972644kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:332kB writeback:0kB mapped:95408kB shmem:604kB slab_reclaimable:1280528kB slab_unreclaimable:359160kB kernel_stack:7720kB pagetables:36704kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:50:04 hydra01 kernel: [2092511.581251] lowmem_reserve[]: 0 0 0 0 Sep 16 12:50:04 hydra01 kernel: [2092511.581293] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:50:04 hydra01 kernel: [2092511.581369] Node 0 DMA32: 3756*4kB (UM) 508*8kB (UEM) 6389*16kB (UEM) 470*32kB (UEMR) 8*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 137376kB Sep 16 12:50:04 hydra01 kernel: [2092511.581389] Node 0 Normal: 38976*4kB (UEMR) 928*8kB (UEMR) 51*16kB (UEMR) 30*32kB (UEMR) 9*64kB (UMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 166960kB Sep 16 12:50:04 hydra01 kernel: [2092511.581408] 5849993 total pagecache pages Sep 16 12:50:04 hydra01 kernel: [2092511.581411] 430679 pages in swap cache Sep 16 12:50:04 hydra01 kernel: [2092511.581414] Swap cache stats: add 20778293, delete 20347614, find 12663592/14318440 Sep 16 12:50:04 hydra01 kernel: [2092511.581416] Free swap = 185319916kB Sep 16 12:50:04 hydra01 kernel: [2092511.581418] Total swap = 187617276kB Sep 16 12:50:04 hydra01 kernel: [2092511.769060] 8388592 pages RAM Sep 16 12:50:04 hydra01 kernel: [2092511.769065] 151696 pages reserved Sep 16 12:50:04 hydra01 kernel: [2092511.769068] 9123598 pages shared Sep 16 12:50:04 hydra01 kernel: [2092511.769070] 3014042 pages non-shared Sep 16 12:54:14 hydra01 kernel: [2092760.700093] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:54:14 hydra01 kernel: [2092760.700102] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:54:14 hydra01 kernel: [2092760.700104] Call Trace: Sep 16 12:54:14 hydra01 kernel: [2092760.700118] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:54:14 hydra01 kernel: [2092760.700126] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:54:14 hydra01 kernel: [2092760.700132] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:54:14 hydra01 kernel: [2092760.700139] [] alloc_pages_current+0xb8/0x180 Sep 16 12:54:14 hydra01 kernel: [2092760.700145] [] __get_free_pages+0xe/0x50 Sep 16 12:54:14 hydra01 kernel: [2092760.700152] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:54:14 hydra01 kernel: [2092760.700158] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:54:14 hydra01 kernel: [2092760.700163] [] __kmalloc+0x161/0x170 Sep 16 12:54:14 hydra01 kernel: [2092760.700197] [] ? get_iowait_time+0x60/0x60 Sep 16 12:54:14 hydra01 kernel: [2092760.700202] [] stat_open+0x56/0xc0 Sep 16 12:54:14 hydra01 kernel: [2092760.700209] [] proc_reg_open+0xa0/0x180 Sep 16 12:54:14 hydra01 kernel: [2092760.700216] [] ? seq_release_private+0x60/0x60 Sep 16 12:54:14 hydra01 kernel: [2092760.700221] [] do_dentry_open+0x203/0x290 Sep 16 12:54:14 hydra01 kernel: [2092760.700226] [] ? init_once+0x20/0x20 Sep 16 12:54:14 hydra01 kernel: [2092760.700230] [] vfs_open+0x49/0x50 Sep 16 12:54:14 hydra01 kernel: [2092760.700235] [] do_last+0x296/0xea0 Sep 16 12:54:14 hydra01 kernel: [2092760.700240] [] ? inode_permission+0x18/0x50 Sep 16 12:54:14 hydra01 kernel: [2092760.700245] [] ? link_path_walk+0x6b/0x880 Sep 16 12:54:14 hydra01 kernel: [2092760.700250] [] path_openat+0xb3/0x4b0 Sep 16 12:54:14 hydra01 kernel: [2092760.700257] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:54:14 hydra01 kernel: [2092760.700263] [] ? unmap_region+0xdb/0x120 Sep 16 12:54:14 hydra01 kernel: [2092760.700268] [] do_filp_open+0x41/0xa0 Sep 16 12:54:14 hydra01 kernel: [2092760.700273] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:54:14 hydra01 kernel: [2092760.700278] [] do_sys_open+0xf3/0x230 Sep 16 12:54:14 hydra01 kernel: [2092760.700282] [] sys_open+0x21/0x30 Sep 16 12:54:14 hydra01 kernel: [2092760.700288] [] system_call_fastpath+0x1a/0x1f Sep 16 12:54:14 hydra01 kernel: [2092760.700291] Mem-Info: Sep 16 12:54:14 hydra01 kernel: [2092760.700293] Node 0 DMA per-cpu: Sep 16 12:54:14 hydra01 kernel: [2092760.700298] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700301] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700303] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700306] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700309] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700311] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700314] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700316] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700319] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700322] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700324] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700327] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700329] Node 0 DMA32 per-cpu: Sep 16 12:54:14 hydra01 kernel: [2092760.700333] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700336] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700339] CPU 2: hi: 186, btch: 31 usd: 1 Sep 16 12:54:14 hydra01 kernel: [2092760.700341] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700344] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700346] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700349] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700352] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700354] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700359] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700362] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700365] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700367] Node 0 Normal per-cpu: Sep 16 12:54:14 hydra01 kernel: [2092760.700370] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700373] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700376] CPU 2: hi: 186, btch: 31 usd: 174 Sep 16 12:54:14 hydra01 kernel: [2092760.700378] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700381] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700384] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700386] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700389] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700392] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700394] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700397] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700399] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700408] active_anon:1283732 inactive_anon:821113 isolated_anon:0 Sep 16 12:54:14 hydra01 kernel: [2092760.700408] active_file:1832296 inactive_file:3613166 isolated_file:0 Sep 16 12:54:14 hydra01 kernel: [2092760.700408] unevictable:0 dirty:27 writeback:23 unstable:0 Sep 16 12:54:14 hydra01 kernel: [2092760.700408] free:77386 slab_reclaimable:352002 slab_unreclaimable:100133 Sep 16 12:54:14 hydra01 kernel: [2092760.700408] mapped:24862 shmem:151 pagetables:9560 bounce:0 Sep 16 12:54:14 hydra01 kernel: [2092760.700408] free_cma:0 Sep 16 12:54:14 hydra01 kernel: [2092760.700414] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:54:14 hydra01 kernel: [2092760.700424] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:54:14 hydra01 kernel: [2092760.700430] Node 0 DMA32 free:141340kB min:6224kB low:7780kB high:9336kB active_anon:437980kB inactive_anon:792992kB active_file:95916kB inactive_file:1368288kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2988kB shmem:0kB slab_reclaimable:173140kB slab_unreclaimable:42532kB kernel_stack:1136kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:54:14 hydra01 kernel: [2092760.700439] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:54:14 hydra01 kernel: [2092760.700444] Node 0 Normal free:152652kB min:61324kB low:76652kB high:91984kB active_anon:4696948kB inactive_anon:2491460kB active_file:7233268kB inactive_file:13083992kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:108kB writeback:92kB mapped:96460kB shmem:604kB slab_reclaimable:1234868kB slab_unreclaimable:358000kB kernel_stack:7680kB pagetables:36708kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:54:14 hydra01 kernel: [2092760.700453] lowmem_reserve[]: 0 0 0 0 Sep 16 12:54:14 hydra01 kernel: [2092760.700458] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:54:14 hydra01 kernel: [2092760.700480] Node 0 DMA32: 1148*4kB (UEM) 3076*8kB (UEM) 6196*16kB (UEM) 411*32kB (UEMR) 1*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 142064kB Sep 16 12:54:14 hydra01 kernel: [2092760.700499] Node 0 Normal: 33715*4kB (UEMR) 466*8kB (UEMR) 664*16kB (UEMR) 90*32kB (UEMR) 11*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 154076kB Sep 16 12:54:14 hydra01 kernel: [2092760.700518] 5874928 total pagecache pages Sep 16 12:54:14 hydra01 kernel: [2092760.700521] 429473 pages in swap cache Sep 16 12:54:14 hydra01 kernel: [2092760.700524] Swap cache stats: add 20778657, delete 20349184, find 12663606/14318456 Sep 16 12:54:14 hydra01 kernel: [2092760.700526] Free swap = 185318776kB Sep 16 12:54:14 hydra01 kernel: [2092760.700528] Total swap = 187617276kB Sep 16 12:54:14 hydra01 kernel: [2092760.880303] 8388592 pages RAM Sep 16 12:54:14 hydra01 kernel: [2092760.880308] 151696 pages reserved Sep 16 12:54:14 hydra01 kernel: [2092760.880310] 9172531 pages shared Sep 16 12:54:14 hydra01 kernel: [2092760.880312] 2962105 pages non-shared Sep 16 12:54:20 hydra01 kernel: [2092766.613305] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:54:20 hydra01 kernel: [2092766.613311] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:54:20 hydra01 kernel: [2092766.613312] Call Trace: Sep 16 12:54:20 hydra01 kernel: [2092766.613321] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:54:20 hydra01 kernel: [2092766.613326] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:54:20 hydra01 kernel: [2092766.613329] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:54:20 hydra01 kernel: [2092766.613334] [] alloc_pages_current+0xb8/0x180 Sep 16 12:54:20 hydra01 kernel: [2092766.613337] [] __get_free_pages+0xe/0x50 Sep 16 12:54:20 hydra01 kernel: [2092766.613341] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:54:20 hydra01 kernel: [2092766.613344] [] __kmalloc+0x161/0x170 Sep 16 12:54:20 hydra01 kernel: [2092766.613348] [] ? get_iowait_time+0x60/0x60 Sep 16 12:54:20 hydra01 kernel: [2092766.613350] [] stat_open+0x56/0xc0 Sep 16 12:54:20 hydra01 kernel: [2092766.613354] [] proc_reg_open+0xa0/0x180 Sep 16 12:54:20 hydra01 kernel: [2092766.613358] [] ? seq_release_private+0x60/0x60 Sep 16 12:54:20 hydra01 kernel: [2092766.613361] [] do_dentry_open+0x203/0x290 Sep 16 12:54:20 hydra01 kernel: [2092766.613364] [] ? init_once+0x20/0x20 Sep 16 12:54:20 hydra01 kernel: [2092766.613366] [] vfs_open+0x49/0x50 Sep 16 12:54:20 hydra01 kernel: [2092766.613369] [] do_last+0x296/0xea0 Sep 16 12:54:20 hydra01 kernel: [2092766.613372] [] ? inode_permission+0x18/0x50 Sep 16 12:54:20 hydra01 kernel: [2092766.613374] [] ? link_path_walk+0x6b/0x880 Sep 16 12:54:20 hydra01 kernel: [2092766.613377] [] path_openat+0xb3/0x4b0 Sep 16 12:54:20 hydra01 kernel: [2092766.613379] [] do_filp_open+0x41/0xa0 Sep 16 12:54:20 hydra01 kernel: [2092766.613382] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:54:20 hydra01 kernel: [2092766.613385] [] do_sys_open+0xf3/0x230 Sep 16 12:54:20 hydra01 kernel: [2092766.613388] [] ? sys_select+0xcb/0x110 Sep 16 12:54:20 hydra01 kernel: [2092766.613390] [] sys_open+0x21/0x30 Sep 16 12:54:20 hydra01 kernel: [2092766.613393] [] system_call_fastpath+0x1a/0x1f Sep 16 12:54:20 hydra01 kernel: [2092766.613395] Mem-Info: Sep 16 12:54:20 hydra01 kernel: [2092766.613396] Node 0 DMA per-cpu: Sep 16 12:54:20 hydra01 kernel: [2092766.613399] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613400] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613402] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613403] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613405] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613406] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613407] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613409] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613410] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613412] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613423] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613424] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613425] Node 0 DMA32 per-cpu: Sep 16 12:54:20 hydra01 kernel: [2092766.613428] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613429] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613430] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613432] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613433] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613434] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613436] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613437] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613439] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613440] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613441] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613443] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613444] Node 0 Normal per-cpu: Sep 16 12:54:20 hydra01 kernel: [2092766.613446] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613447] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613449] CPU 2: hi: 186, btch: 31 usd: 30 Sep 16 12:54:20 hydra01 kernel: [2092766.613450] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613452] CPU 4: hi: 186, btch: 31 usd: 167 Sep 16 12:54:20 hydra01 kernel: [2092766.613453] CPU 5: hi: 186, btch: 31 usd: 59 Sep 16 12:54:20 hydra01 kernel: [2092766.613455] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613456] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613458] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613459] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613461] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613467] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613471] active_anon:1283850 inactive_anon:821176 isolated_anon:0 Sep 16 12:54:20 hydra01 kernel: [2092766.613471] active_file:1832299 inactive_file:3610083 isolated_file:0 Sep 16 12:54:20 hydra01 kernel: [2092766.613471] unevictable:0 dirty:0 writeback:0 unstable:0 Sep 16 12:54:20 hydra01 kernel: [2092766.613471] free:80991 slab_reclaimable:351714 slab_unreclaimable:100132 Sep 16 12:54:20 hydra01 kernel: [2092766.613471] mapped:24981 shmem:151 pagetables:9560 bounce:0 Sep 16 12:54:20 hydra01 kernel: [2092766.613471] free_cma:0 Sep 16 12:54:20 hydra01 kernel: [2092766.613475] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:54:20 hydra01 kernel: [2092766.613480] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:54:20 hydra01 kernel: [2092766.613484] Node 0 DMA32 free:143532kB min:6224kB low:7780kB high:9336kB active_anon:439240kB inactive_anon:793000kB active_file:95916kB inactive_file:1365496kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3792kB shmem:20kB slab_reclaimable:173092kB slab_unreclaimable:42548kB kernel_stack:1144kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:54:20 hydra01 kernel: [2092766.613489] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:54:20 hydra01 kernel: [2092766.613492] Node 0 Normal free:164588kB min:61324kB low:76652kB high:91984kB active_anon:4696160kB inactive_anon:2491704kB active_file:7233280kB inactive_file:13074836kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:0kB mapped:96132kB shmem:584kB slab_reclaimable:1233764kB slab_unreclaimable:357980kB kernel_stack:7680kB pagetables:36708kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:54:20 hydra01 kernel: [2092766.613497] lowmem_reserve[]: 0 0 0 0 Sep 16 12:54:20 hydra01 kernel: [2092766.613499] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:54:20 hydra01 kernel: [2092766.613511] Node 0 DMA32: 897*4kB (UEM) 3416*8kB (UEM) 6195*16kB (UEM) 416*32kB (UMR) 1*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 143924kB Sep 16 12:54:20 hydra01 kernel: [2092766.613521] Node 0 Normal: 25892*4kB (UEMR) 3782*8kB (UEMR) 1435*16kB (UEMR) 196*32kB (UEMR) 18*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 165488kB Sep 16 12:54:20 hydra01 kernel: [2092766.613532] 5871810 total pagecache pages Sep 16 12:54:20 hydra01 kernel: [2092766.613533] 429403 pages in swap cache Sep 16 12:54:20 hydra01 kernel: [2092766.613535] Swap cache stats: add 20778696, delete 20349293, find 12663679/14318543 Sep 16 12:54:20 hydra01 kernel: [2092766.613536] Free swap = 185318940kB Sep 16 12:54:20 hydra01 kernel: [2092766.613537] Total swap = 187617276kB Sep 16 12:54:20 hydra01 kernel: [2092766.717089] 8388592 pages RAM Sep 16 12:54:20 hydra01 kernel: [2092766.717093] 151696 pages reserved Sep 16 12:54:20 hydra01 kernel: [2092766.717095] 9173107 pages shared Sep 16 12:54:20 hydra01 kernel: [2092766.717096] 2961710 pages non-shared Sep 16 12:54:53 hydra01 kernel: [2092799.944221] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:54:53 hydra01 kernel: [2092799.944229] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:54:53 hydra01 kernel: [2092799.944232] Call Trace: Sep 16 12:54:53 hydra01 kernel: [2092799.944246] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:54:53 hydra01 kernel: [2092799.944253] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:54:53 hydra01 kernel: [2092799.944260] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:54:53 hydra01 kernel: [2092799.944267] [] alloc_pages_current+0xb8/0x180 Sep 16 12:54:53 hydra01 kernel: [2092799.944272] [] __get_free_pages+0xe/0x50 Sep 16 12:54:53 hydra01 kernel: [2092799.944280] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:54:53 hydra01 kernel: [2092799.944286] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:54:53 hydra01 kernel: [2092799.944291] [] __kmalloc+0x161/0x170 Sep 16 12:54:53 hydra01 kernel: [2092799.944299] [] ? setkey+0x81/0x100 Sep 16 12:54:53 hydra01 kernel: [2092799.944305] [] ? get_iowait_time+0x60/0x60 Sep 16 12:54:53 hydra01 kernel: [2092799.944310] [] stat_open+0x56/0xc0 Sep 16 12:54:53 hydra01 kernel: [2092799.944316] [] proc_reg_open+0xa0/0x180 Sep 16 12:54:53 hydra01 kernel: [2092799.944324] [] ? seq_release_private+0x60/0x60 Sep 16 12:54:53 hydra01 kernel: [2092799.944329] [] do_dentry_open+0x203/0x290 Sep 16 12:54:53 hydra01 kernel: [2092799.944334] [] ? init_once+0x20/0x20 Sep 16 12:54:53 hydra01 kernel: [2092799.944338] [] vfs_open+0x49/0x50 Sep 16 12:54:53 hydra01 kernel: [2092799.944344] [] do_last+0x296/0xea0 Sep 16 12:54:53 hydra01 kernel: [2092799.944349] [] ? inode_permission+0x18/0x50 Sep 16 12:54:53 hydra01 kernel: [2092799.944353] [] ? link_path_walk+0x6b/0x880 Sep 16 12:54:53 hydra01 kernel: [2092799.944358] [] path_openat+0xb3/0x4b0 Sep 16 12:54:53 hydra01 kernel: [2092799.944365] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:54:53 hydra01 kernel: [2092799.944371] [] ? unmap_region+0xdb/0x120 Sep 16 12:54:53 hydra01 kernel: [2092799.944376] [] do_filp_open+0x41/0xa0 Sep 16 12:54:53 hydra01 kernel: [2092799.944381] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:54:53 hydra01 kernel: [2092799.944386] [] do_sys_open+0xf3/0x230 Sep 16 12:54:53 hydra01 kernel: [2092799.944391] [] sys_open+0x21/0x30 Sep 16 12:54:53 hydra01 kernel: [2092799.944396] [] system_call_fastpath+0x1a/0x1f Sep 16 12:54:53 hydra01 kernel: [2092799.944399] Mem-Info: Sep 16 12:54:53 hydra01 kernel: [2092799.944401] Node 0 DMA per-cpu: Sep 16 12:54:53 hydra01 kernel: [2092799.944406] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944409] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944411] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944414] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944416] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944419] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944422] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944424] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944427] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944429] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944432] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944435] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944437] Node 0 DMA32 per-cpu: Sep 16 12:54:53 hydra01 kernel: [2092799.944441] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944444] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944446] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944449] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944452] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944454] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944457] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944460] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944462] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944465] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944468] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944470] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944472] Node 0 Normal per-cpu: Sep 16 12:54:53 hydra01 kernel: [2092799.944476] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944478] CPU 1: hi: 186, btch: 31 usd: 170 Sep 16 12:54:53 hydra01 kernel: [2092799.944481] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944484] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944486] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944489] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944491] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944494] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944497] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944499] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944502] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944505] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944512] active_anon:1283883 inactive_anon:821213 isolated_anon:0 Sep 16 12:54:53 hydra01 kernel: [2092799.944512] active_file:1832376 inactive_file:3612741 isolated_file:0 Sep 16 12:54:53 hydra01 kernel: [2092799.944512] unevictable:0 dirty:23 writeback:0 unstable:0 Sep 16 12:54:53 hydra01 kernel: [2092799.944512] free:81492 slab_reclaimable:350163 slab_unreclaimable:100065 Sep 16 12:54:53 hydra01 kernel: [2092799.944512] mapped:24472 shmem:151 pagetables:9560 bounce:0 Sep 16 12:54:53 hydra01 kernel: [2092799.944512] free_cma:0 Sep 16 12:54:53 hydra01 kernel: [2092799.944519] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:54:53 hydra01 kernel: [2092799.944530] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:54:53 hydra01 kernel: [2092799.944539] Node 0 DMA32 free:140528kB min:6224kB low:7780kB high:9336kB active_anon:437988kB inactive_anon:792992kB active_file:95960kB inactive_file:1370652kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:28kB writeback:0kB mapped:2988kB shmem:0kB slab_reclaimable:173208kB slab_unreclaimable:42356kB kernel_stack:1152kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:54:53 hydra01 kernel: [2092799.944554] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:54:53 hydra01 kernel: [2092799.944561] Node 0 Normal free:169596kB min:61324kB low:76652kB high:91984kB active_anon:4697544kB inactive_anon:2491860kB active_file:7233544kB inactive_file:13080312kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:64kB writeback:0kB mapped:94900kB shmem:604kB slab_reclaimable:1227444kB slab_unreclaimable:357904kB kernel_stack:7680kB pagetables:36708kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:54:53 hydra01 kernel: [2092799.944574] lowmem_reserve[]: 0 0 0 0 Sep 16 12:54:53 hydra01 kernel: [2092799.944582] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:54:53 hydra01 kernel: [2092799.944605] Node 0 DMA32: 1074*4kB (UEM) 2793*8kB (UEM) 6172*16kB (UEM) 468*32kB (UEMR) 5*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141200kB Sep 16 12:54:53 hydra01 kernel: [2092799.944625] Node 0 Normal: 37177*4kB (UEMR) 2172*8kB (UEMR) 61*16kB (UEMR) 66*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 171092kB Sep 16 12:54:53 hydra01 kernel: [2092799.944645] 5874318 total pagecache pages Sep 16 12:54:53 hydra01 kernel: [2092799.944647] 429120 pages in swap cache Sep 16 12:54:53 hydra01 kernel: [2092799.944650] Swap cache stats: add 20778800, delete 20349680, find 12663700/14318569 Sep 16 12:54:53 hydra01 kernel: [2092799.944653] Free swap = 185318772kB Sep 16 12:54:53 hydra01 kernel: [2092799.944655] Total swap = 187617276kB Sep 16 12:54:53 hydra01 kernel: [2092800.134539] 8388592 pages RAM Sep 16 12:54:53 hydra01 kernel: [2092800.134551] 151696 pages reserved Sep 16 12:54:53 hydra01 kernel: [2092800.134561] 9173475 pages shared Sep 16 12:54:53 hydra01 kernel: [2092800.134571] 2960996 pages non-shared Sep 16 12:54:53 hydra01 kernel: [2092800.136768] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:54:53 hydra01 kernel: [2092800.136772] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:54:53 hydra01 kernel: [2092800.136773] Call Trace: Sep 16 12:54:53 hydra01 kernel: [2092800.136782] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:54:53 hydra01 kernel: [2092800.136786] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:54:53 hydra01 kernel: [2092800.136790] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:54:53 hydra01 kernel: [2092800.136794] [] alloc_pages_current+0xb8/0x180 Sep 16 12:54:53 hydra01 kernel: [2092800.136797] [] __get_free_pages+0xe/0x50 Sep 16 12:54:53 hydra01 kernel: [2092800.136801] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:54:53 hydra01 kernel: [2092800.136804] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:54:53 hydra01 kernel: [2092800.136822] [] __kmalloc+0x161/0x170 Sep 16 12:54:53 hydra01 kernel: [2092800.136830] [] ? setkey+0x80/0x100 Sep 16 12:54:53 hydra01 kernel: [2092800.136836] [] ? get_iowait_time+0x60/0x60 Sep 16 12:54:53 hydra01 kernel: [2092800.136841] [] stat_open+0x56/0xc0 Sep 16 12:54:53 hydra01 kernel: [2092800.136847] [] proc_reg_open+0xa0/0x180 Sep 16 12:54:53 hydra01 kernel: [2092800.136854] [] ? seq_release_private+0x60/0x60 Sep 16 12:54:53 hydra01 kernel: [2092800.136860] [] do_dentry_open+0x203/0x290 Sep 16 12:54:53 hydra01 kernel: [2092800.136865] [] ? init_once+0x20/0x20 Sep 16 12:54:53 hydra01 kernel: [2092800.136869] [] vfs_open+0x49/0x50 Sep 16 12:54:53 hydra01 kernel: [2092800.136874] [] do_last+0x296/0xea0 Sep 16 12:54:53 hydra01 kernel: [2092800.136879] [] ? inode_permission+0x18/0x50 Sep 16 12:54:53 hydra01 kernel: [2092800.136883] [] ? link_path_walk+0x6b/0x880 Sep 16 12:54:53 hydra01 kernel: [2092800.136888] [] path_openat+0xb3/0x4b0 Sep 16 12:54:53 hydra01 kernel: [2092800.136895] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:54:53 hydra01 kernel: [2092800.136901] [] ? unmap_region+0xdb/0x120 Sep 16 12:54:53 hydra01 kernel: [2092800.136906] [] do_filp_open+0x41/0xa0 Sep 16 12:54:53 hydra01 kernel: [2092800.136912] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:54:53 hydra01 kernel: [2092800.136916] [] do_sys_open+0xf3/0x230 Sep 16 12:54:53 hydra01 kernel: [2092800.136921] [] sys_open+0x21/0x30 Sep 16 12:54:53 hydra01 kernel: [2092800.136926] [] system_call_fastpath+0x1a/0x1f Sep 16 12:54:53 hydra01 kernel: [2092800.136929] Mem-Info: Sep 16 12:54:53 hydra01 kernel: [2092800.136931] Node 0 DMA per-cpu: Sep 16 12:54:53 hydra01 kernel: [2092800.136948] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136949] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136950] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136952] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136953] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136955] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136956] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136957] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136959] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136960] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136961] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136963] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136964] Node 0 DMA32 per-cpu: Sep 16 12:54:53 hydra01 kernel: [2092800.136966] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136967] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136969] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136970] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136971] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136973] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136974] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136976] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136977] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136978] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136980] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136981] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136982] Node 0 Normal per-cpu: Sep 16 12:54:53 hydra01 kernel: [2092800.136984] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136985] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136987] CPU 2: hi: 186, btch: 31 usd: 168 Sep 16 12:54:53 hydra01 kernel: [2092800.136988] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136990] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136991] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136992] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136994] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.136995] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.137011] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.137013] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.137016] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:54:53 hydra01 kernel: [2092800.137024] active_anon:1283883 inactive_anon:821213 isolated_anon:0 Sep 16 12:54:53 hydra01 kernel: [2092800.137024] active_file:1832376 inactive_file:3615842 isolated_file:0 Sep 16 12:54:53 hydra01 kernel: [2092800.137024] unevictable:0 dirty:132 writeback:114 unstable:0 Sep 16 12:54:53 hydra01 kernel: [2092800.137024] free:77976 slab_reclaimable:350163 slab_unreclaimable:100065 Sep 16 12:54:53 hydra01 kernel: [2092800.137024] mapped:24472 shmem:151 pagetables:9560 bounce:0 Sep 16 12:54:53 hydra01 kernel: [2092800.137024] free_cma:0 Sep 16 12:54:53 hydra01 kernel: [2092800.137030] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:54:53 hydra01 kernel: [2092800.137039] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:54:53 hydra01 kernel: [2092800.137045] Node 0 DMA32 free:139396kB min:6224kB low:7780kB high:9336kB active_anon:437988kB inactive_anon:792992kB active_file:95960kB inactive_file:1370652kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:28kB writeback:0kB mapped:2988kB shmem:0kB slab_reclaimable:173208kB slab_unreclaimable:42356kB kernel_stack:1152kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:54:53 hydra01 kernel: [2092800.137055] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:54:53 hydra01 kernel: [2092800.137072] Node 0 Normal free:156956kB min:61324kB low:76652kB high:91984kB active_anon:4697544kB inactive_anon:2491860kB active_file:7233544kB inactive_file:13092716kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:500kB writeback:456kB mapped:94900kB shmem:604kB slab_reclaimable:1227444kB slab_unreclaimable:357904kB kernel_stack:7680kB pagetables:36708kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:54:53 hydra01 kernel: [2092800.137077] lowmem_reserve[]: 0 0 0 0 Sep 16 12:54:53 hydra01 kernel: [2092800.137079] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:54:53 hydra01 kernel: [2092800.137090] Node 0 DMA32: 1074*4kB (UEM) 2793*8kB (UEM) 6172*16kB (UEM) 427*32kB (UEMR) 1*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139632kB Sep 16 12:54:53 hydra01 kernel: [2092800.137101] Node 0 Normal: 32139*4kB (UEMR) 3149*8kB (UEMR) 61*16kB (UEMR) 66*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 158756kB Sep 16 12:54:53 hydra01 kernel: [2092800.137111] 5877370 total pagecache pages Sep 16 12:54:53 hydra01 kernel: [2092800.137112] 429120 pages in swap cache Sep 16 12:54:53 hydra01 kernel: [2092800.137114] Swap cache stats: add 20778800, delete 20349680, find 12663700/14318569 Sep 16 12:54:53 hydra01 kernel: [2092800.137115] Free swap = 185318772kB Sep 16 12:54:53 hydra01 kernel: [2092800.137116] Total swap = 187617276kB Sep 16 12:54:53 hydra01 kernel: [2092800.324490] 8388592 pages RAM Sep 16 12:54:53 hydra01 kernel: [2092800.324497] 151696 pages reserved Sep 16 12:54:53 hydra01 kernel: [2092800.324500] 8912009 pages shared Sep 16 12:54:53 hydra01 kernel: [2092800.324502] 2961422 pages non-shared Sep 16 12:57:53 hydra01 kernel: [2092980.296521] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:57:53 hydra01 kernel: [2092980.296526] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:57:53 hydra01 kernel: [2092980.296528] Call Trace: Sep 16 12:57:53 hydra01 kernel: [2092980.296537] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:57:53 hydra01 kernel: [2092980.296542] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:57:53 hydra01 kernel: [2092980.296546] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:57:53 hydra01 kernel: [2092980.296551] [] alloc_pages_current+0xb8/0x180 Sep 16 12:57:53 hydra01 kernel: [2092980.296555] [] __get_free_pages+0xe/0x50 Sep 16 12:57:53 hydra01 kernel: [2092980.296559] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:57:53 hydra01 kernel: [2092980.296563] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 12:57:53 hydra01 kernel: [2092980.296566] [] __kmalloc+0x161/0x170 Sep 16 12:57:53 hydra01 kernel: [2092980.296572] [] ? setkey+0x81/0x100 Sep 16 12:57:53 hydra01 kernel: [2092980.296576] [] ? get_iowait_time+0x60/0x60 Sep 16 12:57:53 hydra01 kernel: [2092980.296579] [] stat_open+0x56/0xc0 Sep 16 12:57:53 hydra01 kernel: [2092980.296583] [] proc_reg_open+0xa0/0x180 Sep 16 12:57:53 hydra01 kernel: [2092980.296588] [] ? seq_release_private+0x60/0x60 Sep 16 12:57:53 hydra01 kernel: [2092980.296591] [] do_dentry_open+0x203/0x290 Sep 16 12:57:53 hydra01 kernel: [2092980.296594] [] ? init_once+0x20/0x20 Sep 16 12:57:53 hydra01 kernel: [2092980.296597] [] vfs_open+0x49/0x50 Sep 16 12:57:53 hydra01 kernel: [2092980.296600] [] do_last+0x296/0xea0 Sep 16 12:57:53 hydra01 kernel: [2092980.296603] [] ? inode_permission+0x18/0x50 Sep 16 12:57:53 hydra01 kernel: [2092980.296606] [] ? link_path_walk+0x6b/0x880 Sep 16 12:57:53 hydra01 kernel: [2092980.296609] [] path_openat+0xb3/0x4b0 Sep 16 12:57:53 hydra01 kernel: [2092980.296613] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 12:57:53 hydra01 kernel: [2092980.296617] [] ? unmap_region+0xdb/0x120 Sep 16 12:57:53 hydra01 kernel: [2092980.296620] [] do_filp_open+0x41/0xa0 Sep 16 12:57:53 hydra01 kernel: [2092980.296623] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:57:53 hydra01 kernel: [2092980.296626] [] do_sys_open+0xf3/0x230 Sep 16 12:57:53 hydra01 kernel: [2092980.296629] [] sys_open+0x21/0x30 Sep 16 12:57:53 hydra01 kernel: [2092980.296632] [] system_call_fastpath+0x1a/0x1f Sep 16 12:57:53 hydra01 kernel: [2092980.296634] Mem-Info: Sep 16 12:57:53 hydra01 kernel: [2092980.296636] Node 0 DMA per-cpu: Sep 16 12:57:53 hydra01 kernel: [2092980.296639] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296640] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296642] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296643] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296645] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296646] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296648] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296649] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296651] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296653] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296654] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296656] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296657] Node 0 DMA32 per-cpu: Sep 16 12:57:53 hydra01 kernel: [2092980.296659] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296661] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296663] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296664] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296666] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296667] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296669] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296670] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296672] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296674] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296675] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296677] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296678] Node 0 Normal per-cpu: Sep 16 12:57:53 hydra01 kernel: [2092980.296680] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296682] CPU 1: hi: 186, btch: 31 usd: 183 Sep 16 12:57:53 hydra01 kernel: [2092980.296683] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296685] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296687] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296688] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296690] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296691] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296693] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296694] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296696] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296698] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296702] active_anon:1280517 inactive_anon:821215 isolated_anon:0 Sep 16 12:57:53 hydra01 kernel: [2092980.296702] active_file:1832613 inactive_file:3644156 isolated_file:0 Sep 16 12:57:53 hydra01 kernel: [2092980.296702] unevictable:0 dirty:13 writeback:0 unstable:0 Sep 16 12:57:53 hydra01 kernel: [2092980.296702] free:66617 slab_reclaimable:343779 slab_unreclaimable:99806 Sep 16 12:57:53 hydra01 kernel: [2092980.296702] mapped:24732 shmem:153 pagetables:9702 bounce:0 Sep 16 12:57:53 hydra01 kernel: [2092980.296702] free_cma:0 Sep 16 12:57:53 hydra01 kernel: [2092980.296706] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:57:53 hydra01 kernel: [2092980.296712] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:57:53 hydra01 kernel: [2092980.296715] Node 0 DMA32 free:140044kB min:6224kB low:7780kB high:9336kB active_anon:440820kB inactive_anon:792992kB active_file:96052kB inactive_file:1370524kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:32kB writeback:0kB mapped:2924kB shmem:0kB slab_reclaimable:173724kB slab_unreclaimable:41908kB kernel_stack:1120kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:57:53 hydra01 kernel: [2092980.296721] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:57:53 hydra01 kernel: [2092980.296724] Node 0 Normal free:110580kB min:61324kB low:76652kB high:91984kB active_anon:4681248kB inactive_anon:2491868kB active_file:7234400kB inactive_file:13206100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:20kB writeback:0kB mapped:96004kB shmem:612kB slab_reclaimable:1201392kB slab_unreclaimable:357316kB kernel_stack:7712kB pagetables:37276kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:57:53 hydra01 kernel: [2092980.296730] lowmem_reserve[]: 0 0 0 0 Sep 16 12:57:53 hydra01 kernel: [2092980.296733] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:57:53 hydra01 kernel: [2092980.296746] Node 0 DMA32: 1007*4kB (UEM) 1291*8kB (UEM) 6814*16kB (UEM) 515*32kB (UEMR) 7*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140820kB Sep 16 12:57:53 hydra01 kernel: [2092980.296758] Node 0 Normal: 21109*4kB (UEMR) 2963*8kB (UEMR) 51*16kB (UEMR) 50*32kB (UER) 14*64kB (UER) 3*128kB (ER) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 112860kB Sep 16 12:57:53 hydra01 kernel: [2092980.296770] 5904091 total pagecache pages Sep 16 12:57:53 hydra01 kernel: [2092980.296771] 427401 pages in swap cache Sep 16 12:57:53 hydra01 kernel: [2092980.296773] Swap cache stats: add 20779240, delete 20351839, find 12663704/14318574 Sep 16 12:57:53 hydra01 kernel: [2092980.296774] Free swap = 185317576kB Sep 16 12:57:53 hydra01 kernel: [2092980.296776] Total swap = 187617276kB Sep 16 12:57:53 hydra01 kernel: [2092980.413450] 8388592 pages RAM Sep 16 12:57:53 hydra01 kernel: [2092980.413458] 151696 pages reserved Sep 16 12:57:53 hydra01 kernel: [2092980.413477] 9201423 pages shared Sep 16 12:57:53 hydra01 kernel: [2092980.413482] 2930168 pages non-shared Sep 16 12:59:15 hydra01 kernel: [2093061.489501] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 12:59:15 hydra01 kernel: [2093061.489520] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 12:59:15 hydra01 kernel: [2093061.489522] Call Trace: Sep 16 12:59:15 hydra01 kernel: [2093061.489534] [] warn_alloc_failed+0xe9/0x140 Sep 16 12:59:15 hydra01 kernel: [2093061.489540] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 12:59:15 hydra01 kernel: [2093061.489546] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 12:59:15 hydra01 kernel: [2093061.489552] [] alloc_pages_current+0xb8/0x180 Sep 16 12:59:15 hydra01 kernel: [2093061.489556] [] __get_free_pages+0xe/0x50 Sep 16 12:59:15 hydra01 kernel: [2093061.489562] [] kmalloc_order_trace+0x39/0xb0 Sep 16 12:59:15 hydra01 kernel: [2093061.489567] [] __kmalloc+0x161/0x170 Sep 16 12:59:15 hydra01 kernel: [2093061.489572] [] ? get_iowait_time+0x60/0x60 Sep 16 12:59:15 hydra01 kernel: [2093061.489576] [] stat_open+0x56/0xc0 Sep 16 12:59:15 hydra01 kernel: [2093061.489581] [] proc_reg_open+0xa0/0x180 Sep 16 12:59:15 hydra01 kernel: [2093061.489587] [] ? seq_release_private+0x60/0x60 Sep 16 12:59:15 hydra01 kernel: [2093061.489592] [] do_dentry_open+0x203/0x290 Sep 16 12:59:15 hydra01 kernel: [2093061.489596] [] ? init_once+0x20/0x20 Sep 16 12:59:15 hydra01 kernel: [2093061.489599] [] vfs_open+0x49/0x50 Sep 16 12:59:15 hydra01 kernel: [2093061.489604] [] do_last+0x296/0xea0 Sep 16 12:59:15 hydra01 kernel: [2093061.489607] [] ? inode_permission+0x18/0x50 Sep 16 12:59:15 hydra01 kernel: [2093061.489611] [] ? link_path_walk+0x6b/0x880 Sep 16 12:59:15 hydra01 kernel: [2093061.489615] [] path_openat+0xb3/0x4b0 Sep 16 12:59:15 hydra01 kernel: [2093061.489619] [] do_filp_open+0x41/0xa0 Sep 16 12:59:15 hydra01 kernel: [2093061.489624] [] ? __alloc_fd+0xd9/0x130 Sep 16 12:59:15 hydra01 kernel: [2093061.489628] [] do_sys_open+0xf3/0x230 Sep 16 12:59:15 hydra01 kernel: [2093061.489632] [] ? sys_select+0xcb/0x110 Sep 16 12:59:15 hydra01 kernel: [2093061.489649] [] sys_open+0x21/0x30 Sep 16 12:59:15 hydra01 kernel: [2093061.489654] [] system_call_fastpath+0x1a/0x1f Sep 16 12:59:15 hydra01 kernel: [2093061.489657] Mem-Info: Sep 16 12:59:15 hydra01 kernel: [2093061.489660] Node 0 DMA per-cpu: Sep 16 12:59:15 hydra01 kernel: [2093061.489664] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489667] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489670] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489672] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489675] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489678] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489680] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489683] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489685] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489688] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489691] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489694] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489696] Node 0 DMA32 per-cpu: Sep 16 12:59:15 hydra01 kernel: [2093061.489700] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489702] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489705] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489708] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489710] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489713] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489716] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489718] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489721] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489723] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489726] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489729] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489731] Node 0 Normal per-cpu: Sep 16 12:59:15 hydra01 kernel: [2093061.489734] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489737] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489739] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489742] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489745] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489747] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489750] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489767] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489769] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489771] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489773] CPU 10: hi: 186, btch: 31 usd: 167 Sep 16 12:59:15 hydra01 kernel: [2093061.489775] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489781] active_anon:1280485 inactive_anon:821113 isolated_anon:0 Sep 16 12:59:15 hydra01 kernel: [2093061.489781] active_file:1832661 inactive_file:3643872 isolated_file:0 Sep 16 12:59:15 hydra01 kernel: [2093061.489781] unevictable:0 dirty:52 writeback:0 unstable:0 Sep 16 12:59:15 hydra01 kernel: [2093061.489781] free:70390 slab_reclaimable:341957 slab_unreclaimable:99786 Sep 16 12:59:15 hydra01 kernel: [2093061.489781] mapped:24388 shmem:151 pagetables:9673 bounce:0 Sep 16 12:59:15 hydra01 kernel: [2093061.489781] free_cma:0 Sep 16 12:59:15 hydra01 kernel: [2093061.489786] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 12:59:15 hydra01 kernel: [2093061.489793] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 12:59:15 hydra01 kernel: [2093061.489798] Node 0 DMA32 free:139144kB min:6224kB low:7780kB high:9336kB active_anon:440668kB inactive_anon:792992kB active_file:96068kB inactive_file:1369864kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:2884kB shmem:0kB slab_reclaimable:173548kB slab_unreclaimable:41852kB kernel_stack:1136kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 12:59:15 hydra01 kernel: [2093061.489806] lowmem_reserve[]: 0 0 29232 29232 Sep 16 12:59:15 hydra01 kernel: [2093061.489810] Node 0 Normal free:126996kB min:61324kB low:76652kB high:91984kB active_anon:4681272kB inactive_anon:2491460kB active_file:7234576kB inactive_file:13205240kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:204kB writeback:0kB mapped:94668kB shmem:604kB slab_reclaimable:1194280kB slab_unreclaimable:357292kB kernel_stack:7704kB pagetables:37160kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 12:59:15 hydra01 kernel: [2093061.489832] lowmem_reserve[]: 0 0 0 0 Sep 16 12:59:15 hydra01 kernel: [2093061.489837] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 12:59:15 hydra01 kernel: [2093061.489858] Node 0 DMA32: 921*4kB (UEM) 1361*8kB (UEM) 6861*16kB (UEM) 458*32kB (UEMR) 1*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139580kB Sep 16 12:59:15 hydra01 kernel: [2093061.489877] Node 0 Normal: 25345*4kB (UEMR) 405*8kB (UEMR) 450*16kB (UEMR) 193*32kB (UEMR) 120*64kB (UEMR) 16*128kB (EMR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 128748kB Sep 16 12:59:15 hydra01 kernel: [2093061.489897] 5903374 total pagecache pages Sep 16 12:59:15 hydra01 kernel: [2093061.489912] 426639 pages in swap cache Sep 16 12:59:15 hydra01 kernel: [2093061.489913] Swap cache stats: add 20779303, delete 20352664, find 12663704/14318574 Sep 16 12:59:15 hydra01 kernel: [2093061.489915] Free swap = 185317908kB Sep 16 12:59:15 hydra01 kernel: [2093061.489916] Total swap = 187617276kB Sep 16 12:59:15 hydra01 kernel: [2093061.661588] 8388592 pages RAM Sep 16 12:59:15 hydra01 kernel: [2093061.661593] 151696 pages reserved Sep 16 12:59:15 hydra01 kernel: [2093061.661594] 8944110 pages shared Sep 16 12:59:15 hydra01 kernel: [2093061.661596] 2926722 pages non-shared Sep 16 13:14:10 hydra01 kernel: [2093956.358330] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:14:10 hydra01 kernel: [2093956.358336] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:14:10 hydra01 kernel: [2093956.358338] Call Trace: Sep 16 13:14:10 hydra01 kernel: [2093956.358348] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:14:10 hydra01 kernel: [2093956.358353] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 13:14:10 hydra01 kernel: [2093956.358357] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:14:10 hydra01 kernel: [2093956.358362] [] alloc_pages_current+0xb8/0x180 Sep 16 13:14:10 hydra01 kernel: [2093956.358366] [] __get_free_pages+0xe/0x50 Sep 16 13:14:10 hydra01 kernel: [2093956.358371] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:14:10 hydra01 kernel: [2093956.358374] [] __kmalloc+0x161/0x170 Sep 16 13:14:10 hydra01 kernel: [2093956.358379] [] ? get_iowait_time+0x60/0x60 Sep 16 13:14:10 hydra01 kernel: [2093956.358382] [] stat_open+0x56/0xc0 Sep 16 13:14:10 hydra01 kernel: [2093956.358386] [] proc_reg_open+0xa0/0x180 Sep 16 13:14:10 hydra01 kernel: [2093956.358391] [] ? seq_release_private+0x60/0x60 Sep 16 13:14:10 hydra01 kernel: [2093956.358394] [] do_dentry_open+0x203/0x290 Sep 16 13:14:10 hydra01 kernel: [2093956.358397] [] ? init_once+0x20/0x20 Sep 16 13:14:10 hydra01 kernel: [2093956.358400] [] vfs_open+0x49/0x50 Sep 16 13:14:10 hydra01 kernel: [2093956.358404] [] do_last+0x296/0xea0 Sep 16 13:14:10 hydra01 kernel: [2093956.358407] [] ? inode_permission+0x18/0x50 Sep 16 13:14:10 hydra01 kernel: [2093956.358410] [] ? link_path_walk+0x6b/0x880 Sep 16 13:14:10 hydra01 kernel: [2093956.358413] [] path_openat+0xb3/0x4b0 Sep 16 13:14:10 hydra01 kernel: [2093956.358416] [] do_filp_open+0x41/0xa0 Sep 16 13:14:10 hydra01 kernel: [2093956.358420] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:14:10 hydra01 kernel: [2093956.358423] [] do_sys_open+0xf3/0x230 Sep 16 13:14:10 hydra01 kernel: [2093956.358426] [] ? sys_select+0xcb/0x110 Sep 16 13:14:10 hydra01 kernel: [2093956.358429] [] sys_open+0x21/0x30 Sep 16 13:14:10 hydra01 kernel: [2093956.358433] [] system_call_fastpath+0x1a/0x1f Sep 16 13:14:10 hydra01 kernel: [2093956.358434] Mem-Info: Sep 16 13:14:10 hydra01 kernel: [2093956.358436] Node 0 DMA per-cpu: Sep 16 13:14:10 hydra01 kernel: [2093956.358439] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358441] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358443] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358444] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358446] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358447] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358449] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358451] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358452] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358454] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358456] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358457] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358459] Node 0 DMA32 per-cpu: Sep 16 13:14:10 hydra01 kernel: [2093956.358461] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358463] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358464] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358466] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358468] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358469] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358471] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358473] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358474] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358476] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358478] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358479] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358481] Node 0 Normal per-cpu: Sep 16 13:14:10 hydra01 kernel: [2093956.358483] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358484] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358486] CPU 2: hi: 186, btch: 31 usd: 173 Sep 16 13:14:10 hydra01 kernel: [2093956.358488] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358490] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358491] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358493] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358495] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358496] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358498] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358500] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358501] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358506] active_anon:1269091 inactive_anon:814153 isolated_anon:0 Sep 16 13:14:10 hydra01 kernel: [2093956.358506] active_file:1841416 inactive_file:3674044 isolated_file:0 Sep 16 13:14:10 hydra01 kernel: [2093956.358506] unevictable:0 dirty:135 writeback:0 unstable:0 Sep 16 13:14:10 hydra01 kernel: [2093956.358506] free:70571 slab_reclaimable:324622 slab_unreclaimable:97018 Sep 16 13:14:10 hydra01 kernel: [2093956.358506] mapped:25655 shmem:151 pagetables:9683 bounce:0 Sep 16 13:14:10 hydra01 kernel: [2093956.358506] free_cma:0 Sep 16 13:14:10 hydra01 kernel: [2093956.358510] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:14:10 hydra01 kernel: [2093956.358516] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:14:10 hydra01 kernel: [2093956.358520] Node 0 DMA32 free:140692kB min:6224kB low:7780kB high:9336kB active_anon:434584kB inactive_anon:792988kB active_file:96984kB inactive_file:1380876kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:12kB writeback:0kB mapped:3004kB shmem:0kB slab_reclaimable:166068kB slab_unreclaimable:37988kB kernel_stack:1128kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:14:10 hydra01 kernel: [2093956.358526] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:14:10 hydra01 kernel: [2093956.358529] Node 0 Normal free:126040kB min:61324kB low:76652kB high:91984kB active_anon:4641780kB inactive_anon:2463624kB active_file:7268680kB inactive_file:13315300kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:528kB writeback:0kB mapped:99616kB shmem:604kB slab_reclaimable:1132420kB slab_unreclaimable:350084kB kernel_stack:7688kB pagetables:37200kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:14:10 hydra01 kernel: [2093956.358535] lowmem_reserve[]: 0 0 0 0 Sep 16 13:14:10 hydra01 kernel: [2093956.358538] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:14:10 hydra01 kernel: [2093956.358552] Node 0 DMA32: 2730*4kB (UEM) 2399*8kB (UEM) 5746*16kB (UEM) 588*32kB (UEMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141504kB Sep 16 13:14:10 hydra01 kernel: [2093956.358564] Node 0 Normal: 19442*4kB (UEMR) 5257*8kB (UEMR) 121*16kB (UEMR) 123*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 127616kB Sep 16 13:14:10 hydra01 kernel: [2093956.358576] 5941722 total pagecache pages Sep 16 13:14:10 hydra01 kernel: [2093956.358578] 425998 pages in swap cache Sep 16 13:14:10 hydra01 kernel: [2093956.358580] Swap cache stats: add 20779433, delete 20353435, find 12663853/14318734 Sep 16 13:14:10 hydra01 kernel: [2093956.358581] Free swap = 185320332kB Sep 16 13:14:10 hydra01 kernel: [2093956.358583] Total swap = 187617276kB Sep 16 13:14:10 hydra01 kernel: [2093956.484141] 8388592 pages RAM Sep 16 13:14:10 hydra01 kernel: [2093956.484144] 151696 pages reserved Sep 16 13:14:10 hydra01 kernel: [2093956.484145] 8939326 pages shared Sep 16 13:14:10 hydra01 kernel: [2093956.484147] 3208886 pages non-shared Sep 16 13:14:12 hydra01 kernel: [2093958.237418] iostat: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:14:12 hydra01 kernel: [2093958.237427] Pid: 106710, comm: iostat Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:14:12 hydra01 kernel: [2093958.237429] Call Trace: Sep 16 13:14:12 hydra01 kernel: [2093958.237443] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:14:12 hydra01 kernel: [2093958.237451] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 13:14:12 hydra01 kernel: [2093958.237457] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:14:12 hydra01 kernel: [2093958.237465] [] alloc_pages_current+0xb8/0x180 Sep 16 13:14:12 hydra01 kernel: [2093958.237470] [] __get_free_pages+0xe/0x50 Sep 16 13:14:12 hydra01 kernel: [2093958.237477] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:14:12 hydra01 kernel: [2093958.237483] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 13:14:12 hydra01 kernel: [2093958.237489] [] __kmalloc+0x161/0x170 Sep 16 13:14:12 hydra01 kernel: [2093958.237495] [] ? get_iowait_time+0x60/0x60 Sep 16 13:14:12 hydra01 kernel: [2093958.237500] [] stat_open+0x56/0xc0 Sep 16 13:14:12 hydra01 kernel: [2093958.237506] [] proc_reg_open+0xa0/0x180 Sep 16 13:14:12 hydra01 kernel: [2093958.237513] [] ? seq_release_private+0x60/0x60 Sep 16 13:14:12 hydra01 kernel: [2093958.237519] [] do_dentry_open+0x203/0x290 Sep 16 13:14:12 hydra01 kernel: [2093958.237524] [] ? init_once+0x20/0x20 Sep 16 13:14:12 hydra01 kernel: [2093958.237528] [] vfs_open+0x49/0x50 Sep 16 13:14:12 hydra01 kernel: [2093958.237533] [] do_last+0x296/0xea0 Sep 16 13:14:12 hydra01 kernel: [2093958.237538] [] ? inode_permission+0x18/0x50 Sep 16 13:14:12 hydra01 kernel: [2093958.237543] [] ? link_path_walk+0x6b/0x880 Sep 16 13:14:12 hydra01 kernel: [2093958.237548] [] path_openat+0xb3/0x4b0 Sep 16 13:14:12 hydra01 kernel: [2093958.237554] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 13:14:12 hydra01 kernel: [2093958.237560] [] ? unmap_region+0xdb/0x120 Sep 16 13:14:12 hydra01 kernel: [2093958.237565] [] do_filp_open+0x41/0xa0 Sep 16 13:14:12 hydra01 kernel: [2093958.237570] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:14:12 hydra01 kernel: [2093958.237575] [] do_sys_open+0xf3/0x230 Sep 16 13:14:12 hydra01 kernel: [2093958.237580] [] sys_open+0x21/0x30 Sep 16 13:14:12 hydra01 kernel: [2093958.237585] [] system_call_fastpath+0x1a/0x1f Sep 16 13:14:12 hydra01 kernel: [2093958.237588] Mem-Info: Sep 16 13:14:12 hydra01 kernel: [2093958.237590] Node 0 DMA per-cpu: Sep 16 13:14:12 hydra01 kernel: [2093958.237595] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237598] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237601] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237603] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237606] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237609] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237611] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237614] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237616] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237619] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237622] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237624] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237626] Node 0 DMA32 per-cpu: Sep 16 13:14:12 hydra01 kernel: [2093958.237630] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237633] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237636] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237638] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237641] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237643] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237646] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237649] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237651] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237654] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237657] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237659] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237661] Node 0 Normal per-cpu: Sep 16 13:14:12 hydra01 kernel: [2093958.237665] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237667] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237670] CPU 2: hi: 186, btch: 31 usd: 170 Sep 16 13:14:12 hydra01 kernel: [2093958.237673] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237675] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237678] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237681] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237683] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237686] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237689] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237691] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237694] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237701] active_anon:1268960 inactive_anon:814154 isolated_anon:0 Sep 16 13:14:12 hydra01 kernel: [2093958.237701] active_file:1841417 inactive_file:3676944 isolated_file:0 Sep 16 13:14:12 hydra01 kernel: [2093958.237701] unevictable:0 dirty:70 writeback:0 unstable:0 Sep 16 13:14:12 hydra01 kernel: [2093958.237701] free:67547 slab_reclaimable:324612 slab_unreclaimable:97038 Sep 16 13:14:12 hydra01 kernel: [2093958.237701] mapped:24681 shmem:151 pagetables:9537 bounce:0 Sep 16 13:14:12 hydra01 kernel: [2093958.237701] free_cma:0 Sep 16 13:14:12 hydra01 kernel: [2093958.237708] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:14:12 hydra01 kernel: [2093958.237717] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:14:12 hydra01 kernel: [2093958.237723] Node 0 DMA32 free:140052kB min:6224kB low:7780kB high:9336kB active_anon:434424kB inactive_anon:792988kB active_file:96984kB inactive_file:1380876kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:12kB writeback:0kB mapped:3004kB shmem:0kB slab_reclaimable:166068kB slab_unreclaimable:38052kB kernel_stack:1128kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:14:12 hydra01 kernel: [2093958.237732] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:14:12 hydra01 kernel: [2093958.237737] Node 0 Normal free:114584kB min:61324kB low:76652kB high:91984kB active_anon:4641416kB inactive_anon:2463628kB active_file:7268684kB inactive_file:13326900kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:268kB writeback:0kB mapped:95720kB shmem:604kB slab_reclaimable:1132380kB slab_unreclaimable:350100kB kernel_stack:7720kB pagetables:36616kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 13:14:12 hydra01 kernel: [2093958.237747] lowmem_reserve[]: 0 0 0 0 Sep 16 13:14:12 hydra01 kernel: [2093958.237752] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:14:12 hydra01 kernel: [2093958.237774] Node 0 DMA32: 2739*4kB (UEM) 2404*8kB (UEM) 5745*16kB (UEM) 556*32kB (UEMR) 3*64kB (UR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140604kB Sep 16 13:14:12 hydra01 kernel: [2093958.237793] Node 0 Normal: 19438*4kB (UEMR) 3826*8kB (UEMR) 121*16kB (UEMR) 123*32kB (UEMR) 10*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 116152kB Sep 16 13:14:12 hydra01 kernel: [2093958.237813] 5944500 total pagecache pages Sep 16 13:14:12 hydra01 kernel: [2093958.237815] 425998 pages in swap cache Sep 16 13:14:12 hydra01 kernel: [2093958.237818] Swap cache stats: add 20779433, delete 20353435, find 12663853/14318734 Sep 16 13:14:12 hydra01 kernel: [2093958.237821] Free swap = 185320332kB Sep 16 13:14:12 hydra01 kernel: [2093958.237823] Total swap = 187617276kB Sep 16 13:14:12 hydra01 kernel: [2093958.413917] 8388592 pages RAM Sep 16 13:14:12 hydra01 kernel: [2093958.413920] 151696 pages reserved Sep 16 13:14:12 hydra01 kernel: [2093958.413922] 8937903 pages shared Sep 16 13:14:12 hydra01 kernel: [2093958.413923] 3211285 pages non-shared Sep 16 13:22:40 hydra01 kernel: [2094466.259887] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:22:40 hydra01 kernel: [2094466.259895] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:22:40 hydra01 kernel: [2094466.259898] Call Trace: Sep 16 13:22:40 hydra01 kernel: [2094466.259912] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:22:40 hydra01 kernel: [2094466.259920] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 13:22:40 hydra01 kernel: [2094466.259927] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:22:40 hydra01 kernel: [2094466.259934] [] alloc_pages_current+0xb8/0x180 Sep 16 13:22:40 hydra01 kernel: [2094466.259940] [] __get_free_pages+0xe/0x50 Sep 16 13:22:40 hydra01 kernel: [2094466.259947] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:22:40 hydra01 kernel: [2094466.259953] [] __kmalloc+0x161/0x170 Sep 16 13:22:40 hydra01 kernel: [2094466.259960] [] ? get_iowait_time+0x60/0x60 Sep 16 13:22:40 hydra01 kernel: [2094466.259964] [] stat_open+0x56/0xc0 Sep 16 13:22:40 hydra01 kernel: [2094466.259986] [] proc_reg_open+0xa0/0x180 Sep 16 13:22:40 hydra01 kernel: [2094466.259994] [] ? seq_release_private+0x60/0x60 Sep 16 13:22:40 hydra01 kernel: [2094466.259999] [] do_dentry_open+0x203/0x290 Sep 16 13:22:40 hydra01 kernel: [2094466.260004] [] ? init_once+0x20/0x20 Sep 16 13:22:40 hydra01 kernel: [2094466.260009] [] vfs_open+0x49/0x50 Sep 16 13:22:40 hydra01 kernel: [2094466.260014] [] do_last+0x296/0xea0 Sep 16 13:22:40 hydra01 kernel: [2094466.260018] [] ? inode_permission+0x18/0x50 Sep 16 13:22:40 hydra01 kernel: [2094466.260023] [] ? link_path_walk+0x6b/0x880 Sep 16 13:22:40 hydra01 kernel: [2094466.260028] [] path_openat+0xb3/0x4b0 Sep 16 13:22:40 hydra01 kernel: [2094466.260033] [] do_filp_open+0x41/0xa0 Sep 16 13:22:40 hydra01 kernel: [2094466.260039] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:22:40 hydra01 kernel: [2094466.260044] [] do_sys_open+0xf3/0x230 Sep 16 13:22:40 hydra01 kernel: [2094466.260049] [] ? sys_select+0xcb/0x110 Sep 16 13:22:40 hydra01 kernel: [2094466.260054] [] sys_open+0x21/0x30 Sep 16 13:22:40 hydra01 kernel: [2094466.260059] [] system_call_fastpath+0x1a/0x1f Sep 16 13:22:40 hydra01 kernel: [2094466.260062] Mem-Info: Sep 16 13:22:40 hydra01 kernel: [2094466.260064] Node 0 DMA per-cpu: Sep 16 13:22:40 hydra01 kernel: [2094466.260069] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260072] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260075] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260077] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260080] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260083] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260085] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260088] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260090] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260093] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260096] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260098] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260100] Node 0 DMA32 per-cpu: Sep 16 13:22:40 hydra01 kernel: [2094466.260105] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260107] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260110] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260113] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260115] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260118] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260121] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260123] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260126] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260129] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260131] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260134] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260136] Node 0 Normal per-cpu: Sep 16 13:22:40 hydra01 kernel: [2094466.260142] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260144] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260147] CPU 2: hi: 186, btch: 31 usd: 184 Sep 16 13:22:40 hydra01 kernel: [2094466.260150] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260152] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260155] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260157] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260160] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260163] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260165] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260168] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260171] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260178] active_anon:1276692 inactive_anon:818657 isolated_anon:0 Sep 16 13:22:40 hydra01 kernel: [2094466.260178] active_file:1916960 inactive_file:3589927 isolated_file:0 Sep 16 13:22:40 hydra01 kernel: [2094466.260178] unevictable:0 dirty:42 writeback:981 unstable:0 Sep 16 13:22:40 hydra01 kernel: [2094466.260178] free:71495 slab_reclaimable:319043 slab_unreclaimable:95179 Sep 16 13:22:40 hydra01 kernel: [2094466.260178] mapped:24509 shmem:151 pagetables:9566 bounce:0 Sep 16 13:22:40 hydra01 kernel: [2094466.260178] free_cma:0 Sep 16 13:22:40 hydra01 kernel: [2094466.260185] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:22:40 hydra01 kernel: [2094466.260194] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:22:40 hydra01 kernel: [2094466.260200] Node 0 DMA32 free:134216kB min:6224kB low:7780kB high:9336kB active_anon:442648kB inactive_anon:793352kB active_file:120924kB inactive_file:1350772kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:4kB writeback:0kB mapped:3040kB shmem:8kB slab_reclaimable:164072kB slab_unreclaimable:36072kB kernel_stack:1168kB pagetables:1544kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:22:40 hydra01 kernel: [2094466.260209] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:22:40 hydra01 kernel: [2094466.260215] Node 0 Normal free:136212kB min:61324kB low:76652kB high:91984kB active_anon:4664120kB inactive_anon:2481276kB active_file:7546916kB inactive_file:13008552kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:164kB writeback:3924kB mapped:94996kB shmem:596kB slab_reclaimable:1112100kB slab_unreclaimable:344644kB kernel_stack:7664kB pagetables:36720kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 13:22:40 hydra01 kernel: [2094466.260224] lowmem_reserve[]: 0 0 0 0 Sep 16 13:22:40 hydra01 kernel: [2094466.260229] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:22:40 hydra01 kernel: [2094466.260251] Node 0 DMA32: 4138*4kB (UEM) 916*8kB (UEM) 5738*16kB (UEM) 579*32kB (UEMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 134856kB Sep 16 13:22:40 hydra01 kernel: [2094466.260270] Node 0 Normal: 16187*4kB (UEMR) 8726*8kB (UEMR) 53*16kB (UEMR) 41*32kB (UEMR) 9*64kB (UER) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 138572kB Sep 16 13:22:40 hydra01 kernel: [2094466.260290] 5931937 total pagecache pages Sep 16 13:22:40 hydra01 kernel: [2094466.260293] 425180 pages in swap cache Sep 16 13:22:40 hydra01 kernel: [2094466.260296] Swap cache stats: add 20781076, delete 20355896, find 12668038/14323773 Sep 16 13:22:40 hydra01 kernel: [2094466.260298] Free swap = 185330068kB Sep 16 13:22:40 hydra01 kernel: [2094466.260300] Total swap = 187617276kB Sep 16 13:22:40 hydra01 kernel: [2094466.451778] 8388592 pages RAM Sep 16 13:22:40 hydra01 kernel: [2094466.451782] 151696 pages reserved Sep 16 13:22:40 hydra01 kernel: [2094466.451785] 8901842 pages shared Sep 16 13:22:40 hydra01 kernel: [2094466.451787] 3233390 pages non-shared Sep 16 13:31:31 hydra01 kernel: [2094996.168565] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:31:31 hydra01 kernel: [2094996.168574] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:31:31 hydra01 kernel: [2094996.168577] Call Trace: Sep 16 13:31:31 hydra01 kernel: [2094996.168590] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:31:31 hydra01 kernel: [2094996.168598] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 13:31:31 hydra01 kernel: [2094996.168604] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:31:31 hydra01 kernel: [2094996.168612] [] alloc_pages_current+0xb8/0x180 Sep 16 13:31:31 hydra01 kernel: [2094996.168617] [] __get_free_pages+0xe/0x50 Sep 16 13:31:31 hydra01 kernel: [2094996.168624] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:31:31 hydra01 kernel: [2094996.168630] [] __kmalloc+0x161/0x170 Sep 16 13:31:31 hydra01 kernel: [2094996.168637] [] ? get_iowait_time+0x60/0x60 Sep 16 13:31:31 hydra01 kernel: [2094996.168641] [] stat_open+0x56/0xc0 Sep 16 13:31:31 hydra01 kernel: [2094996.168648] [] proc_reg_open+0xa0/0x180 Sep 16 13:31:31 hydra01 kernel: [2094996.168655] [] ? seq_release_private+0x60/0x60 Sep 16 13:31:31 hydra01 kernel: [2094996.168660] [] do_dentry_open+0x203/0x290 Sep 16 13:31:31 hydra01 kernel: [2094996.168665] [] ? init_once+0x20/0x20 Sep 16 13:31:31 hydra01 kernel: [2094996.168669] [] vfs_open+0x49/0x50 Sep 16 13:31:31 hydra01 kernel: [2094996.168675] [] do_last+0x296/0xea0 Sep 16 13:31:31 hydra01 kernel: [2094996.168679] [] ? inode_permission+0x18/0x50 Sep 16 13:31:31 hydra01 kernel: [2094996.168684] [] ? link_path_walk+0x6b/0x880 Sep 16 13:31:31 hydra01 kernel: [2094996.168689] [] path_openat+0xb3/0x4b0 Sep 16 13:31:31 hydra01 kernel: [2094996.168694] [] do_filp_open+0x41/0xa0 Sep 16 13:31:31 hydra01 kernel: [2094996.168700] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:31:31 hydra01 kernel: [2094996.168705] [] do_sys_open+0xf3/0x230 Sep 16 13:31:31 hydra01 kernel: [2094996.168710] [] ? sys_select+0xcb/0x110 Sep 16 13:31:31 hydra01 kernel: [2094996.168715] [] sys_open+0x21/0x30 Sep 16 13:31:31 hydra01 kernel: [2094996.168720] [] system_call_fastpath+0x1a/0x1f Sep 16 13:31:31 hydra01 kernel: [2094996.168722] Mem-Info: Sep 16 13:31:31 hydra01 kernel: [2094996.168725] Node 0 DMA per-cpu: Sep 16 13:31:31 hydra01 kernel: [2094996.168730] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168733] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168735] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168738] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168741] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168743] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168746] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168748] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168751] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168754] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168756] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168759] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168761] Node 0 DMA32 per-cpu: Sep 16 13:31:31 hydra01 kernel: [2094996.168765] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168768] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168770] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168773] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168775] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168778] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168781] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168783] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168786] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168789] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168791] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168794] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168796] Node 0 Normal per-cpu: Sep 16 13:31:31 hydra01 kernel: [2094996.168799] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168802] CPU 1: hi: 186, btch: 31 usd: 185 Sep 16 13:31:31 hydra01 kernel: [2094996.168805] CPU 2: hi: 186, btch: 31 usd: 32 Sep 16 13:31:31 hydra01 kernel: [2094996.168807] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168810] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168813] CPU 5: hi: 186, btch: 31 usd: 89 Sep 16 13:31:31 hydra01 kernel: [2094996.168815] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168818] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168821] CPU 8: hi: 186, btch: 31 usd: 140 Sep 16 13:31:31 hydra01 kernel: [2094996.168824] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168826] CPU 10: hi: 186, btch: 31 usd: 135 Sep 16 13:31:31 hydra01 kernel: [2094996.168829] CPU 11: hi: 186, btch: 31 usd: 71 Sep 16 13:31:31 hydra01 kernel: [2094996.168837] active_anon:1281865 inactive_anon:820425 isolated_anon:0 Sep 16 13:31:31 hydra01 kernel: [2094996.168837] active_file:1918765 inactive_file:3534518 isolated_file:0 Sep 16 13:31:31 hydra01 kernel: [2094996.168837] unevictable:0 dirty:77 writeback:0 unstable:0 Sep 16 13:31:31 hydra01 kernel: [2094996.168837] free:63476 slab_reclaimable:319921 slab_unreclaimable:95148 Sep 16 13:31:31 hydra01 kernel: [2094996.168837] mapped:24856 shmem:151 pagetables:9641 bounce:0 Sep 16 13:31:31 hydra01 kernel: [2094996.168837] free_cma:0 Sep 16 13:31:31 hydra01 kernel: [2094996.168843] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:31:31 hydra01 kernel: [2094996.168852] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:31:31 hydra01 kernel: [2094996.168858] Node 0 DMA32 free:140704kB min:6224kB low:7780kB high:9336kB active_anon:445572kB inactive_anon:792824kB active_file:122096kB inactive_file:1324092kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:64kB writeback:0kB mapped:2976kB shmem:12kB slab_reclaimable:164344kB slab_unreclaimable:36484kB kernel_stack:1048kB pagetables:1832kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:31:31 hydra01 kernel: [2094996.168868] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:31:31 hydra01 kernel: [2094996.168873] Node 0 Normal free:97648kB min:61324kB low:76652kB high:91984kB active_anon:4681888kB inactive_anon:2488876kB active_file:7552964kB inactive_file:12813980kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:244kB writeback:0kB mapped:96448kB shmem:592kB slab_reclaimable:1115340kB slab_unreclaimable:344108kB kernel_stack:7768kB pagetables:36732kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 13:31:31 hydra01 kernel: [2094996.168883] lowmem_reserve[]: 0 0 0 0 Sep 16 13:31:31 hydra01 kernel: [2094996.168888] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:31:31 hydra01 kernel: [2094996.168909] Node 0 DMA32: 3604*4kB (UEM) 3456*8kB (UEM) 5042*16kB (UEM) 561*32kB (UEMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141328kB Sep 16 13:31:31 hydra01 kernel: [2094996.168928] Node 0 Normal: 15449*4kB (EMR) 4375*8kB (UMR) 5*16kB (R) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 98732kB Sep 16 13:31:31 hydra01 kernel: [2094996.168948] 5878516 total pagecache pages Sep 16 13:31:31 hydra01 kernel: [2094996.168950] 425130 pages in swap cache Sep 16 13:31:31 hydra01 kernel: [2094996.168953] Swap cache stats: add 20781138, delete 20356008, find 12668113/14323855 Sep 16 13:31:31 hydra01 kernel: [2094996.168955] Free swap = 185330472kB Sep 16 13:31:31 hydra01 kernel: [2094996.168957] Total swap = 187617276kB Sep 16 13:31:31 hydra01 kernel: [2094996.358499] 8388592 pages RAM Sep 16 13:31:31 hydra01 kernel: [2094996.358507] 151696 pages reserved Sep 16 13:31:31 hydra01 kernel: [2094996.358513] 8920751 pages shared Sep 16 13:31:31 hydra01 kernel: [2094996.358520] 3209646 pages non-shared Sep 16 13:36:25 hydra01 kernel: [2095290.105011] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:36:25 hydra01 kernel: [2095290.105022] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:36:25 hydra01 kernel: [2095290.105025] Call Trace: Sep 16 13:36:25 hydra01 kernel: [2095290.105042] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:36:25 hydra01 kernel: [2095290.105052] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 13:36:25 hydra01 kernel: [2095290.105060] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:36:25 hydra01 kernel: [2095290.105069] [] alloc_pages_current+0xb8/0x180 Sep 16 13:36:25 hydra01 kernel: [2095290.105075] [] __get_free_pages+0xe/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.105085] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:36:25 hydra01 kernel: [2095290.105093] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 13:36:25 hydra01 kernel: [2095290.105101] [] __kmalloc+0x161/0x170 Sep 16 13:36:25 hydra01 kernel: [2095290.105111] [] ? setkey+0x81/0x100 Sep 16 13:36:25 hydra01 kernel: [2095290.105121] [] ? get_iowait_time+0x60/0x60 Sep 16 13:36:25 hydra01 kernel: [2095290.105127] [] stat_open+0x56/0xc0 Sep 16 13:36:25 hydra01 kernel: [2095290.105136] [] proc_reg_open+0xa0/0x180 Sep 16 13:36:25 hydra01 kernel: [2095290.105146] [] ? seq_release_private+0x60/0x60 Sep 16 13:36:25 hydra01 kernel: [2095290.105155] [] do_dentry_open+0x203/0x290 Sep 16 13:36:25 hydra01 kernel: [2095290.105163] [] ? init_once+0x20/0x20 Sep 16 13:36:25 hydra01 kernel: [2095290.105169] [] vfs_open+0x49/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.105177] [] do_last+0x296/0xea0 Sep 16 13:36:25 hydra01 kernel: [2095290.105185] [] ? inode_permission+0x18/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.105192] [] ? link_path_walk+0x6b/0x880 Sep 16 13:36:25 hydra01 kernel: [2095290.105199] [] path_openat+0xb3/0x4b0 Sep 16 13:36:25 hydra01 kernel: [2095290.105209] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 13:36:25 hydra01 kernel: [2095290.105218] [] ? unmap_region+0xdb/0x120 Sep 16 13:36:25 hydra01 kernel: [2095290.105226] [] do_filp_open+0x41/0xa0 Sep 16 13:36:25 hydra01 kernel: [2095290.105234] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:36:25 hydra01 kernel: [2095290.105242] [] do_sys_open+0xf3/0x230 Sep 16 13:36:25 hydra01 kernel: [2095290.105249] [] sys_open+0x21/0x30 Sep 16 13:36:25 hydra01 kernel: [2095290.105257] [] system_call_fastpath+0x1a/0x1f Sep 16 13:36:25 hydra01 kernel: [2095290.105262] Mem-Info: Sep 16 13:36:25 hydra01 kernel: [2095290.105266] Node 0 DMA per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.105273] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105277] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105282] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105286] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105290] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105295] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105299] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105304] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105308] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105312] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105317] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105321] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105325] Node 0 DMA32 per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.105331] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105335] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105340] CPU 2: hi: 186, btch: 31 usd: 160 Sep 16 13:36:25 hydra01 kernel: [2095290.105344] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105349] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105353] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105358] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105363] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105367] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105371] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105376] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105381] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105384] Node 0 Normal per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.105389] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105394] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105398] CPU 2: hi: 186, btch: 31 usd: 184 Sep 16 13:36:25 hydra01 kernel: [2095290.105403] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105407] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105412] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105416] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105421] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105426] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105431] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105435] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105440] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105452] active_anon:1279613 inactive_anon:821198 isolated_anon:0 Sep 16 13:36:25 hydra01 kernel: [2095290.105452] active_file:1918781 inactive_file:3511211 isolated_file:0 Sep 16 13:36:25 hydra01 kernel: [2095290.105452] unevictable:0 dirty:23 writeback:0 unstable:0 Sep 16 13:36:25 hydra01 kernel: [2095290.105452] free:64117 slab_reclaimable:319886 slab_unreclaimable:96112 Sep 16 13:36:25 hydra01 kernel: [2095290.105452] mapped:24476 shmem:151 pagetables:9612 bounce:0 Sep 16 13:36:25 hydra01 kernel: [2095290.105452] free_cma:0 Sep 16 13:36:25 hydra01 kernel: [2095290.105462] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:36:25 hydra01 kernel: [2095290.105477] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:36:25 hydra01 kernel: [2095290.105486] Node 0 DMA32 free:133904kB min:6224kB low:7780kB high:9336kB active_anon:445012kB inactive_anon:792816kB active_file:122096kB inactive_file:1318852kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2808kB shmem:0kB slab_reclaimable:164308kB slab_unreclaimable:40308kB kernel_stack:936kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:80 all_unreclaimable? no Sep 16 13:36:25 hydra01 kernel: [2095290.105501] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:36:25 hydra01 kernel: [2095290.105508] Node 0 Normal free:106720kB min:61324kB low:76652kB high:91984kB active_anon:4673440kB inactive_anon:2491976kB active_file:7553028kB inactive_file:12725992kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:92kB writeback:0kB mapped:95096kB shmem:604kB slab_reclaimable:1115236kB slab_unreclaimable:344140kB kernel_stack:7864kB pagetables:36916kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:40 all_unreclaimable? no Sep 16 13:36:25 hydra01 kernel: [2095290.105523] lowmem_reserve[]: 0 0 0 0 Sep 16 13:36:25 hydra01 kernel: [2095290.105531] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:36:25 hydra01 kernel: [2095290.105564] Node 0 DMA32: 4310*4kB (UEM) 3692*8kB (UEM) 5073*16kB (UEM) 185*32kB (UMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 134504kB Sep 16 13:36:25 hydra01 kernel: [2095290.105594] Node 0 Normal: 20325*4kB (UEMR) 3154*8kB (UEMR) 7*16kB (EMR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 108500kB Sep 16 13:36:25 hydra01 kernel: [2095290.105625] 5855340 total pagecache pages Sep 16 13:36:25 hydra01 kernel: [2095290.105629] 425124 pages in swap cache Sep 16 13:36:25 hydra01 kernel: [2095290.105634] Swap cache stats: add 20781138, delete 20356014, find 12668117/14323859 Sep 16 13:36:25 hydra01 kernel: [2095290.105637] Free swap = 185330488kB Sep 16 13:36:25 hydra01 kernel: [2095290.105641] Total swap = 187617276kB Sep 16 13:36:25 hydra01 kernel: [2095290.296627] 8388592 pages RAM Sep 16 13:36:25 hydra01 kernel: [2095290.296640] 151696 pages reserved Sep 16 13:36:25 hydra01 kernel: [2095290.296649] 8906477 pages shared Sep 16 13:36:25 hydra01 kernel: [2095290.296657] 3226718 pages non-shared Sep 16 13:36:25 hydra01 kernel: [2095290.299321] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:36:25 hydra01 kernel: [2095290.299327] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:36:25 hydra01 kernel: [2095290.299330] Call Trace: Sep 16 13:36:25 hydra01 kernel: [2095290.299342] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:36:25 hydra01 kernel: [2095290.299349] [] ? drain_local_pages+0x16/0x20 Sep 16 13:36:25 hydra01 kernel: [2095290.299357] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 13:36:25 hydra01 kernel: [2095290.299363] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:36:25 hydra01 kernel: [2095290.299370] [] alloc_pages_current+0xb8/0x180 Sep 16 13:36:25 hydra01 kernel: [2095290.299376] [] __get_free_pages+0xe/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.299384] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:36:25 hydra01 kernel: [2095290.299392] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 13:36:25 hydra01 kernel: [2095290.299398] [] __kmalloc+0x161/0x170 Sep 16 13:36:25 hydra01 kernel: [2095290.299406] [] ? setkey+0x81/0x100 Sep 16 13:36:25 hydra01 kernel: [2095290.299412] [] ? get_iowait_time+0x60/0x60 Sep 16 13:36:25 hydra01 kernel: [2095290.299417] [] stat_open+0x56/0xc0 Sep 16 13:36:25 hydra01 kernel: [2095290.299423] [] proc_reg_open+0xa0/0x180 Sep 16 13:36:25 hydra01 kernel: [2095290.299430] [] ? seq_release_private+0x60/0x60 Sep 16 13:36:25 hydra01 kernel: [2095290.299435] [] do_dentry_open+0x203/0x290 Sep 16 13:36:25 hydra01 kernel: [2095290.299441] [] ? init_once+0x20/0x20 Sep 16 13:36:25 hydra01 kernel: [2095290.299445] [] vfs_open+0x49/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.299450] [] do_last+0x296/0xea0 Sep 16 13:36:25 hydra01 kernel: [2095290.299455] [] ? inode_permission+0x18/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.299459] [] ? link_path_walk+0x6b/0x880 Sep 16 13:36:25 hydra01 kernel: [2095290.299464] [] path_openat+0xb3/0x4b0 Sep 16 13:36:25 hydra01 kernel: [2095290.299471] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 13:36:25 hydra01 kernel: [2095290.299477] [] ? unmap_region+0xdb/0x120 Sep 16 13:36:25 hydra01 kernel: [2095290.299482] [] do_filp_open+0x41/0xa0 Sep 16 13:36:25 hydra01 kernel: [2095290.299487] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:36:25 hydra01 kernel: [2095290.299492] [] do_sys_open+0xf3/0x230 Sep 16 13:36:25 hydra01 kernel: [2095290.299496] [] sys_open+0x21/0x30 Sep 16 13:36:25 hydra01 kernel: [2095290.299502] [] system_call_fastpath+0x1a/0x1f Sep 16 13:36:25 hydra01 kernel: [2095290.299505] Mem-Info: Sep 16 13:36:25 hydra01 kernel: [2095290.299507] Node 0 DMA per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.299512] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299514] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299517] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299520] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299522] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299525] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299527] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299530] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299532] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299535] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299538] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299540] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299543] Node 0 DMA32 per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.299546] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299549] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299552] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299554] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299557] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299560] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299562] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299565] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299568] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299570] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299573] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299576] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299577] Node 0 Normal per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.299581] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299584] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299587] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299589] CPU 3: hi: 186, btch: 31 usd: 186 Sep 16 13:36:25 hydra01 kernel: [2095290.299592] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299595] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299597] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299600] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299603] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299605] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299608] CPU 10: hi: 186, btch: 31 usd: 35 Sep 16 13:36:25 hydra01 kernel: [2095290.299611] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299618] active_anon:1279183 inactive_anon:821774 isolated_anon:0 Sep 16 13:36:25 hydra01 kernel: [2095290.299618] active_file:1918781 inactive_file:3493831 isolated_file:0 Sep 16 13:36:25 hydra01 kernel: [2095290.299618] unevictable:0 dirty:23 writeback:0 unstable:0 Sep 16 13:36:25 hydra01 kernel: [2095290.299618] free:81990 slab_reclaimable:319886 slab_unreclaimable:96112 Sep 16 13:36:25 hydra01 kernel: [2095290.299618] mapped:24476 shmem:151 pagetables:9612 bounce:0 Sep 16 13:36:25 hydra01 kernel: [2095290.299618] free_cma:0 Sep 16 13:36:25 hydra01 kernel: [2095290.299624] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:36:25 hydra01 kernel: [2095290.299633] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:36:25 hydra01 kernel: [2095290.299639] Node 0 DMA32 free:135092kB min:6224kB low:7780kB high:9336kB active_anon:445012kB inactive_anon:792816kB active_file:122096kB inactive_file:1318596kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2808kB shmem:0kB slab_reclaimable:164308kB slab_unreclaimable:40308kB kernel_stack:936kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:36:25 hydra01 kernel: [2095290.299649] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:36:25 hydra01 kernel: [2095290.299654] Node 0 Normal free:177316kB min:61324kB low:76652kB high:91984kB active_anon:4671720kB inactive_anon:2494280kB active_file:7553028kB inactive_file:12656728kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:92kB writeback:0kB mapped:95096kB shmem:604kB slab_reclaimable:1115236kB slab_unreclaimable:344140kB kernel_stack:7864kB pagetables:36916kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 13:36:25 hydra01 kernel: [2095290.299664] lowmem_reserve[]: 0 0 0 0 Sep 16 13:36:25 hydra01 kernel: [2095290.299669] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:36:25 hydra01 kernel: [2095290.299690] Node 0 DMA32: 4468*4kB (UEM) 3730*8kB (UEM) 5072*16kB (UEM) 187*32kB (UMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 135488kB Sep 16 13:36:25 hydra01 kernel: [2095290.299710] Node 0 Normal: 22540*4kB (UEMR) 10399*8kB (UEMR) 63*16kB (UEMR) 68*32kB (UEMR) 12*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 178584kB Sep 16 13:36:25 hydra01 kernel: [2095290.299730] 5837911 total pagecache pages Sep 16 13:36:25 hydra01 kernel: [2095290.299732] 425124 pages in swap cache Sep 16 13:36:25 hydra01 kernel: [2095290.299735] Swap cache stats: add 20781138, delete 20356014, find 12668117/14323859 Sep 16 13:36:25 hydra01 kernel: [2095290.299738] Free swap = 185330488kB Sep 16 13:36:25 hydra01 kernel: [2095290.299740] Total swap = 187617276kB Sep 16 13:36:25 hydra01 kernel: [2095290.493215] 8388592 pages RAM Sep 16 13:36:25 hydra01 kernel: [2095290.493219] 151696 pages reserved Sep 16 13:36:25 hydra01 kernel: [2095290.493222] 8903124 pages shared Sep 16 13:36:25 hydra01 kernel: [2095290.493224] 3227181 pages non-shared Sep 16 13:36:25 hydra01 kernel: [2095290.498151] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:36:25 hydra01 kernel: [2095290.498155] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:36:25 hydra01 kernel: [2095290.498156] Call Trace: Sep 16 13:36:25 hydra01 kernel: [2095290.498165] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:36:25 hydra01 kernel: [2095290.498169] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 13:36:25 hydra01 kernel: [2095290.498173] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:36:25 hydra01 kernel: [2095290.498177] [] alloc_pages_current+0xb8/0x180 Sep 16 13:36:25 hydra01 kernel: [2095290.498180] [] __get_free_pages+0xe/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.498184] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:36:25 hydra01 kernel: [2095290.498188] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 13:36:25 hydra01 kernel: [2095290.498191] [] __kmalloc+0x161/0x170 Sep 16 13:36:25 hydra01 kernel: [2095290.498195] [] ? amiga_partition+0x2c6/0x4d0 Sep 16 13:36:25 hydra01 kernel: [2095290.498198] [] ? get_iowait_time+0x60/0x60 Sep 16 13:36:25 hydra01 kernel: [2095290.498201] [] stat_open+0x56/0xc0 Sep 16 13:36:25 hydra01 kernel: [2095290.498204] [] proc_reg_open+0xa0/0x180 Sep 16 13:36:25 hydra01 kernel: [2095290.498208] [] ? seq_release_private+0x60/0x60 Sep 16 13:36:25 hydra01 kernel: [2095290.498212] [] do_dentry_open+0x203/0x290 Sep 16 13:36:25 hydra01 kernel: [2095290.498214] [] ? init_once+0x20/0x20 Sep 16 13:36:25 hydra01 kernel: [2095290.498217] [] vfs_open+0x49/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.498219] [] do_last+0x296/0xea0 Sep 16 13:36:25 hydra01 kernel: [2095290.498222] [] ? inode_permission+0x18/0x50 Sep 16 13:36:25 hydra01 kernel: [2095290.498224] [] ? link_path_walk+0x6b/0x880 Sep 16 13:36:25 hydra01 kernel: [2095290.498227] [] path_openat+0xb3/0x4b0 Sep 16 13:36:25 hydra01 kernel: [2095290.498231] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 13:36:25 hydra01 kernel: [2095290.498234] [] ? unmap_region+0xdb/0x120 Sep 16 13:36:25 hydra01 kernel: [2095290.498237] [] do_filp_open+0x41/0xa0 Sep 16 13:36:25 hydra01 kernel: [2095290.498240] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:36:25 hydra01 kernel: [2095290.498243] [] do_sys_open+0xf3/0x230 Sep 16 13:36:25 hydra01 kernel: [2095290.498245] [] sys_open+0x21/0x30 Sep 16 13:36:25 hydra01 kernel: [2095290.498248] [] system_call_fastpath+0x1a/0x1f Sep 16 13:36:25 hydra01 kernel: [2095290.498249] Mem-Info: Sep 16 13:36:25 hydra01 kernel: [2095290.498250] Node 0 DMA per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.498253] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498255] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498256] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498257] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498259] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498260] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498261] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498263] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498264] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498265] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498267] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498268] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498269] Node 0 DMA32 per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.498271] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498273] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498274] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498275] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498277] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498278] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498280] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498281] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498282] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498284] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498285] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498286] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498287] Node 0 Normal per-cpu: Sep 16 13:36:25 hydra01 kernel: [2095290.498289] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498291] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498292] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498293] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498295] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498296] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498298] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498299] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498300] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498302] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498303] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498304] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498308] active_anon:1279183 inactive_anon:821774 isolated_anon:0 Sep 16 13:36:25 hydra01 kernel: [2095290.498308] active_file:1918781 inactive_file:3493351 isolated_file:0 Sep 16 13:36:25 hydra01 kernel: [2095290.498308] unevictable:0 dirty:23 writeback:0 unstable:0 Sep 16 13:36:25 hydra01 kernel: [2095290.498308] free:82765 slab_reclaimable:319886 slab_unreclaimable:96112 Sep 16 13:36:25 hydra01 kernel: [2095290.498308] mapped:24476 shmem:151 pagetables:9612 bounce:0 Sep 16 13:36:25 hydra01 kernel: [2095290.498308] free_cma:0 Sep 16 13:36:25 hydra01 kernel: [2095290.498312] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:36:25 hydra01 kernel: [2095290.498316] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:36:25 hydra01 kernel: [2095290.498320] Node 0 DMA32 free:135092kB min:6224kB low:7780kB high:9336kB active_anon:445012kB inactive_anon:792816kB active_file:122096kB inactive_file:1318596kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2808kB shmem:0kB slab_reclaimable:164308kB slab_unreclaimable:40308kB kernel_stack:936kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:36:25 hydra01 kernel: [2095290.498325] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:36:25 hydra01 kernel: [2095290.498327] Node 0 Normal free:180124kB min:61324kB low:76652kB high:91984kB active_anon:4671720kB inactive_anon:2494280kB active_file:7553028kB inactive_file:12654808kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:92kB writeback:0kB mapped:95096kB shmem:604kB slab_reclaimable:1115236kB slab_unreclaimable:344140kB kernel_stack:7864kB pagetables:36916kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:36:25 hydra01 kernel: [2095290.498332] lowmem_reserve[]: 0 0 0 0 Sep 16 13:36:25 hydra01 kernel: [2095290.498335] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:36:25 hydra01 kernel: [2095290.498346] Node 0 DMA32: 4468*4kB (UEM) 3730*8kB (UEM) 5072*16kB (UEM) 187*32kB (UMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 135488kB Sep 16 13:36:25 hydra01 kernel: [2095290.498356] Node 0 Normal: 22821*4kB (UEMR) 10634*8kB (UEMR) 63*16kB (UEMR) 68*32kB (UEMR) 12*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 181588kB Sep 16 13:36:25 hydra01 kernel: [2095290.498367] 5837475 total pagecache pages Sep 16 13:36:25 hydra01 kernel: [2095290.498368] 425124 pages in swap cache Sep 16 13:36:25 hydra01 kernel: [2095290.498370] Swap cache stats: add 20781138, delete 20356014, find 12668117/14323859 Sep 16 13:36:25 hydra01 kernel: [2095290.498371] Free swap = 185330488kB Sep 16 13:36:25 hydra01 kernel: [2095290.498372] Total swap = 187617276kB Sep 16 13:36:25 hydra01 kernel: [2095290.604367] 8388592 pages RAM Sep 16 13:36:25 hydra01 kernel: [2095290.604374] 151696 pages reserved Sep 16 13:36:25 hydra01 kernel: [2095290.604375] 8903103 pages shared Sep 16 13:36:25 hydra01 kernel: [2095290.604376] 3227177 pages non-shared Sep 16 13:53:56 hydra01 kernel: [2096340.609213] vnstatd: page allocation failure: order:4, mode:0x1040d0 Sep 16 13:53:56 hydra01 kernel: [2096340.609218] Pid: 6330, comm: vnstatd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 13:53:56 hydra01 kernel: [2096340.609220] Call Trace: Sep 16 13:53:56 hydra01 kernel: [2096340.609228] [] warn_alloc_failed+0xe9/0x140 Sep 16 13:53:56 hydra01 kernel: [2096340.609232] [] ? drain_local_pages+0x16/0x20 Sep 16 13:53:56 hydra01 kernel: [2096340.609236] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 13:53:56 hydra01 kernel: [2096340.609240] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 13:53:56 hydra01 kernel: [2096340.609244] [] alloc_pages_current+0xb8/0x180 Sep 16 13:53:56 hydra01 kernel: [2096340.609247] [] __get_free_pages+0xe/0x50 Sep 16 13:53:56 hydra01 kernel: [2096340.609251] [] kmalloc_order_trace+0x39/0xb0 Sep 16 13:53:56 hydra01 kernel: [2096340.609255] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 13:53:56 hydra01 kernel: [2096340.609258] [] __kmalloc+0x161/0x170 Sep 16 13:53:56 hydra01 kernel: [2096340.609263] [] ? setkey+0x81/0x100 Sep 16 13:53:56 hydra01 kernel: [2096340.609267] [] ? get_iowait_time+0x60/0x60 Sep 16 13:53:56 hydra01 kernel: [2096340.609269] [] stat_open+0x56/0xc0 Sep 16 13:53:56 hydra01 kernel: [2096340.609273] [] proc_reg_open+0xa0/0x180 Sep 16 13:53:56 hydra01 kernel: [2096340.609277] [] ? seq_release_private+0x60/0x60 Sep 16 13:53:56 hydra01 kernel: [2096340.609280] [] do_dentry_open+0x203/0x290 Sep 16 13:53:56 hydra01 kernel: [2096340.609283] [] ? init_once+0x20/0x20 Sep 16 13:53:56 hydra01 kernel: [2096340.609285] [] vfs_open+0x49/0x50 Sep 16 13:53:56 hydra01 kernel: [2096340.609288] [] do_last+0x296/0xea0 Sep 16 13:53:56 hydra01 kernel: [2096340.609290] [] ? inode_permission+0x18/0x50 Sep 16 13:53:56 hydra01 kernel: [2096340.609293] [] ? link_path_walk+0x6b/0x880 Sep 16 13:53:56 hydra01 kernel: [2096340.609295] [] path_openat+0xb3/0x4b0 Sep 16 13:53:56 hydra01 kernel: [2096340.609299] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 13:53:56 hydra01 kernel: [2096340.609303] [] ? unmap_region+0xdb/0x120 Sep 16 13:53:56 hydra01 kernel: [2096340.609305] [] do_filp_open+0x41/0xa0 Sep 16 13:53:56 hydra01 kernel: [2096340.609308] [] ? __alloc_fd+0xd9/0x130 Sep 16 13:53:56 hydra01 kernel: [2096340.609311] [] do_sys_open+0xf3/0x230 Sep 16 13:53:56 hydra01 kernel: [2096340.609313] [] sys_open+0x21/0x30 Sep 16 13:53:56 hydra01 kernel: [2096340.609316] [] system_call_fastpath+0x1a/0x1f Sep 16 13:53:56 hydra01 kernel: [2096340.609318] Mem-Info: Sep 16 13:53:56 hydra01 kernel: [2096340.609319] Node 0 DMA per-cpu: Sep 16 13:53:56 hydra01 kernel: [2096340.609322] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609323] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609324] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609326] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609327] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609328] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609330] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609331] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609332] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609334] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609335] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609337] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609338] Node 0 DMA32 per-cpu: Sep 16 13:53:56 hydra01 kernel: [2096340.609340] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609341] CPU 1: hi: 186, btch: 31 usd: 70 Sep 16 13:53:56 hydra01 kernel: [2096340.609343] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609344] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609345] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609347] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609348] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609349] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609351] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609352] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609354] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609355] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609356] Node 0 Normal per-cpu: Sep 16 13:53:56 hydra01 kernel: [2096340.609358] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609359] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609360] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609362] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609363] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609365] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609366] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609367] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609369] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609370] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609371] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609373] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609377] active_anon:1285570 inactive_anon:824683 isolated_anon:0 Sep 16 13:53:56 hydra01 kernel: [2096340.609377] active_file:2254105 inactive_file:2941228 isolated_file:0 Sep 16 13:53:56 hydra01 kernel: [2096340.609377] unevictable:0 dirty:129 writeback:0 unstable:0 Sep 16 13:53:56 hydra01 kernel: [2096340.609377] free:173761 slab_reclaimable:325124 slab_unreclaimable:105569 Sep 16 13:53:56 hydra01 kernel: [2096340.609377] mapped:24603 shmem:151 pagetables:9575 bounce:0 Sep 16 13:53:56 hydra01 kernel: [2096340.609377] free_cma:0 Sep 16 13:53:56 hydra01 kernel: [2096340.609380] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 13:53:56 hydra01 kernel: [2096340.609385] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 13:53:56 hydra01 kernel: [2096340.609388] Node 0 DMA32 free:167384kB min:6224kB low:7780kB high:9336kB active_anon:443420kB inactive_anon:792800kB active_file:226292kB inactive_file:1160968kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:16kB writeback:0kB mapped:2808kB shmem:0kB slab_reclaimable:164088kB slab_unreclaimable:52500kB kernel_stack:848kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:70 all_unreclaimable? no Sep 16 13:53:56 hydra01 kernel: [2096340.609393] lowmem_reserve[]: 0 0 29232 29232 Sep 16 13:53:56 hydra01 kernel: [2096340.609396] Node 0 Normal free:511816kB min:61324kB low:76652kB high:91984kB active_anon:4698860kB inactive_anon:2505932kB active_file:8790128kB inactive_file:10603944kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:500kB writeback:0kB mapped:95604kB shmem:604kB slab_reclaimable:1136408kB slab_unreclaimable:369776kB kernel_stack:7984kB pagetables:36768kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 13:53:56 hydra01 kernel: [2096340.609401] lowmem_reserve[]: 0 0 0 0 Sep 16 13:53:56 hydra01 kernel: [2096340.609403] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 13:53:56 hydra01 kernel: [2096340.609415] Node 0 DMA32: 4178*4kB (UEM) 10873*8kB (UEM) 3982*16kB (UM) 3*32kB (UMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 168144kB Sep 16 13:53:56 hydra01 kernel: [2096340.609425] Node 0 Normal: 120488*4kB (UEMR) 1128*8kB (UEMR) 619*16kB (UEMR) 179*32kB (UEMR) 81*64kB (UMR) 3*128kB (MR) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 513200kB Sep 16 13:53:56 hydra01 kernel: [2096340.609435] 5620378 total pagecache pages Sep 16 13:53:56 hydra01 kernel: [2096340.609437] 424980 pages in swap cache Sep 16 13:53:56 hydra01 kernel: [2096340.609438] Swap cache stats: add 20781241, delete 20356261, find 12668164/14323913 Sep 16 13:53:56 hydra01 kernel: [2096340.609440] Free swap = 185331068kB Sep 16 13:53:56 hydra01 kernel: [2096340.609441] Total swap = 187617276kB Sep 16 13:53:56 hydra01 kernel: [2096340.744136] 8388592 pages RAM Sep 16 13:53:56 hydra01 kernel: [2096340.744144] 151696 pages reserved Sep 16 13:53:56 hydra01 kernel: [2096340.744152] 8704281 pages shared Sep 16 13:53:56 hydra01 kernel: [2096340.744158] 3266668 pages non-shared Sep 16 14:00:34 hydra01 kernel: [2096738.079701] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:00:34 hydra01 kernel: [2096738.079709] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:00:34 hydra01 kernel: [2096738.079712] Call Trace: Sep 16 14:00:34 hydra01 kernel: [2096738.079725] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:00:34 hydra01 kernel: [2096738.079733] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 14:00:34 hydra01 kernel: [2096738.079739] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:00:34 hydra01 kernel: [2096738.079747] [] alloc_pages_current+0xb8/0x180 Sep 16 14:00:34 hydra01 kernel: [2096738.079752] [] __get_free_pages+0xe/0x50 Sep 16 14:00:34 hydra01 kernel: [2096738.079759] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:00:34 hydra01 kernel: [2096738.079765] [] __kmalloc+0x161/0x170 Sep 16 14:00:34 hydra01 kernel: [2096738.079772] [] ? get_iowait_time+0x60/0x60 Sep 16 14:00:34 hydra01 kernel: [2096738.079776] [] stat_open+0x56/0xc0 Sep 16 14:00:34 hydra01 kernel: [2096738.079783] [] proc_reg_open+0xa0/0x180 Sep 16 14:00:34 hydra01 kernel: [2096738.079790] [] ? seq_release_private+0x60/0x60 Sep 16 14:00:34 hydra01 kernel: [2096738.079795] [] do_dentry_open+0x203/0x290 Sep 16 14:00:34 hydra01 kernel: [2096738.079800] [] ? init_once+0x20/0x20 Sep 16 14:00:34 hydra01 kernel: [2096738.079805] [] vfs_open+0x49/0x50 Sep 16 14:00:34 hydra01 kernel: [2096738.079810] [] do_last+0x296/0xea0 Sep 16 14:00:34 hydra01 kernel: [2096738.079815] [] ? inode_permission+0x18/0x50 Sep 16 14:00:34 hydra01 kernel: [2096738.079819] [] ? link_path_walk+0x6b/0x880 Sep 16 14:00:34 hydra01 kernel: [2096738.079824] [] path_openat+0xb3/0x4b0 Sep 16 14:00:34 hydra01 kernel: [2096738.079831] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 14:00:34 hydra01 kernel: [2096738.079837] [] ? unmap_region+0xdb/0x120 Sep 16 14:00:34 hydra01 kernel: [2096738.079842] [] do_filp_open+0x41/0xa0 Sep 16 14:00:34 hydra01 kernel: [2096738.079847] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:00:34 hydra01 kernel: [2096738.079852] [] do_sys_open+0xf3/0x230 Sep 16 14:00:34 hydra01 kernel: [2096738.079856] [] sys_open+0x21/0x30 Sep 16 14:00:34 hydra01 kernel: [2096738.079862] [] system_call_fastpath+0x1a/0x1f Sep 16 14:00:34 hydra01 kernel: [2096738.079865] Mem-Info: Sep 16 14:00:34 hydra01 kernel: [2096738.079867] Node 0 DMA per-cpu: Sep 16 14:00:34 hydra01 kernel: [2096738.079872] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079875] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079877] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079880] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079882] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079885] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079888] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079890] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079893] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079895] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079898] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079901] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079903] Node 0 DMA32 per-cpu: Sep 16 14:00:34 hydra01 kernel: [2096738.079906] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079909] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079912] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079915] CPU 3: hi: 186, btch: 31 usd: 3 Sep 16 14:00:34 hydra01 kernel: [2096738.079917] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079920] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079922] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079925] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079928] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079930] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079933] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079936] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079938] Node 0 Normal per-cpu: Sep 16 14:00:34 hydra01 kernel: [2096738.079941] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079944] CPU 1: hi: 186, btch: 31 usd: 29 Sep 16 14:00:34 hydra01 kernel: [2096738.079946] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079949] CPU 3: hi: 186, btch: 31 usd: 171 Sep 16 14:00:34 hydra01 kernel: [2096738.079952] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079955] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079957] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079960] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079962] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079965] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079968] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079970] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:00:34 hydra01 kernel: [2096738.079978] active_anon:1259537 inactive_anon:819468 isolated_anon:0 Sep 16 14:00:34 hydra01 kernel: [2096738.079978] active_file:2502186 inactive_file:2796965 isolated_file:0 Sep 16 14:00:34 hydra01 kernel: [2096738.079978] unevictable:0 dirty:20 writeback:0 unstable:0 Sep 16 14:00:34 hydra01 kernel: [2096738.079978] free:63662 slab_reclaimable:330690 slab_unreclaimable:108576 Sep 16 14:00:34 hydra01 kernel: [2096738.079978] mapped:25405 shmem:151 pagetables:9597 bounce:0 Sep 16 14:00:34 hydra01 kernel: [2096738.079978] free_cma:0 Sep 16 14:00:34 hydra01 kernel: [2096738.079984] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:00:34 hydra01 kernel: [2096738.079993] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:00:34 hydra01 kernel: [2096738.079999] Node 0 DMA32 free:140476kB min:6224kB low:7780kB high:9336kB active_anon:435232kB inactive_anon:792796kB active_file:286528kB inactive_file:1134980kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:2912kB shmem:0kB slab_reclaimable:166848kB slab_unreclaimable:55700kB kernel_stack:864kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:00:34 hydra01 kernel: [2096738.080009] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:00:34 hydra01 kernel: [2096738.080014] Node 0 Normal free:98328kB min:61324kB low:76652kB high:91984kB active_anon:4602916kB inactive_anon:2485076kB active_file:9722216kB inactive_file:10052476kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:80kB writeback:0kB mapped:98708kB shmem:604kB slab_reclaimable:1155912kB slab_unreclaimable:378604kB kernel_stack:7976kB pagetables:36856kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:00:34 hydra01 kernel: [2096738.080023] lowmem_reserve[]: 0 0 0 0 Sep 16 14:00:34 hydra01 kernel: [2096738.080028] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:00:34 hydra01 kernel: [2096738.080050] Node 0 DMA32: 2078*4kB (UEM) 8838*8kB (UEM) 3835*16kB (UM) 10*32kB (MR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 141336kB Sep 16 14:00:34 hydra01 kernel: [2096738.080069] Node 0 Normal: 12051*4kB (UEMR) 1509*8kB (UEMR) 1806*16kB (UER) 302*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 100500kB Sep 16 14:00:34 hydra01 kernel: [2096738.080090] 5724083 total pagecache pages Sep 16 14:00:34 hydra01 kernel: [2096738.080092] 424892 pages in swap cache Sep 16 14:00:34 hydra01 kernel: [2096738.080095] Swap cache stats: add 20781524, delete 20356632, find 12668188/14323939 Sep 16 14:00:34 hydra01 kernel: [2096738.080098] Free swap = 185330272kB Sep 16 14:00:34 hydra01 kernel: [2096738.080100] Total swap = 187617276kB Sep 16 14:00:34 hydra01 kernel: [2096738.262375] 8388592 pages RAM Sep 16 14:00:34 hydra01 kernel: [2096738.262380] 151696 pages reserved Sep 16 14:00:34 hydra01 kernel: [2096738.262382] 8624013 pages shared Sep 16 14:00:34 hydra01 kernel: [2096738.262395] 3529244 pages non-shared Sep 16 14:01:34 hydra01 kernel: [2096798.039948] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:01:34 hydra01 kernel: [2096798.039957] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:01:34 hydra01 kernel: [2096798.039960] Call Trace: Sep 16 14:01:34 hydra01 kernel: [2096798.039974] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:01:34 hydra01 kernel: [2096798.039982] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 14:01:34 hydra01 kernel: [2096798.039988] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:01:34 hydra01 kernel: [2096798.039995] [] alloc_pages_current+0xb8/0x180 Sep 16 14:01:34 hydra01 kernel: [2096798.040001] [] __get_free_pages+0xe/0x50 Sep 16 14:01:34 hydra01 kernel: [2096798.040008] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:01:34 hydra01 kernel: [2096798.040014] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 14:01:34 hydra01 kernel: [2096798.040020] [] __kmalloc+0x161/0x170 Sep 16 14:01:34 hydra01 kernel: [2096798.040027] [] ? get_iowait_time+0x60/0x60 Sep 16 14:01:34 hydra01 kernel: [2096798.040031] [] stat_open+0x56/0xc0 Sep 16 14:01:34 hydra01 kernel: [2096798.040038] [] proc_reg_open+0xa0/0x180 Sep 16 14:01:34 hydra01 kernel: [2096798.040045] [] ? seq_release_private+0x60/0x60 Sep 16 14:01:34 hydra01 kernel: [2096798.040050] [] do_dentry_open+0x203/0x290 Sep 16 14:01:34 hydra01 kernel: [2096798.040056] [] ? init_once+0x20/0x20 Sep 16 14:01:34 hydra01 kernel: [2096798.040060] [] vfs_open+0x49/0x50 Sep 16 14:01:34 hydra01 kernel: [2096798.040065] [] do_last+0x296/0xea0 Sep 16 14:01:34 hydra01 kernel: [2096798.040070] [] ? inode_permission+0x18/0x50 Sep 16 14:01:34 hydra01 kernel: [2096798.040074] [] ? link_path_walk+0x6b/0x880 Sep 16 14:01:34 hydra01 kernel: [2096798.040079] [] path_openat+0xb3/0x4b0 Sep 16 14:01:34 hydra01 kernel: [2096798.040085] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 14:01:34 hydra01 kernel: [2096798.040091] [] ? unmap_region+0xdb/0x120 Sep 16 14:01:34 hydra01 kernel: [2096798.040096] [] do_filp_open+0x41/0xa0 Sep 16 14:01:34 hydra01 kernel: [2096798.040102] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:01:34 hydra01 kernel: [2096798.040107] [] do_sys_open+0xf3/0x230 Sep 16 14:01:34 hydra01 kernel: [2096798.040111] [] sys_open+0x21/0x30 Sep 16 14:01:34 hydra01 kernel: [2096798.040117] [] system_call_fastpath+0x1a/0x1f Sep 16 14:01:34 hydra01 kernel: [2096798.040119] Mem-Info: Sep 16 14:01:34 hydra01 kernel: [2096798.040122] Node 0 DMA per-cpu: Sep 16 14:01:34 hydra01 kernel: [2096798.040127] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040130] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040132] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040135] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040137] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040140] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040142] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040145] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040148] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040150] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040153] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040156] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040158] Node 0 DMA32 per-cpu: Sep 16 14:01:34 hydra01 kernel: [2096798.040162] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040164] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040167] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040170] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040172] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040175] CPU 5: hi: 186, btch: 31 usd: 70 Sep 16 14:01:34 hydra01 kernel: [2096798.040178] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040180] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040183] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040186] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040188] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040191] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040193] Node 0 Normal per-cpu: Sep 16 14:01:34 hydra01 kernel: [2096798.040197] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040199] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040202] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040205] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040207] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040210] CPU 5: hi: 186, btch: 31 usd: 174 Sep 16 14:01:34 hydra01 kernel: [2096798.040213] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040215] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040218] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040221] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040223] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040226] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040234] active_anon:1265930 inactive_anon:815668 isolated_anon:0 Sep 16 14:01:34 hydra01 kernel: [2096798.040234] active_file:2504274 inactive_file:2821325 isolated_file:0 Sep 16 14:01:34 hydra01 kernel: [2096798.040234] unevictable:0 dirty:53 writeback:0 unstable:0 Sep 16 14:01:34 hydra01 kernel: [2096798.040234] free:63356 slab_reclaimable:329535 slab_unreclaimable:108588 Sep 16 14:01:34 hydra01 kernel: [2096798.040234] mapped:26687 shmem:151 pagetables:9614 bounce:0 Sep 16 14:01:34 hydra01 kernel: [2096798.040234] free_cma:0 Sep 16 14:01:34 hydra01 kernel: [2096798.040240] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:01:34 hydra01 kernel: [2096798.040249] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:01:34 hydra01 kernel: [2096798.040255] Node 0 DMA32 free:130980kB min:6224kB low:7780kB high:9336kB active_anon:448424kB inactive_anon:793032kB active_file:286532kB inactive_file:1143788kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:48kB writeback:0kB mapped:3108kB shmem:0kB slab_reclaimable:166224kB slab_unreclaimable:55684kB kernel_stack:912kB pagetables:1532kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:69 all_unreclaimable? no Sep 16 14:01:34 hydra01 kernel: [2096798.040265] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:01:34 hydra01 kernel: [2096798.040270] Node 0 Normal free:106892kB min:61324kB low:76652kB high:91984kB active_anon:4615296kB inactive_anon:2469640kB active_file:9730564kB inactive_file:10141512kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:164kB writeback:0kB mapped:103640kB shmem:604kB slab_reclaimable:1151916kB slab_unreclaimable:378668kB kernel_stack:7976kB pagetables:36924kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:01:34 hydra01 kernel: [2096798.040280] lowmem_reserve[]: 0 0 0 0 Sep 16 14:01:34 hydra01 kernel: [2096798.040284] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:01:34 hydra01 kernel: [2096798.040306] Node 0 DMA32: 2936*4kB (UEM) 6770*8kB (UEM) 4017*16kB (UEM) 33*32kB (MR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 131872kB Sep 16 14:01:34 hydra01 kernel: [2096798.040325] Node 0 Normal: 19779*4kB (UEMR) 3471*8kB (UEMR) 8*16kB (UR) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 108868kB Sep 16 14:01:34 hydra01 kernel: [2096798.040345] 5749021 total pagecache pages Sep 16 14:01:34 hydra01 kernel: [2096798.040347] 423399 pages in swap cache Sep 16 14:01:34 hydra01 kernel: [2096798.040350] Swap cache stats: add 20785818, delete 20362419, find 12669734/14325856 Sep 16 14:01:34 hydra01 kernel: [2096798.040353] Free swap = 185320008kB Sep 16 14:01:34 hydra01 kernel: [2096798.040355] Total swap = 187617276kB Sep 16 14:01:34 hydra01 kernel: [2096798.236879] 8388592 pages RAM Sep 16 14:01:34 hydra01 kernel: [2096798.236886] 151696 pages reserved Sep 16 14:01:34 hydra01 kernel: [2096798.236891] 8244616 pages shared Sep 16 14:01:34 hydra01 kernel: [2096798.236897] 3901746 pages non-shared Sep 16 14:09:44 hydra01 kernel: [2097287.717897] iostat: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:09:44 hydra01 kernel: [2097287.717901] Pid: 164468, comm: iostat Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:09:44 hydra01 kernel: [2097287.717903] Call Trace: Sep 16 14:09:44 hydra01 kernel: [2097287.717912] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:09:44 hydra01 kernel: [2097287.717916] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 14:09:44 hydra01 kernel: [2097287.717920] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:09:44 hydra01 kernel: [2097287.717922] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:09:44 hydra01 kernel: [2097287.717924] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:09:44 hydra01 kernel: [2097287.717925] Call Trace: Sep 16 14:09:44 hydra01 kernel: [2097287.717931] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:09:44 hydra01 kernel: [2097287.717936] [] alloc_pages_current+0xb8/0x180 Sep 16 14:09:44 hydra01 kernel: [2097287.717940] [] ? drain_local_pages+0x16/0x20 Sep 16 14:09:44 hydra01 kernel: [2097287.717943] [] __get_free_pages+0xe/0x50 Sep 16 14:09:44 hydra01 kernel: [2097287.717948] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 14:09:44 hydra01 kernel: [2097287.717953] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:09:44 hydra01 kernel: [2097287.717956] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:09:44 hydra01 kernel: [2097287.717960] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 14:09:44 hydra01 kernel: [2097287.717964] [] __kmalloc+0x161/0x170 Sep 16 14:09:44 hydra01 kernel: [2097287.717968] [] ? get_iowait_time+0x60/0x60 Sep 16 14:09:44 hydra01 kernel: [2097287.717971] [] alloc_pages_current+0xb8/0x180 Sep 16 14:09:44 hydra01 kernel: [2097287.717974] [] __get_free_pages+0xe/0x50 Sep 16 14:09:44 hydra01 kernel: [2097287.717977] [] stat_open+0x56/0xc0 Sep 16 14:09:44 hydra01 kernel: [2097287.717981] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:09:44 hydra01 kernel: [2097287.717985] [] proc_reg_open+0xa0/0x180 Sep 16 14:09:44 hydra01 kernel: [2097287.717988] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 14:09:44 hydra01 kernel: [2097287.717992] [] ? seq_release_private+0x60/0x60 Sep 16 14:09:44 hydra01 kernel: [2097287.717995] [] __kmalloc+0x161/0x170 Sep 16 14:09:44 hydra01 kernel: [2097287.717998] [] do_dentry_open+0x203/0x290 Sep 16 14:09:44 hydra01 kernel: [2097287.718002] [] ? get_iowait_time+0x60/0x60 Sep 16 14:09:44 hydra01 kernel: [2097287.718005] [] ? init_once+0x20/0x20 Sep 16 14:09:44 hydra01 kernel: [2097287.718008] [] stat_open+0x56/0xc0 Sep 16 14:09:44 hydra01 kernel: [2097287.718010] [] vfs_open+0x49/0x50 Sep 16 14:09:44 hydra01 kernel: [2097287.718013] [] proc_reg_open+0xa0/0x180 Sep 16 14:09:44 hydra01 kernel: [2097287.718017] [] do_last+0x296/0xea0 Sep 16 14:09:44 hydra01 kernel: [2097287.718020] [] ? seq_release_private+0x60/0x60 Sep 16 14:09:44 hydra01 kernel: [2097287.718023] [] ? inode_permission+0x18/0x50 Sep 16 14:09:44 hydra01 kernel: [2097287.718026] [] do_dentry_open+0x203/0x290 Sep 16 14:09:44 hydra01 kernel: [2097287.718029] [] ? link_path_walk+0x6b/0x880 Sep 16 14:09:44 hydra01 kernel: [2097287.718032] [] ? init_once+0x20/0x20 Sep 16 14:09:44 hydra01 kernel: [2097287.718035] [] path_openat+0xb3/0x4b0 Sep 16 14:09:44 hydra01 kernel: [2097287.718036] [] vfs_open+0x49/0x50 Sep 16 14:09:44 hydra01 kernel: [2097287.718039] [] do_last+0x296/0xea0 Sep 16 14:09:44 hydra01 kernel: [2097287.718042] [] ? inode_permission+0x18/0x50 Sep 16 14:09:44 hydra01 kernel: [2097287.718045] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 14:09:44 hydra01 kernel: [2097287.718048] [] ? link_path_walk+0x6b/0x880 Sep 16 14:09:44 hydra01 kernel: [2097287.718052] [] ? unmap_region+0xdb/0x120 Sep 16 14:09:44 hydra01 kernel: [2097287.718054] [] path_openat+0xb3/0x4b0 Sep 16 14:09:44 hydra01 kernel: [2097287.718057] [] do_filp_open+0x41/0xa0 Sep 16 14:09:44 hydra01 kernel: [2097287.718060] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 14:09:44 hydra01 kernel: [2097287.718063] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:09:44 hydra01 kernel: [2097287.718067] [] ? unmap_region+0xdb/0x120 Sep 16 14:09:44 hydra01 kernel: [2097287.718069] [] do_sys_open+0xf3/0x230 Sep 16 14:09:44 hydra01 kernel: [2097287.718072] [] do_filp_open+0x41/0xa0 Sep 16 14:09:44 hydra01 kernel: [2097287.718075] [] sys_open+0x21/0x30 Sep 16 14:09:44 hydra01 kernel: [2097287.718077] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:09:44 hydra01 kernel: [2097287.718081] [] system_call_fastpath+0x1a/0x1f Sep 16 14:09:44 hydra01 kernel: [2097287.718083] [] do_sys_open+0xf3/0x230 Sep 16 14:09:44 hydra01 kernel: [2097287.718085] Mem-Info: Sep 16 14:09:44 hydra01 kernel: [2097287.718087] Node 0 DMA per-cpu: Sep 16 14:09:44 hydra01 kernel: [2097287.718089] [] sys_open+0x21/0x30 Sep 16 14:09:44 hydra01 kernel: [2097287.718091] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718093] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718094] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718096] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718097] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718099] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718100] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718101] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718103] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718104] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718106] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718107] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718109] Node 0 DMA32 per-cpu: Sep 16 14:09:44 hydra01 kernel: [2097287.718112] [] system_call_fastpath+0x1a/0x1f Sep 16 14:09:44 hydra01 kernel: [2097287.718114] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718116] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718117] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718119] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718120] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718122] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718123] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718125] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718126] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718127] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718129] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718130] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718132] Node 0 Normal per-cpu: Sep 16 14:09:44 hydra01 kernel: [2097287.718133] Mem-Info: Sep 16 14:09:44 hydra01 kernel: [2097287.718135] Node 0 DMA per-cpu: Sep 16 14:09:44 hydra01 kernel: [2097287.718137] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718139] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718140] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718142] CPU 3: hi: 186, btch: 31 usd: 155 Sep 16 14:09:44 hydra01 kernel: [2097287.718143] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718145] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718146] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718147] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718149] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718150] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718152] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718153] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718157] active_anon:1205516 inactive_anon:770712 isolated_anon:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718157] active_file:2504720 inactive_file:3130567 isolated_file:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718157] unevictable:0 dirty:0 writeback:109 unstable:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718157] free:62226 slab_reclaimable:281440 slab_unreclaimable:106787 Sep 16 14:09:44 hydra01 kernel: [2097287.718157] mapped:28296 shmem:151 pagetables:9585 bounce:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718157] free_cma:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718165] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:09:44 hydra01 kernel: [2097287.718167] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718169] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718170] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718171] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718173] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718174] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718176] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718177] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718178] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718180] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718181] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718183] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718185] Node 0 DMA32 per-cpu: Sep 16 14:09:44 hydra01 kernel: [2097287.718188] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718190] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718191] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718193] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718194] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718196] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718197] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718199] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718200] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718202] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718203] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718205] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718186] lowmem_reserve[]: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718206] Node 0 Normal per-cpu: Sep 16 14:09:44 hydra01 kernel: [2097287.718209] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718210] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718212] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718213] CPU 3: hi: 186, btch: 31 usd: 186 Sep 16 14:09:44 hydra01 kernel: [2097287.718215] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718216] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718217] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718219] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718220] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718222] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718223] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718225] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718229] active_anon:1205516 inactive_anon:770712 isolated_anon:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718229] active_file:2504720 inactive_file:3130471 isolated_file:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718229] unevictable:0 dirty:0 writeback:109 unstable:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718229] free:62299 slab_reclaimable:281440 slab_unreclaimable:106787 Sep 16 14:09:44 hydra01 kernel: [2097287.718229] mapped:28296 shmem:151 pagetables:9585 bounce:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718229] free_cma:0 Sep 16 14:09:44 hydra01 kernel: [2097287.718232] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:09:44 hydra01 kernel: [2097287.718238] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:09:44 hydra01 kernel: [2097287.718243] Node 0 DMA32 free:139720kB min:6224kB low:7780kB high:9336kB active_anon:448548kB inactive_anon:803572kB active_file:286548kB inactive_file:1182484kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:4336kB shmem:16kB slab_reclaimable:140660kB slab_unreclaimable:52724kB kernel_stack:1192kB pagetables:1544kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:09:44 hydra01 kernel: [2097287.718250] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:09:44 hydra01 kernel: [2097287.718255] Node 0 Normal free:93924kB min:61324kB low:76652kB high:91984kB active_anon:4373516kB inactive_anon:2279276kB active_file:9732332kB inactive_file:11339016kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:436kB mapped:108848kB shmem:588kB slab_reclaimable:985100kB slab_unreclaimable:374424kB kernel_stack:7752kB pagetables:36796kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 14:09:44 hydra01 kernel: [2097287.718261] lowmem_reserve[]: 0 0 0 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718266] Node 0 DMA: 1*4kB 2967 32199 32199 Sep 16 14:09:44 hydra01 kernel: [2097287.718271] (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:09:44 hydra01 kernel: [2097287.718288] Node 0 DMA32: 7259*4kB Sep 16 14:09:44 hydra01 kernel: [2097287.718297] Node 0 DMA32 free:139720kB min:6224kB low:7780kB high:9336kB active_anon:448548kB inactive_anon:803572kB active_file:286548kB inactive_file:1182484kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:4336kB shmem:16kB slab_reclaimable:140660kB slab_unreclaimable:52724kB kernel_stack:1192kB pagetables:1544kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:09:44 hydra01 kernel: [2097287.718298] (UEM) 2580*8kB Sep 16 14:09:44 hydra01 kernel: [2097287.718301] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:09:44 hydra01 kernel: [2097287.718306] Node 0 Normal free:94216kB min:61324kB low:76652kB high:91984kB active_anon:4373516kB inactive_anon:2279276kB active_file:9732332kB inactive_file:11338632kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:0kB writeback:436kB mapped:108848kB shmem:588kB slab_reclaimable:985100kB slab_unreclaimable:374424kB kernel_stack:7752kB pagetables:36796kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 14:09:44 hydra01 kernel: [2097287.718312] lowmem_reserve[]: 0 0 0 0 Sep 16 14:09:44 hydra01 kernel: [2097287.718317] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:09:44 hydra01 kernel: [2097287.718337] Node 0 DMA32: 7259*4kB (UEM) 2580*8kB (UEM) 5150*16kB (UEM) 238*32kB (UEMR) 4*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140460kB Sep 16 14:09:44 hydra01 kernel: [2097287.718355] Node 0 Normal: 22522*4kB (UEMR) 300*8kB (UEMR) 69*16kB (UEMR) 10*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 95576kB Sep 16 14:09:44 hydra01 kernel: [2097287.718373] 6049449 total pagecache pages Sep 16 14:09:44 hydra01 kernel: [2097287.718375] 414403 pages in swap cache Sep 16 14:09:44 hydra01 kernel: [2097287.718376] Swap cache stats: add 20909869, delete 20495466, find 12678713/14337845 Sep 16 14:09:44 hydra01 kernel: [2097287.718378] Free swap = 184934672kB Sep 16 14:09:44 hydra01 kernel: [2097287.718379] Total swap = 187617276kB Sep 16 14:09:44 hydra01 kernel: [2097287.718380] (UEM) 5150*16kB (UEM) 238*32kB (UEMR) 4*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 140460kB Sep 16 14:09:44 hydra01 kernel: [2097287.718388] Node 0 Normal: 22522*4kB (UEMR) 300*8kB (UEMR) 69*16kB (UEMR) 10*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 95576kB Sep 16 14:09:44 hydra01 kernel: [2097287.718399] 6049449 total pagecache pages Sep 16 14:09:44 hydra01 kernel: [2097287.718400] 414403 pages in swap cache Sep 16 14:09:44 hydra01 kernel: [2097287.718402] Swap cache stats: add 20909869, delete 20495466, find 12678713/14337845 Sep 16 14:09:44 hydra01 kernel: [2097287.718403] Free swap = 184934672kB Sep 16 14:09:44 hydra01 kernel: [2097287.718404] Total swap = 187617276kB Sep 16 14:09:44 hydra01 kernel: [2097287.817866] 8388592 pages RAM Sep 16 14:09:44 hydra01 kernel: [2097287.817875] 151696 pages reserved Sep 16 14:09:44 hydra01 kernel: [2097287.817880] 6363549 pages shared Sep 16 14:09:44 hydra01 kernel: [2097287.817886] 5770101 pages non-shared Sep 16 14:09:44 hydra01 kernel: [2097287.922900] 8388592 pages RAM Sep 16 14:09:44 hydra01 kernel: [2097287.922903] 151696 pages reserved Sep 16 14:09:44 hydra01 kernel: [2097287.922905] 6364102 pages shared Sep 16 14:09:44 hydra01 kernel: [2097287.922906] 5769505 pages non-shared Sep 16 14:09:52 hydra01 kernel: [2097295.657868] iostat: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:09:52 hydra01 kernel: [2097295.657873] Pid: 153219, comm: iostat Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:09:52 hydra01 kernel: [2097295.657875] Call Trace: Sep 16 14:09:52 hydra01 kernel: [2097295.657884] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:09:52 hydra01 kernel: [2097295.657888] [] ? drain_local_pages+0x16/0x20 Sep 16 14:09:52 hydra01 kernel: [2097295.657893] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 14:09:52 hydra01 kernel: [2097295.657896] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:09:52 hydra01 kernel: [2097295.657901] [] alloc_pages_current+0xb8/0x180 Sep 16 14:09:52 hydra01 kernel: [2097295.657903] [] __get_free_pages+0xe/0x50 Sep 16 14:09:52 hydra01 kernel: [2097295.657908] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:09:52 hydra01 kernel: [2097295.657911] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 14:09:52 hydra01 kernel: [2097295.657914] [] __kmalloc+0x161/0x170 Sep 16 14:09:52 hydra01 kernel: [2097295.657918] [] ? get_iowait_time+0x60/0x60 Sep 16 14:09:52 hydra01 kernel: [2097295.657922] [] stat_open+0x56/0xc0 Sep 16 14:09:52 hydra01 kernel: [2097295.657926] [] proc_reg_open+0xa0/0x180 Sep 16 14:09:52 hydra01 kernel: [2097295.657932] [] ? seq_release_private+0x60/0x60 Sep 16 14:09:52 hydra01 kernel: [2097295.657936] [] do_dentry_open+0x203/0x290 Sep 16 14:09:52 hydra01 kernel: [2097295.657938] [] ? init_once+0x20/0x20 Sep 16 14:09:52 hydra01 kernel: [2097295.657940] [] vfs_open+0x49/0x50 Sep 16 14:09:52 hydra01 kernel: [2097295.657943] [] do_last+0x296/0xea0 Sep 16 14:09:52 hydra01 kernel: [2097295.657946] [] ? inode_permission+0x18/0x50 Sep 16 14:09:52 hydra01 kernel: [2097295.657948] [] ? link_path_walk+0x6b/0x880 Sep 16 14:09:52 hydra01 kernel: [2097295.657951] [] path_openat+0xb3/0x4b0 Sep 16 14:09:52 hydra01 kernel: [2097295.657955] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 14:09:52 hydra01 kernel: [2097295.657958] [] ? unmap_region+0xdb/0x120 Sep 16 14:09:52 hydra01 kernel: [2097295.657961] [] do_filp_open+0x41/0xa0 Sep 16 14:09:52 hydra01 kernel: [2097295.657964] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:09:52 hydra01 kernel: [2097295.657967] [] do_sys_open+0xf3/0x230 Sep 16 14:09:52 hydra01 kernel: [2097295.657969] [] sys_open+0x21/0x30 Sep 16 14:09:52 hydra01 kernel: [2097295.657972] [] system_call_fastpath+0x1a/0x1f Sep 16 14:09:52 hydra01 kernel: [2097295.657974] Mem-Info: Sep 16 14:09:52 hydra01 kernel: [2097295.657975] Node 0 DMA per-cpu: Sep 16 14:09:52 hydra01 kernel: [2097295.657978] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657979] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657981] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657982] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657983] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657985] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657986] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657987] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657989] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657990] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657992] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657993] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657994] Node 0 DMA32 per-cpu: Sep 16 14:09:52 hydra01 kernel: [2097295.657996] CPU 0: hi: 186, btch: 31 usd: 72 Sep 16 14:09:52 hydra01 kernel: [2097295.657998] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.657999] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658000] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658002] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658003] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658004] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658006] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658007] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658009] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658010] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658011] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658012] Node 0 Normal per-cpu: Sep 16 14:09:52 hydra01 kernel: [2097295.658014] CPU 0: hi: 186, btch: 31 usd: 184 Sep 16 14:09:52 hydra01 kernel: [2097295.658016] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658017] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658018] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658020] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658021] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658022] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658024] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658025] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658027] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658028] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658029] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658033] active_anon:1214059 inactive_anon:772066 isolated_anon:0 Sep 16 14:09:52 hydra01 kernel: [2097295.658033] active_file:2504742 inactive_file:3113612 isolated_file:0 Sep 16 14:09:52 hydra01 kernel: [2097295.658033] unevictable:0 dirty:99 writeback:0 unstable:0 Sep 16 14:09:52 hydra01 kernel: [2097295.658033] free:70305 slab_reclaimable:281203 slab_unreclaimable:106787 Sep 16 14:09:52 hydra01 kernel: [2097295.658033] mapped:28815 shmem:151 pagetables:9634 bounce:0 Sep 16 14:09:52 hydra01 kernel: [2097295.658033] free_cma:0 Sep 16 14:09:52 hydra01 kernel: [2097295.658037] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:09:52 hydra01 kernel: [2097295.658043] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:09:52 hydra01 kernel: [2097295.658048] Node 0 DMA32 free:132724kB min:6224kB low:7780kB high:9336kB active_anon:452684kB inactive_anon:804332kB active_file:286548kB inactive_file:1182772kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:88kB writeback:0kB mapped:4228kB shmem:0kB slab_reclaimable:140612kB slab_unreclaimable:52852kB kernel_stack:1248kB pagetables:1552kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:72 all_unreclaimable? no Sep 16 14:09:52 hydra01 kernel: [2097295.658056] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:09:52 hydra01 kernel: [2097295.658059] Node 0 Normal free:132944kB min:61324kB low:76652kB high:91984kB active_anon:4403552kB inactive_anon:2283932kB active_file:9732420kB inactive_file:11271292kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:308kB writeback:0kB mapped:111032kB shmem:604kB slab_reclaimable:984200kB slab_unreclaimable:374296kB kernel_stack:7720kB pagetables:36984kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no Sep 16 14:09:52 hydra01 kernel: [2097295.658064] lowmem_reserve[]: 0 0 0 0 Sep 16 14:09:52 hydra01 kernel: [2097295.658067] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:09:52 hydra01 kernel: [2097295.658081] Node 0 DMA32: 6356*4kB (UEM) 2558*8kB (UEM) 5017*16kB (UEM) 205*32kB (UEMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 133360kB Sep 16 14:09:52 hydra01 kernel: [2097295.658094] Node 0 Normal: 29591*4kB (UMR) 1872*8kB (UEMR) 5*16kB (R) 6*32kB (R) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 135276kB Sep 16 14:09:52 hydra01 kernel: [2097295.658109] 6034456 total pagecache pages Sep 16 14:09:52 hydra01 kernel: [2097295.658110] 416246 pages in swap cache Sep 16 14:09:52 hydra01 kernel: [2097295.658112] Swap cache stats: add 20914343, delete 20498097, find 12680132/14339664 Sep 16 14:09:52 hydra01 kernel: [2097295.658114] Free swap = 184934264kB Sep 16 14:09:52 hydra01 kernel: [2097295.658116] Total swap = 187617276kB Sep 16 14:09:52 hydra01 kernel: [2097295.760058] 8388592 pages RAM Sep 16 14:09:52 hydra01 kernel: [2097295.760060] 151696 pages reserved Sep 16 14:09:52 hydra01 kernel: [2097295.760061] 6368153 pages shared Sep 16 14:09:52 hydra01 kernel: [2097295.760062] 5766047 pages non-shared Sep 16 14:09:52 hydra01 kernel: [2097296.018940] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:09:52 hydra01 kernel: [2097296.018945] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:09:52 hydra01 kernel: [2097296.018947] Call Trace: Sep 16 14:09:52 hydra01 kernel: [2097296.018955] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:09:52 hydra01 kernel: [2097296.018960] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 14:09:52 hydra01 kernel: [2097296.018963] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:09:52 hydra01 kernel: [2097296.018967] [] alloc_pages_current+0xb8/0x180 Sep 16 14:09:52 hydra01 kernel: [2097296.018970] [] __get_free_pages+0xe/0x50 Sep 16 14:09:52 hydra01 kernel: [2097296.018975] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:09:52 hydra01 kernel: [2097296.018978] [] __kmalloc+0x161/0x170 Sep 16 14:09:52 hydra01 kernel: [2097296.018982] [] ? get_iowait_time+0x60/0x60 Sep 16 14:09:52 hydra01 kernel: [2097296.018984] [] stat_open+0x56/0xc0 Sep 16 14:09:52 hydra01 kernel: [2097296.018988] [] proc_reg_open+0xa0/0x180 Sep 16 14:09:52 hydra01 kernel: [2097296.018992] [] ? seq_release_private+0x60/0x60 Sep 16 14:09:52 hydra01 kernel: [2097296.018995] [] do_dentry_open+0x203/0x290 Sep 16 14:09:52 hydra01 kernel: [2097296.018997] [] ? init_once+0x20/0x20 Sep 16 14:09:52 hydra01 kernel: [2097296.018999] [] vfs_open+0x49/0x50 Sep 16 14:09:52 hydra01 kernel: [2097296.019002] [] do_last+0x296/0xea0 Sep 16 14:09:52 hydra01 kernel: [2097296.019005] [] ? inode_permission+0x18/0x50 Sep 16 14:09:52 hydra01 kernel: [2097296.019007] [] ? link_path_walk+0x6b/0x880 Sep 16 14:09:52 hydra01 kernel: [2097296.019010] [] path_openat+0xb3/0x4b0 Sep 16 14:09:52 hydra01 kernel: [2097296.019013] [] do_filp_open+0x41/0xa0 Sep 16 14:09:52 hydra01 kernel: [2097296.019016] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:09:52 hydra01 kernel: [2097296.019018] [] do_sys_open+0xf3/0x230 Sep 16 14:09:52 hydra01 kernel: [2097296.019021] [] ? sys_select+0xcb/0x110 Sep 16 14:09:52 hydra01 kernel: [2097296.019023] [] sys_open+0x21/0x30 Sep 16 14:09:52 hydra01 kernel: [2097296.019026] [] system_call_fastpath+0x1a/0x1f Sep 16 14:09:52 hydra01 kernel: [2097296.019028] Mem-Info: Sep 16 14:09:52 hydra01 kernel: [2097296.019029] Node 0 DMA per-cpu: Sep 16 14:09:52 hydra01 kernel: [2097296.019032] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019033] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019035] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019036] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019037] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019039] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019040] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019041] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019043] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019044] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019045] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019047] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019048] Node 0 DMA32 per-cpu: Sep 16 14:09:52 hydra01 kernel: [2097296.019050] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019051] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019053] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019054] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019056] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019057] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019058] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019060] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019061] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019062] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019064] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019065] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019066] Node 0 Normal per-cpu: Sep 16 14:09:52 hydra01 kernel: [2097296.019068] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019070] CPU 1: hi: 186, btch: 31 usd: 174 Sep 16 14:09:52 hydra01 kernel: [2097296.019071] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019072] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019074] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019075] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019076] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019078] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019079] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019081] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019082] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019083] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019087] active_anon:1214132 inactive_anon:772066 isolated_anon:0 Sep 16 14:09:52 hydra01 kernel: [2097296.019087] active_file:2504742 inactive_file:3105914 isolated_file:0 Sep 16 14:09:52 hydra01 kernel: [2097296.019087] unevictable:0 dirty:208 writeback:109 unstable:0 Sep 16 14:09:52 hydra01 kernel: [2097296.019087] free:78312 slab_reclaimable:281203 slab_unreclaimable:106787 Sep 16 14:09:52 hydra01 kernel: [2097296.019087] mapped:28815 shmem:151 pagetables:9743 bounce:0 Sep 16 14:09:52 hydra01 kernel: [2097296.019087] free_cma:0 Sep 16 14:09:52 hydra01 kernel: [2097296.019091] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:09:52 hydra01 kernel: [2097296.019095] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:09:52 hydra01 kernel: [2097296.019099] Node 0 DMA32 free:136144kB min:6224kB low:7780kB high:9336kB active_anon:452684kB inactive_anon:804332kB active_file:286548kB inactive_file:1181044kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:88kB writeback:0kB mapped:4228kB shmem:0kB slab_reclaimable:140612kB slab_unreclaimable:52852kB kernel_stack:1248kB pagetables:1552kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:09:52 hydra01 kernel: [2097296.019104] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:09:52 hydra01 kernel: [2097296.019106] Node 0 Normal free:161260kB min:61324kB low:76652kB high:91984kB active_anon:4403844kB inactive_anon:2283932kB active_file:9732420kB inactive_file:11242612kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:744kB writeback:436kB mapped:111032kB shmem:604kB slab_reclaimable:984200kB slab_unreclaimable:374296kB kernel_stack:7720kB pagetables:37420kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:09:52 hydra01 kernel: [2097296.019111] lowmem_reserve[]: 0 0 0 0 Sep 16 14:09:52 hydra01 kernel: [2097296.019114] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:09:52 hydra01 kernel: [2097296.019125] Node 0 DMA32: 6860*4kB (UEM) 2558*8kB (UEM) 5017*16kB (UEM) 242*32kB (UEMR) 4*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 136688kB Sep 16 14:09:52 hydra01 kernel: [2097296.019135] Node 0 Normal: 26085*4kB (UMR) 6977*8kB (UEMR) 56*16kB (UEMR) 12*32kB (UER) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 163100kB Sep 16 14:09:52 hydra01 kernel: [2097296.019146] 6026930 total pagecache pages Sep 16 14:09:52 hydra01 kernel: [2097296.019147] 416246 pages in swap cache Sep 16 14:09:52 hydra01 kernel: [2097296.019149] Swap cache stats: add 20914343, delete 20498097, find 12680132/14339664 Sep 16 14:09:52 hydra01 kernel: [2097296.019150] Free swap = 184934264kB Sep 16 14:09:52 hydra01 kernel: [2097296.019151] Total swap = 187617276kB Sep 16 14:09:52 hydra01 kernel: [2097296.117002] 8388592 pages RAM Sep 16 14:09:52 hydra01 kernel: [2097296.117008] 151696 pages reserved Sep 16 14:09:52 hydra01 kernel: [2097296.117013] 6369219 pages shared Sep 16 14:09:52 hydra01 kernel: [2097296.117018] 5764826 pages non-shared Sep 16 14:09:57 hydra01 kernel: [2097301.016229] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:09:57 hydra01 kernel: [2097301.016235] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:09:57 hydra01 kernel: [2097301.016236] Call Trace: Sep 16 14:09:57 hydra01 kernel: [2097301.016245] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:09:57 hydra01 kernel: [2097301.016249] [] ? drain_local_pages+0x16/0x20 Sep 16 14:09:57 hydra01 kernel: [2097301.016254] [] ? on_each_cpu_mask+0x50/0x70 Sep 16 14:09:57 hydra01 kernel: [2097301.016257] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:09:57 hydra01 kernel: [2097301.016261] [] alloc_pages_current+0xb8/0x180 Sep 16 14:09:57 hydra01 kernel: [2097301.016264] [] __get_free_pages+0xe/0x50 Sep 16 14:09:57 hydra01 kernel: [2097301.016268] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:09:57 hydra01 kernel: [2097301.016272] [] __kmalloc+0x161/0x170 Sep 16 14:09:57 hydra01 kernel: [2097301.016276] [] ? get_iowait_time+0x60/0x60 Sep 16 14:09:57 hydra01 kernel: [2097301.016278] [] stat_open+0x56/0xc0 Sep 16 14:09:57 hydra01 kernel: [2097301.016282] [] proc_reg_open+0xa0/0x180 Sep 16 14:09:57 hydra01 kernel: [2097301.016286] [] ? seq_release_private+0x60/0x60 Sep 16 14:09:57 hydra01 kernel: [2097301.016289] [] do_dentry_open+0x203/0x290 Sep 16 14:09:57 hydra01 kernel: [2097301.016292] [] ? init_once+0x20/0x20 Sep 16 14:09:57 hydra01 kernel: [2097301.016294] [] vfs_open+0x49/0x50 Sep 16 14:09:57 hydra01 kernel: [2097301.016297] [] do_last+0x296/0xea0 Sep 16 14:09:57 hydra01 kernel: [2097301.016300] [] ? inode_permission+0x18/0x50 Sep 16 14:09:57 hydra01 kernel: [2097301.016302] [] ? link_path_walk+0x6b/0x880 Sep 16 14:09:57 hydra01 kernel: [2097301.016305] [] path_openat+0xb3/0x4b0 Sep 16 14:09:57 hydra01 kernel: [2097301.016308] [] do_filp_open+0x41/0xa0 Sep 16 14:09:57 hydra01 kernel: [2097301.016311] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:09:57 hydra01 kernel: [2097301.016313] [] do_sys_open+0xf3/0x230 Sep 16 14:09:57 hydra01 kernel: [2097301.016316] [] ? sys_select+0xcb/0x110 Sep 16 14:09:57 hydra01 kernel: [2097301.016318] [] sys_open+0x21/0x30 Sep 16 14:09:57 hydra01 kernel: [2097301.016322] [] system_call_fastpath+0x1a/0x1f Sep 16 14:09:57 hydra01 kernel: [2097301.016324] Mem-Info: Sep 16 14:09:57 hydra01 kernel: [2097301.016325] Node 0 DMA per-cpu: Sep 16 14:09:57 hydra01 kernel: [2097301.016327] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016329] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016330] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016331] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016333] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016334] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016336] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016337] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016338] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016340] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016341] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016342] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016343] Node 0 DMA32 per-cpu: Sep 16 14:09:57 hydra01 kernel: [2097301.016345] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016347] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016348] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016350] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016351] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016352] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016354] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016355] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016356] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016358] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016359] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016360] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016361] Node 0 Normal per-cpu: Sep 16 14:09:57 hydra01 kernel: [2097301.016363] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016365] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016366] CPU 2: hi: 186, btch: 31 usd: 182 Sep 16 14:09:57 hydra01 kernel: [2097301.016368] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016369] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016370] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016372] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016373] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016374] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016376] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016377] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016379] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016383] active_anon:1219870 inactive_anon:772758 isolated_anon:0 Sep 16 14:09:57 hydra01 kernel: [2097301.016383] active_file:2504745 inactive_file:3115276 isolated_file:0 Sep 16 14:09:57 hydra01 kernel: [2097301.016383] unevictable:0 dirty:197 writeback:109 unstable:0 Sep 16 14:09:57 hydra01 kernel: [2097301.016383] free:63401 slab_reclaimable:281045 slab_unreclaimable:106799 Sep 16 14:09:57 hydra01 kernel: [2097301.016383] mapped:37777 shmem:151 pagetables:9659 bounce:0 Sep 16 14:09:57 hydra01 kernel: [2097301.016383] free_cma:0 Sep 16 14:09:57 hydra01 kernel: [2097301.016386] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:09:57 hydra01 kernel: [2097301.016391] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:09:57 hydra01 kernel: [2097301.016394] Node 0 DMA32 free:138952kB min:6224kB low:7780kB high:9336kB active_anon:453984kB inactive_anon:804348kB active_file:286548kB inactive_file:1177124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:8kB writeback:0kB mapped:4232kB shmem:0kB slab_reclaimable:140604kB slab_unreclaimable:52916kB kernel_stack:1248kB pagetables:1560kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:09:57 hydra01 kernel: [2097301.016399] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:09:57 hydra01 kernel: [2097301.016402] Node 0 Normal free:98808kB min:61324kB low:76652kB high:91984kB active_anon:4425496kB inactive_anon:2286684kB active_file:9732432kB inactive_file:11283980kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:780kB writeback:436kB mapped:146876kB shmem:604kB slab_reclaimable:983576kB slab_unreclaimable:374280kB kernel_stack:7712kB pagetables:37076kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:16 all_unreclaimable? no Sep 16 14:09:57 hydra01 kernel: [2097301.016407] lowmem_reserve[]: 0 0 0 0 Sep 16 14:09:57 hydra01 kernel: [2097301.016409] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:09:57 hydra01 kernel: [2097301.016421] Node 0 DMA32: 6905*4kB (UEM) 2908*8kB (UEM) 5022*16kB (UEM) 240*32kB (UEMR) 4*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 139684kB Sep 16 14:09:57 hydra01 kernel: [2097301.016431] Node 0 Normal: 19507*4kB (UEMR) 2501*8kB (UEMR) 34*16kB (UER) 10*32kB (UR) 6*64kB (R) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 100564kB Sep 16 14:09:57 hydra01 kernel: [2097301.016441] 6037176 total pagecache pages Sep 16 14:09:57 hydra01 kernel: [2097301.016443] 416859 pages in swap cache Sep 16 14:09:57 hydra01 kernel: [2097301.016444] Swap cache stats: add 20916467, delete 20499608, find 12681023/14340751 Sep 16 14:09:57 hydra01 kernel: [2097301.016446] Free swap = 184935248kB Sep 16 14:09:57 hydra01 kernel: [2097301.016447] Total swap = 187617276kB Sep 16 14:09:57 hydra01 kernel: [2097301.115105] 8388592 pages RAM Sep 16 14:09:57 hydra01 kernel: [2097301.115107] 151696 pages reserved Sep 16 14:09:57 hydra01 kernel: [2097301.115109] 6384074 pages shared Sep 16 14:09:57 hydra01 kernel: [2097301.115110] 5758136 pages non-shared Sep 16 14:13:02 hydra01 kernel: [2097486.096554] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:13:02 hydra01 kernel: [2097486.096559] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:13:02 hydra01 kernel: [2097486.096561] Call Trace: Sep 16 14:13:02 hydra01 kernel: [2097486.096570] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:13:02 hydra01 kernel: [2097486.096575] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 14:13:02 hydra01 kernel: [2097486.096578] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:13:02 hydra01 kernel: [2097486.096583] [] alloc_pages_current+0xb8/0x180 Sep 16 14:13:02 hydra01 kernel: [2097486.096585] [] __get_free_pages+0xe/0x50 Sep 16 14:13:02 hydra01 kernel: [2097486.096590] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:13:02 hydra01 kernel: [2097486.096593] [] __kmalloc+0x161/0x170 Sep 16 14:13:02 hydra01 kernel: [2097486.096597] [] ? get_iowait_time+0x60/0x60 Sep 16 14:13:02 hydra01 kernel: [2097486.096600] [] stat_open+0x56/0xc0 Sep 16 14:13:02 hydra01 kernel: [2097486.096603] [] proc_reg_open+0xa0/0x180 Sep 16 14:13:02 hydra01 kernel: [2097486.096607] [] ? seq_release_private+0x60/0x60 Sep 16 14:13:02 hydra01 kernel: [2097486.096610] [] do_dentry_open+0x203/0x290 Sep 16 14:13:02 hydra01 kernel: [2097486.096613] [] ? init_once+0x20/0x20 Sep 16 14:13:02 hydra01 kernel: [2097486.096615] [] vfs_open+0x49/0x50 Sep 16 14:13:02 hydra01 kernel: [2097486.096618] [] do_last+0x296/0xea0 Sep 16 14:13:02 hydra01 kernel: [2097486.096621] [] ? inode_permission+0x18/0x50 Sep 16 14:13:02 hydra01 kernel: [2097486.096623] [] ? link_path_walk+0x6b/0x880 Sep 16 14:13:02 hydra01 kernel: [2097486.096626] [] path_openat+0xb3/0x4b0 Sep 16 14:13:02 hydra01 kernel: [2097486.096628] [] do_filp_open+0x41/0xa0 Sep 16 14:13:02 hydra01 kernel: [2097486.096631] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:13:02 hydra01 kernel: [2097486.096634] [] do_sys_open+0xf3/0x230 Sep 16 14:13:02 hydra01 kernel: [2097486.096639] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 14:13:02 hydra01 kernel: [2097486.096641] [] sys_open+0x21/0x30 Sep 16 14:13:02 hydra01 kernel: [2097486.096644] [] system_call_fastpath+0x1a/0x1f Sep 16 14:13:02 hydra01 kernel: [2097486.096646] Mem-Info: Sep 16 14:13:02 hydra01 kernel: [2097486.096647] Node 0 DMA per-cpu: Sep 16 14:13:02 hydra01 kernel: [2097486.096650] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096652] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096653] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096654] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096656] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096657] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096659] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096660] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096661] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096663] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096664] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096666] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096667] Node 0 DMA32 per-cpu: Sep 16 14:13:02 hydra01 kernel: [2097486.096669] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096670] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096672] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096673] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096674] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096676] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096677] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096679] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096680] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096681] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096683] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096684] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096685] Node 0 Normal per-cpu: Sep 16 14:13:02 hydra01 kernel: [2097486.096687] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096688] CPU 1: hi: 186, btch: 31 usd: 18 Sep 16 14:13:02 hydra01 kernel: [2097486.096690] CPU 2: hi: 186, btch: 31 usd: 160 Sep 16 14:13:02 hydra01 kernel: [2097486.096691] CPU 3: hi: 186, btch: 31 usd: 22 Sep 16 14:13:02 hydra01 kernel: [2097486.096692] CPU 4: hi: 186, btch: 31 usd: 82 Sep 16 14:13:02 hydra01 kernel: [2097486.096694] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096695] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096697] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096698] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096699] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096701] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096702] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096706] active_anon:1228845 inactive_anon:777169 isolated_anon:0 Sep 16 14:13:02 hydra01 kernel: [2097486.096706] active_file:2497276 inactive_file:3095042 isolated_file:0 Sep 16 14:13:02 hydra01 kernel: [2097486.096706] unevictable:0 dirty:437 writeback:26 unstable:0 Sep 16 14:13:02 hydra01 kernel: [2097486.096706] free:80281 slab_reclaimable:280360 slab_unreclaimable:106752 Sep 16 14:13:02 hydra01 kernel: [2097486.096706] mapped:37439 shmem:151 pagetables:9816 bounce:0 Sep 16 14:13:02 hydra01 kernel: [2097486.096706] free_cma:0 Sep 16 14:13:02 hydra01 kernel: [2097486.096709] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:13:02 hydra01 kernel: [2097486.096714] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:13:02 hydra01 kernel: [2097486.096717] Node 0 DMA32 free:136316kB min:6224kB low:7780kB high:9336kB active_anon:457116kB inactive_anon:804636kB active_file:286692kB inactive_file:1176328kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:0kB writeback:0kB mapped:12288kB shmem:0kB slab_reclaimable:140612kB slab_unreclaimable:52916kB kernel_stack:1256kB pagetables:1560kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:13:02 hydra01 kernel: [2097486.096722] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:13:02 hydra01 kernel: [2097486.096725] Node 0 Normal free:168964kB min:61324kB low:76652kB high:91984kB active_anon:4458264kB inactive_anon:2304040kB active_file:9702412kB inactive_file:11203840kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:1748kB writeback:112kB mapped:137468kB shmem:604kB slab_reclaimable:980828kB slab_unreclaimable:374092kB kernel_stack:7728kB pagetables:37704kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:13:02 hydra01 kernel: [2097486.096730] lowmem_reserve[]: 0 0 0 0 Sep 16 14:13:02 hydra01 kernel: [2097486.096733] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:13:02 hydra01 kernel: [2097486.096744] Node 0 DMA32: 4208*4kB (UEM) 3759*8kB (UEM) 5054*16kB (UEM) 256*32kB (UEMR) 5*64kB (UMR) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 136792kB Sep 16 14:13:02 hydra01 kernel: [2097486.096754] Node 0 Normal: 23127*4kB (UEMR) 4676*8kB (UEMR) 1522*16kB (UEMR) 387*32kB (UEMR) 24*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 169468kB Sep 16 14:13:02 hydra01 kernel: [2097486.096764] 6012475 total pagecache pages Sep 16 14:13:02 hydra01 kernel: [2097486.096765] 419981 pages in swap cache Sep 16 14:13:02 hydra01 kernel: [2097486.096767] Swap cache stats: add 20924790, delete 20504809, find 12685653/14346321 Sep 16 14:13:02 hydra01 kernel: [2097486.096768] Free swap = 184944748kB Sep 16 14:13:02 hydra01 kernel: [2097486.096769] Total swap = 187617276kB Sep 16 14:13:02 hydra01 kernel: [2097486.198101] 8388592 pages RAM Sep 16 14:13:02 hydra01 kernel: [2097486.198108] 151696 pages reserved Sep 16 14:13:02 hydra01 kernel: [2097486.198113] 6759637 pages shared Sep 16 14:13:02 hydra01 kernel: [2097486.198118] 5384074 pages non-shared Sep 16 14:14:12 hydra01 kernel: [2097556.018535] snmpd: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:14:12 hydra01 kernel: [2097556.018544] Pid: 49312, comm: snmpd Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:14:12 hydra01 kernel: [2097556.018547] Call Trace: Sep 16 14:14:12 hydra01 kernel: [2097556.018561] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:14:12 hydra01 kernel: [2097556.018569] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 14:14:12 hydra01 kernel: [2097556.018575] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:14:12 hydra01 kernel: [2097556.018583] [] alloc_pages_current+0xb8/0x180 Sep 16 14:14:12 hydra01 kernel: [2097556.018589] [] __get_free_pages+0xe/0x50 Sep 16 14:14:12 hydra01 kernel: [2097556.018596] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:14:12 hydra01 kernel: [2097556.018602] [] __kmalloc+0x161/0x170 Sep 16 14:14:12 hydra01 kernel: [2097556.018608] [] ? get_iowait_time+0x60/0x60 Sep 16 14:14:12 hydra01 kernel: [2097556.018613] [] stat_open+0x56/0xc0 Sep 16 14:14:12 hydra01 kernel: [2097556.018619] [] proc_reg_open+0xa0/0x180 Sep 16 14:14:12 hydra01 kernel: [2097556.018626] [] ? seq_release_private+0x60/0x60 Sep 16 14:14:12 hydra01 kernel: [2097556.018632] [] do_dentry_open+0x203/0x290 Sep 16 14:14:12 hydra01 kernel: [2097556.018637] [] ? init_once+0x20/0x20 Sep 16 14:14:12 hydra01 kernel: [2097556.018641] [] vfs_open+0x49/0x50 Sep 16 14:14:12 hydra01 kernel: [2097556.018647] [] do_last+0x296/0xea0 Sep 16 14:14:12 hydra01 kernel: [2097556.018651] [] ? inode_permission+0x18/0x50 Sep 16 14:14:12 hydra01 kernel: [2097556.018656] [] ? link_path_walk+0x6b/0x880 Sep 16 14:14:12 hydra01 kernel: [2097556.018661] [] path_openat+0xb3/0x4b0 Sep 16 14:14:12 hydra01 kernel: [2097556.018666] [] do_filp_open+0x41/0xa0 Sep 16 14:14:12 hydra01 kernel: [2097556.018671] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:14:12 hydra01 kernel: [2097556.018676] [] do_sys_open+0xf3/0x230 Sep 16 14:14:12 hydra01 kernel: [2097556.018684] [] ? fire_user_return_notifiers+0x32/0x40 Sep 16 14:14:12 hydra01 kernel: [2097556.018689] [] sys_open+0x21/0x30 Sep 16 14:14:12 hydra01 kernel: [2097556.018694] [] system_call_fastpath+0x1a/0x1f Sep 16 14:14:12 hydra01 kernel: [2097556.018696] Mem-Info: Sep 16 14:14:12 hydra01 kernel: [2097556.018699] Node 0 DMA per-cpu: Sep 16 14:14:12 hydra01 kernel: [2097556.018704] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018707] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018709] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018712] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018715] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018717] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018720] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018722] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018725] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018728] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018731] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018733] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018735] Node 0 DMA32 per-cpu: Sep 16 14:14:12 hydra01 kernel: [2097556.018739] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018742] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018745] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018748] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018750] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018753] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018755] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018758] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018761] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018763] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018766] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018769] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018771] Node 0 Normal per-cpu: Sep 16 14:14:12 hydra01 kernel: [2097556.018774] CPU 0: hi: 186, btch: 31 usd: 37 Sep 16 14:14:12 hydra01 kernel: [2097556.018777] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018779] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018782] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018785] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018787] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018790] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018793] CPU 7: hi: 186, btch: 31 usd: 25 Sep 16 14:14:12 hydra01 kernel: [2097556.018795] CPU 8: hi: 186, btch: 31 usd: 163 Sep 16 14:14:12 hydra01 kernel: [2097556.018798] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018801] CPU 10: hi: 186, btch: 31 usd: 7 Sep 16 14:14:12 hydra01 kernel: [2097556.018803] CPU 11: hi: 186, btch: 31 usd: 158 Sep 16 14:14:12 hydra01 kernel: [2097556.018811] active_anon:1205679 inactive_anon:766073 isolated_anon:0 Sep 16 14:14:12 hydra01 kernel: [2097556.018811] active_file:2499397 inactive_file:3031532 isolated_file:0 Sep 16 14:14:12 hydra01 kernel: [2097556.018811] unevictable:0 dirty:152 writeback:0 unstable:0 Sep 16 14:14:12 hydra01 kernel: [2097556.018811] free:185274 slab_reclaimable:285024 slab_unreclaimable:106357 Sep 16 14:14:12 hydra01 kernel: [2097556.018811] mapped:37538 shmem:151 pagetables:9895 bounce:0 Sep 16 14:14:12 hydra01 kernel: [2097556.018811] free_cma:0 Sep 16 14:14:12 hydra01 kernel: [2097556.018817] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:14:12 hydra01 kernel: [2097556.018826] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:14:12 hydra01 kernel: [2097556.018833] Node 0 DMA32 free:193248kB min:6224kB low:7780kB high:9336kB active_anon:457636kB inactive_anon:804492kB active_file:286628kB inactive_file:1112628kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:7036kB shmem:0kB slab_reclaimable:149948kB slab_unreclaimable:52596kB kernel_stack:1304kB pagetables:1564kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:14:12 hydra01 kernel: [2097556.018842] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:14:12 hydra01 kernel: [2097556.018848] Node 0 Normal free:532004kB min:61324kB low:76652kB high:91984kB active_anon:4365080kB inactive_anon:2259800kB active_file:9710960kB inactive_file:11013500kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:588kB writeback:0kB mapped:143116kB shmem:604kB slab_reclaimable:990148kB slab_unreclaimable:372832kB kernel_stack:7680kB pagetables:38016kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:14:12 hydra01 kernel: [2097556.018857] lowmem_reserve[]: 0 0 0 0 Sep 16 14:14:12 hydra01 kernel: [2097556.018862] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:14:12 hydra01 kernel: [2097556.018883] Node 0 DMA32: 19335*4kB (UEM) 3356*8kB (UEM) 5525*16kB (UEM) 2*32kB (UR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 193292kB Sep 16 14:14:12 hydra01 kernel: [2097556.018902] Node 0 Normal: 102529*4kB (UEMR) 7909*8kB (UEMR) 2827*16kB (UEMR) 323*32kB (UEMR) 29*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 532092kB Sep 16 14:14:12 hydra01 kernel: [2097556.018922] 5947608 total pagecache pages Sep 16 14:14:12 hydra01 kernel: [2097556.018925] 416528 pages in swap cache Sep 16 14:14:12 hydra01 kernel: [2097556.018928] Swap cache stats: add 20944219, delete 20527691, find 12685681/14346360 Sep 16 14:14:12 hydra01 kernel: [2097556.018930] Free swap = 184867900kB Sep 16 14:14:12 hydra01 kernel: [2097556.018932] Total swap = 187617276kB Sep 16 14:14:12 hydra01 kernel: [2097556.153387] 8388592 pages RAM Sep 16 14:14:12 hydra01 kernel: [2097556.153396] 151696 pages reserved Sep 16 14:14:12 hydra01 kernel: [2097556.153397] 6932300 pages shared Sep 16 14:14:12 hydra01 kernel: [2097556.153398] 5105274 pages non-shared Sep 16 14:14:14 hydra01 kernel: [2097557.540970] irqbalance: page allocation failure: order:4, mode:0x1040d0 Sep 16 14:14:14 hydra01 kernel: [2097557.540993] Pid: 5347, comm: irqbalance Tainted: GF 3.8.0-30-generic #44-Ubuntu Sep 16 14:14:14 hydra01 kernel: [2097557.540994] Call Trace: Sep 16 14:14:14 hydra01 kernel: [2097557.541003] [] warn_alloc_failed+0xe9/0x140 Sep 16 14:14:14 hydra01 kernel: [2097557.541009] [] ? __alloc_pages_direct_compact+0x182/0x194 Sep 16 14:14:14 hydra01 kernel: [2097557.541012] [] __alloc_pages_nodemask+0x6f1/0x920 Sep 16 14:14:14 hydra01 kernel: [2097557.541017] [] alloc_pages_current+0xb8/0x180 Sep 16 14:14:14 hydra01 kernel: [2097557.541020] [] __get_free_pages+0xe/0x50 Sep 16 14:14:14 hydra01 kernel: [2097557.541024] [] kmalloc_order_trace+0x39/0xb0 Sep 16 14:14:14 hydra01 kernel: [2097557.541028] [] ? page_add_new_anon_rmap+0x9d/0x140 Sep 16 14:14:14 hydra01 kernel: [2097557.541031] [] __kmalloc+0x161/0x170 Sep 16 14:14:14 hydra01 kernel: [2097557.541035] [] ? get_iowait_time+0x60/0x60 Sep 16 14:14:14 hydra01 kernel: [2097557.541037] [] stat_open+0x56/0xc0 Sep 16 14:14:14 hydra01 kernel: [2097557.541041] [] proc_reg_open+0xa0/0x180 Sep 16 14:14:14 hydra01 kernel: [2097557.541045] [] ? seq_release_private+0x60/0x60 Sep 16 14:14:14 hydra01 kernel: [2097557.541048] [] do_dentry_open+0x203/0x290 Sep 16 14:14:14 hydra01 kernel: [2097557.541051] [] ? init_once+0x20/0x20 Sep 16 14:14:14 hydra01 kernel: [2097557.541053] [] vfs_open+0x49/0x50 Sep 16 14:14:14 hydra01 kernel: [2097557.541056] [] do_last+0x296/0xea0 Sep 16 14:14:14 hydra01 kernel: [2097557.541058] [] ? inode_permission+0x18/0x50 Sep 16 14:14:14 hydra01 kernel: [2097557.541061] [] ? link_path_walk+0x6b/0x880 Sep 16 14:14:14 hydra01 kernel: [2097557.541063] [] path_openat+0xb3/0x4b0 Sep 16 14:14:14 hydra01 kernel: [2097557.541066] [] ? tlb_flush_mmu+0x5c/0xa0 Sep 16 14:14:14 hydra01 kernel: [2097557.541070] [] ? unmap_region+0xdb/0x120 Sep 16 14:14:14 hydra01 kernel: [2097557.541072] [] do_filp_open+0x41/0xa0 Sep 16 14:14:14 hydra01 kernel: [2097557.541075] [] ? __alloc_fd+0xd9/0x130 Sep 16 14:14:14 hydra01 kernel: [2097557.541078] [] do_sys_open+0xf3/0x230 Sep 16 14:14:14 hydra01 kernel: [2097557.541080] [] sys_open+0x21/0x30 Sep 16 14:14:14 hydra01 kernel: [2097557.541083] [] system_call_fastpath+0x1a/0x1f Sep 16 14:14:14 hydra01 kernel: [2097557.541085] Mem-Info: Sep 16 14:14:14 hydra01 kernel: [2097557.541086] Node 0 DMA per-cpu: Sep 16 14:14:14 hydra01 kernel: [2097557.541089] CPU 0: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541090] CPU 1: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541092] CPU 2: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541093] CPU 3: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541094] CPU 4: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541096] CPU 5: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541097] CPU 6: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541098] CPU 7: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541100] CPU 8: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541101] CPU 9: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541102] CPU 10: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541104] CPU 11: hi: 0, btch: 1 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541105] Node 0 DMA32 per-cpu: Sep 16 14:14:14 hydra01 kernel: [2097557.541107] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541108] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541110] CPU 2: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541111] CPU 3: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541113] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541114] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541115] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541117] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541118] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541119] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541121] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541122] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541123] Node 0 Normal per-cpu: Sep 16 14:14:14 hydra01 kernel: [2097557.541125] CPU 0: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541126] CPU 1: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541128] CPU 2: hi: 186, btch: 31 usd: 30 Sep 16 14:14:14 hydra01 kernel: [2097557.541129] CPU 3: hi: 186, btch: 31 usd: 30 Sep 16 14:14:14 hydra01 kernel: [2097557.541131] CPU 4: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541132] CPU 5: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541134] CPU 6: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541135] CPU 7: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541136] CPU 8: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541138] CPU 9: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541139] CPU 10: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541140] CPU 11: hi: 186, btch: 31 usd: 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541145] active_anon:1208121 inactive_anon:766069 isolated_anon:0 Sep 16 14:14:14 hydra01 kernel: [2097557.541145] active_file:2500046 inactive_file:3033956 isolated_file:0 Sep 16 14:14:14 hydra01 kernel: [2097557.541145] unevictable:0 dirty:81 writeback:0 unstable:0 Sep 16 14:14:14 hydra01 kernel: [2097557.541145] free:179978 slab_reclaimable:285040 slab_unreclaimable:106353 Sep 16 14:14:14 hydra01 kernel: [2097557.541145] mapped:37631 shmem:151 pagetables:9679 bounce:0 Sep 16 14:14:14 hydra01 kernel: [2097557.541145] free_cma:0 Sep 16 14:14:14 hydra01 kernel: [2097557.541148] Node 0 DMA free:15844kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15588kB managed:15844kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 16 14:14:14 hydra01 kernel: [2097557.541153] lowmem_reserve[]: 0 2967 32199 32199 Sep 16 14:14:14 hydra01 kernel: [2097557.541156] Node 0 DMA32 free:193292kB min:6224kB low:7780kB high:9336kB active_anon:457580kB inactive_anon:804492kB active_file:286652kB inactive_file:1112604kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3038624kB managed:3054680kB mlocked:0kB dirty:20kB writeback:0kB mapped:7040kB shmem:0kB slab_reclaimable:149948kB slab_unreclaimable:52596kB kernel_stack:1304kB pagetables:1564kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:14:14 hydra01 kernel: [2097557.541161] lowmem_reserve[]: 0 0 29232 29232 Sep 16 14:14:14 hydra01 kernel: [2097557.541164] Node 0 Normal free:510776kB min:61324kB low:76652kB high:91984kB active_anon:4374904kB inactive_anon:2259784kB active_file:9713532kB inactive_file:11023220kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:29933568kB managed:29857140kB mlocked:0kB dirty:304kB writeback:0kB mapped:143484kB shmem:604kB slab_reclaimable:990212kB slab_unreclaimable:372816kB kernel_stack:7688kB pagetables:37152kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no Sep 16 14:14:14 hydra01 kernel: [2097557.541170] lowmem_reserve[]: 0 0 0 0 Sep 16 14:14:14 hydra01 kernel: [2097557.541173] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15844kB Sep 16 14:14:14 hydra01 kernel: [2097557.541184] Node 0 DMA32: 19341*4kB (UEM) 3356*8kB (UEM) 5525*16kB (UEM) 3*32kB (UMR) 2*64kB (R) 0*128kB 0*256kB 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 193348kB Sep 16 14:14:14 hydra01 kernel: [2097557.541195] Node 0 Normal: 97683*4kB (UEMR) 7912*8kB (UEMR) 2830*16kB (UEMR) 286*32kB (UEMR) 28*64kB (UEMR) 2*128kB (R) 4*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 511532kB Sep 16 14:14:14 hydra01 kernel: [2097557.541205] 5950557 total pagecache pages Sep 16 14:14:14 hydra01 kernel: [2097557.541206] 416527 pages in swap cache Sep 16 14:14:14 hydra01 kernel: [2097557.541208] Swap cache stats: add 20944219, delete 20527692, find 12685682/14346361 Sep 16 14:14:14 hydra01 kernel: [2097557.541209] Free swap = 184867904kB Sep 16 14:14:14 hydra01 kernel: [2097557.541210] Total swap = 187617276kB Sep 16 14:14:14 hydra01 kernel: [2097557.647525] 8388592 pages RAM Sep 16 14:14:14 hydra01 kernel: [2097557.647528] 151696 pages reserved Sep 16 14:14:14 hydra01 kernel: [2097557.647530] 6933876 pages shared Sep 16 14:14:14 hydra01 kernel: [2097557.647531] 5107662 pages non-shared