Sep 15 00:21:48 localhost kernel: [75484.107812] cinder-api invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 Sep 15 00:21:48 localhost kernel: [75484.107818] cinder-api cpuset=/ mems_allowed=0 Sep 15 00:21:48 localhost kernel: [75484.107824] CPU: 2 PID: 6887 Comm: cinder-api Not tainted 3.13.0-24-generic #47-Ubuntu Sep 15 00:21:48 localhost kernel: [75484.107825] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014 Sep 15 00:21:48 localhost kernel: [75484.107827] 0000000000000000 ffff88001ddd9998 ffffffff81715ac4 ffff880134cb2fe0 Sep 15 00:21:48 localhost kernel: [75484.107831] ffff88001ddd9a20 ffffffff817103ff 0000000000000000 0000000000000000 Sep 15 00:21:48 localhost kernel: [75484.107833] 0000000000000000 0000000000000000 0000000000000000 0000000000000000 Sep 15 00:21:48 localhost kernel: [75484.107835] Call Trace: Sep 15 00:21:48 localhost kernel: [75484.107845] [] dump_stack+0x45/0x56 Sep 15 00:21:48 localhost kernel: [75484.107849] [] dump_header+0x7f/0x1f1 Sep 15 00:21:48 localhost kernel: [75484.107852] [] oom_kill_process+0x1ce/0x330 Sep 15 00:21:48 localhost kernel: [75484.107856] [] ? security_capable_noaudit+0x15/0x20 Sep 15 00:21:48 localhost kernel: [75484.107858] [] out_of_memory+0x414/0x450 Sep 15 00:21:48 localhost kernel: [75484.107860] [] __alloc_pages_nodemask+0x983/0xa20 Sep 15 00:21:48 localhost kernel: [75484.107863] [] alloc_pages_current+0xa3/0x160 Sep 15 00:21:48 localhost kernel: [75484.107866] [] __page_cache_alloc+0x97/0xc0 Sep 15 00:21:48 localhost kernel: [75484.107868] [] filemap_fault+0x185/0x410 Sep 15 00:21:48 localhost kernel: [75484.107871] [] __do_fault+0x6f/0x530 Sep 15 00:21:48 localhost kernel: [75484.107873] [] handle_mm_fault+0x492/0xf10 Sep 15 00:21:48 localhost kernel: [75484.107875] [] __do_page_fault+0x184/0x560 Sep 15 00:21:48 localhost kernel: [75484.107879] [] ? acct_account_cputime+0x1c/0x20 Sep 15 00:21:48 localhost kernel: [75484.107882] [] ? account_user_time+0x8b/0xa0 Sep 15 00:21:48 localhost kernel: [75484.107883] [] ? vtime_account_user+0x54/0x60 Sep 15 00:21:48 localhost kernel: [75484.107885] [] do_page_fault+0x1a/0x70 Sep 15 00:21:48 localhost kernel: [75484.107888] [] page_fault+0x28/0x30 Sep 15 00:21:48 localhost kernel: [75484.107889] Mem-Info: Sep 15 00:21:48 localhost kernel: [75484.107892] Node 0 DMA per-cpu: Sep 15 00:21:48 localhost kernel: [75484.107894] CPU 0: hi: 0, btch: 1 usd: 0 Sep 15 00:21:48 localhost kernel: [75484.107894] CPU 1: hi: 0, btch: 1 usd: 0 Sep 15 00:21:48 localhost kernel: [75484.107895] CPU 2: hi: 0, btch: 1 usd: 0 Sep 15 00:21:48 localhost kernel: [75484.107896] CPU 3: hi: 0, btch: 1 usd: 0 Sep 15 00:21:48 localhost kernel: [75484.107897] Node 0 DMA32 per-cpu: Sep 15 00:21:48 localhost kernel: [75484.107898] CPU 0: hi: 186, btch: 31 usd: 0 Sep 15 00:21:48 localhost kernel: [75484.107899] CPU 1: hi: 186, btch: 31 usd: 2 Sep 15 00:21:48 localhost kernel: [75484.107900] CPU 2: hi: 186, btch: 31 usd: 0 Sep 15 00:21:48 localhost kernel: [75484.107901] CPU 3: hi: 186, btch: 31 usd: 74 Sep 15 00:21:48 localhost kernel: [75484.107902] Node 0 Normal per-cpu: Sep 15 00:21:48 localhost kernel: [75484.107903] CPU 0: hi: 186, btch: 31 usd: 15 Sep 15 00:21:48 localhost kernel: [75484.107903] CPU 1: hi: 186, btch: 31 usd: 58 Sep 15 00:21:48 localhost kernel: [75484.107904] CPU 2: hi: 186, btch: 31 usd: 31 Sep 15 00:21:48 localhost kernel: [75484.107905] CPU 3: hi: 186, btch: 31 usd: 83 Sep 15 00:21:48 localhost kernel: [75484.107907] active_anon:687584 inactive_anon:260302 isolated_anon:0 Sep 15 00:21:48 localhost kernel: [75484.107907] active_file:49 inactive_file:61 isolated_file:32 Sep 15 00:21:48 localhost kernel: [75484.107907] unevictable:926 dirty:0 writeback:0 unstable:0 Sep 15 00:21:48 localhost kernel: [75484.107907] free:21736 slab_reclaimable:4477 slab_unreclaimable:7564 Sep 15 00:21:48 localhost kernel: [75484.107907] mapped:871 shmem:24 pagetables:7793 bounce:0 Sep 15 00:21:48 localhost kernel: [75484.107907] free_cma:0 Sep 15 00:21:48 localhost kernel: [75484.107910] Node 0 DMA free:15908kB min:264kB low:328kB high:396kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 15 00:21:48 localhost kernel: [75484.107913] lowmem_reserve[]: 0 2976 3934 3934 Sep 15 00:21:48 localhost kernel: [75484.107915] Node 0 DMA32 free:54672kB min:50912kB low:63640kB high:76368kB active_anon:2302112kB inactive_anon:592184kB active_file:112kB inactive_file:36kB unevictable:3260kB isolated(anon):0kB isolated(file):0kB present:3129280kB managed:3050360kB mlocked:3260kB dirty:0kB writeback:0kB mapped:2892kB shmem:72kB slab_reclaimable:9808kB slab_unreclaimable:18288kB kernel_stack:3000kB pagetables:23792kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:223 all_unreclaimable? yes Sep 15 00:21:48 localhost kernel: [75484.107919] lowmem_reserve[]: 0 0 958 958 Sep 15 00:21:48 localhost kernel: [75484.107921] Node 0 Normal free:16364kB min:16400kB low:20500kB high:24600kB active_anon:448224kB inactive_anon:449024kB active_file:208kB inactive_file:208kB unevictable:444kB isolated(anon):0kB isolated(file):0kB present:1048576kB managed:981812kB mlocked:444kB dirty:0kB writeback:0kB mapped:592kB shmem:24kB slab_reclaimable:8100kB slab_unreclaimable:11968kB kernel_stack:1000kB pagetables:7380kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:535 all_unreclaimable? no Sep 15 00:21:48 localhost kernel: [75484.107924] lowmem_reserve[]: 0 0 0 0 Sep 15 00:21:48 localhost kernel: [75484.107926] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15908kB Sep 15 00:21:48 localhost kernel: [75484.107935] Node 0 DMA32: 1048*4kB (UEMR) 1604*8kB (UEMR) 871*16kB (UEMR) 314*32kB (UEMR) 133*64kB (UEMR) 20*128kB (UEMR) 7*256kB (UEM) 0*512kB 1*1024kB (R) 0*2048kB 0*4096kB = 54896kB Sep 15 00:21:48 localhost kernel: [75484.107943] Node 0 Normal: 309*4kB (UE) 519*8kB (UEM) 225*16kB (UE) 102*32kB (UEM) 54*64kB (UEM) 5*128kB (EM) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 16348kB Sep 15 00:21:48 localhost kernel: [75484.107959] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Sep 15 00:21:48 localhost kernel: [75484.107960] 53292 total pagecache pages Sep 15 00:21:48 localhost kernel: [75484.107961] 52403 pages in swap cache Sep 15 00:21:48 localhost kernel: [75484.107963] Swap cache stats: add 1199325, delete 1146922, find 4164800/4285014 Sep 15 00:21:48 localhost kernel: [75484.107963] Free swap = 0kB Sep 15 00:21:48 localhost kernel: [75484.107964] Total swap = 1044476kB Sep 15 00:21:48 localhost kernel: [75484.107965] 1048462 pages RAM Sep 15 00:21:48 localhost kernel: [75484.107966] 0 pages HighMem/MovableOnly Sep 15 00:21:48 localhost kernel: [75484.107966] 16691 pages reserved Sep 15 00:21:48 localhost kernel: [75484.107967] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name Sep 15 00:21:48 localhost kernel: [75484.107972] [ 588] 0 588 4868 80 13 61 0 upstart-udev-br Sep 15 00:21:48 localhost kernel: [75484.107974] [ 595] 0 595 12867 149 27 174 -1000 systemd-udevd Sep 15 00:21:48 localhost kernel: [75484.107976] [ 965] 0 965 5858 155 17 42 0 rpcbind Sep 15 00:21:48 localhost kernel: [75484.107977] [ 1006] 106 1006 5385 154 16 115 0 rpc.statd Sep 15 00:21:48 localhost kernel: [75484.107979] [ 1022] 0 1022 2557 46 7 568 0 dhclient Sep 15 00:21:48 localhost kernel: [75484.107980] [ 1110] 0 1110 3911 45 11 125 0 upstart-socket- Sep 15 00:21:48 localhost kernel: [75484.107982] [ 1175] 0 1175 5894 74 15 93 0 rpc.idmapd Sep 15 00:21:48 localhost kernel: [75484.107983] [ 1176] 102 1176 10334 128 25 52 0 dbus-daemon Sep 15 00:21:48 localhost kernel: [75484.107985] [ 1183] 101 1183 64485 246 28 106 0 rsyslogd Sep 15 00:21:48 localhost kernel: [75484.107986] [ 1296] 0 1296 4823 79 14 69 0 bluetoothd Sep 15 00:21:48 localhost kernel: [75484.107987] [ 1348] 0 1348 8807 218 22 46 0 systemd-logind Sep 15 00:21:48 localhost kernel: [75484.107989] [ 1552] 0 1552 4161 43 13 383 0 upstart-file-br Sep 15 00:21:48 localhost kernel: [75484.107990] [ 1659] 0 1659 2557 126 8 518 0 dhclient Sep 15 00:21:48 localhost kernel: [75484.107992] [ 2026] 0 2026 1315 108 8 16 0 iscsid Sep 15 00:21:48 localhost kernel: [75484.107993] [ 2028] 0 2028 1440 890 8 0 -17 iscsid Sep 15 00:21:48 localhost kernel: [75484.107995] [ 2140] 0 2140 3954 169 13 37 0 getty Sep 15 00:21:48 localhost kernel: [75484.107996] [ 2144] 0 2144 3954 169 14 40 0 getty Sep 15 00:21:48 localhost kernel: [75484.107998] [ 2153] 0 2153 3954 169 13 37 0 getty Sep 15 00:21:48 localhost kernel: [75484.107999] [ 2154] 0 2154 3954 169 13 38 0 getty Sep 15 00:21:48 localhost kernel: [75484.108000] [ 2157] 0 2157 3954 169 13 36 0 getty Sep 15 00:21:48 localhost kernel: [75484.108002] [ 2184] 0 2184 15341 186 32 141 -1000 sshd Sep 15 00:21:48 localhost kernel: [75484.108003] [ 2196] 0 2196 4784 24 13 40 0 atd Sep 15 00:21:48 localhost kernel: [75484.108004] [ 2197] 0 2197 5913 149 18 30 0 cron Sep 15 00:21:48 localhost kernel: [75484.108006] [ 2215] 0 2215 1091 123 7 36 0 acpid Sep 15 00:21:48 localhost kernel: [75484.108007] [ 2218] 0 2218 4824 139 13 30 0 irqbalance Sep 15 00:21:48 localhost kernel: [75484.108009] [ 2223] 0 2223 4960 110 14 64 0 automount Sep 15 00:21:48 localhost kernel: [75484.108010] [ 2289] 110 2289 81348 149 23 122 0 memcached Sep 15 00:21:48 localhost kernel: [75484.108011] [ 2307] 0 2307 192894 170 42 27 0 nscd Sep 15 00:21:48 localhost kernel: [75484.108013] [ 2358] 108 2358 113216 351 53 127 0 nslcd Sep 15 00:21:48 localhost kernel: [75484.108014] [ 2449] 0 2449 20885 283 40 123 0 vmtoolsd Sep 15 00:21:48 localhost kernel: [75484.108015] [ 2960] 0 2960 1899 162 9 27 0 getty Sep 15 00:21:48 localhost kernel: [75484.108017] [29646] 111 29646 1873 41 9 17 0 epmd Sep 15 00:21:48 localhost kernel: [75484.108018] [29675] 111 29675 1110 115 7 24 0 rabbitmq-server Sep 15 00:21:48 localhost kernel: [75484.108019] [29684] 111 29684 569513 11494 132 1649 0 beam.smp Sep 15 00:21:48 localhost kernel: [75484.108021] [29815] 111 29815 1864 62 9 24 0 inet_gethost Sep 15 00:21:48 localhost kernel: [75484.108022] [29816] 111 29816 1917 62 9 30 0 inet_gethost Sep 15 00:21:48 localhost kernel: [75484.108024] [ 3348] 2050913 3348 593563 17130 124 5377 0 mysqld Sep 15 00:21:48 localhost kernel: [75484.108025] [ 3419] 2048681 3419 7734 1135 19 149 0 screen Sep 15 00:21:48 localhost kernel: [75484.108026] [ 3424] 2048681 3424 2493 144 10 117 0 bash Sep 15 00:21:48 localhost kernel: [75484.108027] [ 3501] 2048681 3501 2493 146 10 116 0 bash Sep 15 00:21:48 localhost kernel: [75484.108029] [ 3513] 2048681 3513 8597 881 21 642 0 dstat Sep 15 00:21:48 localhost kernel: [75484.108030] [ 3725] 2048681 3725 2493 146 10 116 0 bash Sep 15 00:21:48 localhost kernel: [75484.108031] [ 3738] 0 3738 29376 183 59 204 0 sudo Sep 15 00:21:48 localhost kernel: [75484.108033] [ 3746] 0 3746 1095 102 7 18 0 tail Sep 15 00:21:48 localhost kernel: [75484.108034] [ 3748] 2048681 3748 2493 146 9 116 0 bash Sep 15 00:21:48 localhost kernel: [75484.108035] [ 3760] 0 3760 29376 183 60 204 0 sudo Sep 15 00:21:48 localhost kernel: [75484.108037] [ 3764] 0 3764 1095 104 8 17 0 tail Sep 15 00:21:48 localhost kernel: [75484.108038] [ 4921] 0 4921 19126 466 41 51 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108039] [ 5071] 2048681 5071 2493 146 10 117 0 bash Sep 15 00:21:48 localhost kernel: [75484.108041] [ 5088] 0 5088 29376 183 59 205 0 sudo Sep 15 00:21:48 localhost kernel: [75484.108042] [ 5093] 0 5093 1095 93 8 23 0 tail Sep 15 00:21:48 localhost kernel: [75484.108043] [ 6275] 2048681 6275 2493 146 10 117 0 bash Sep 15 00:21:48 localhost kernel: [75484.108045] [ 6287] 2048681 6287 35183 614 72 15805 0 glance-registry Sep 15 00:21:48 localhost kernel: [75484.108046] [ 6296] 2048681 6296 2493 146 9 117 0 bash Sep 15 00:21:48 localhost kernel: [75484.108047] [ 6310] 2048681 6310 36975 12931 76 5238 0 glance-registry Sep 15 00:21:48 localhost kernel: [75484.108049] [ 6311] 2048681 6311 36975 13304 76 4900 0 glance-registry Sep 15 00:21:48 localhost kernel: [75484.108050] [ 6317] 2048681 6317 91159 1873 181 65740 0 glance-api Sep 15 00:21:48 localhost kernel: [75484.108051] [ 6418] 2048681 6418 2493 146 10 116 0 bash Sep 15 00:21:48 localhost kernel: [75484.108053] [ 6430] 2048681 6430 61696 9533 124 20286 0 nova-api Sep 15 00:21:48 localhost kernel: [75484.108054] [ 6447] 2048681 6447 54978 7357 107 15825 0 nova-api Sep 15 00:21:48 localhost kernel: [75484.108055] [ 6448] 2048681 6448 54978 7390 107 15825 0 nova-api Sep 15 00:21:48 localhost kernel: [75484.108057] [ 6451] 2048681 6451 71525 38409 141 1506 0 nova-api Sep 15 00:21:48 localhost kernel: [75484.108058] [ 6452] 2048681 6452 72309 39379 142 1304 0 nova-api Sep 15 00:21:48 localhost kernel: [75484.108059] [ 6462] 2048681 6462 61760 11430 120 18467 0 nova-api Sep 15 00:21:48 localhost kernel: [75484.108060] [ 6463] 2048681 6463 61696 9433 120 20194 0 nova-api Sep 15 00:21:48 localhost kernel: [75484.108062] [ 6484] 2048681 6484 2493 146 10 116 0 bash Sep 15 00:21:48 localhost kernel: [75484.108063] [ 6496] 2048681 6496 85703 4897 170 58993 0 python Sep 15 00:21:48 localhost kernel: [75484.108064] [ 6590] 2048681 6590 2494 200 10 79 0 bash Sep 15 00:21:48 localhost kernel: [75484.108066] [ 6617] 2048681 6617 2493 146 10 117 0 bash Sep 15 00:21:48 localhost kernel: [75484.108067] [ 6645] 2048681 6645 53848 21916 107 1890 0 nova-cert Sep 15 00:21:48 localhost kernel: [75484.108068] [ 6652] 2048681 6652 2493 146 10 116 0 bash Sep 15 00:21:48 localhost kernel: [75484.108069] [ 6681] 2048681 6681 52153 20234 106 1964 0 nova-network Sep 15 00:21:48 localhost kernel: [75484.108071] [ 6689] 2048681 6689 2494 219 11 61 0 bash Sep 15 00:21:48 localhost kernel: [75484.108072] [ 6731] 2048681 6731 2493 146 11 117 0 bash Sep 15 00:21:48 localhost kernel: [75484.108073] [ 6751] 2048681 6751 37345 1763 77 7420 0 nova-novncproxy Sep 15 00:21:48 localhost kernel: [75484.108075] [ 6766] 2048681 6766 2493 155 11 106 0 bash Sep 15 00:21:48 localhost kernel: [75484.108076] [ 6788] 2048681 6788 53708 22325 109 1410 0 nova-consoleaut Sep 15 00:21:48 localhost kernel: [75484.108077] [ 6810] 2048681 6810 2570 292 9 57 0 bash Sep 15 00:21:48 localhost kernel: [75484.108079] [ 6850] 0 6850 16188 146 31 128 -1000 tgtd Sep 15 00:21:48 localhost kernel: [75484.108080] [ 6852] 0 6852 3281 50 11 35 0 tgtd Sep 15 00:21:48 localhost kernel: [75484.108081] [ 6870] 2048681 6870 2493 146 10 116 0 bash Sep 15 00:21:48 localhost kernel: [75484.108083] [ 6887] 2048681 6887 44298 6638 90 9177 0 cinder-api Sep 15 00:21:48 localhost kernel: [75484.108084] [ 6900] 2048681 6900 49246 20020 96 695 0 cinder-api Sep 15 00:21:48 localhost kernel: [75484.108085] [ 6901] 2048681 6901 47991 17459 94 1916 0 cinder-api Sep 15 00:21:48 localhost kernel: [75484.108087] [ 6914] 2048681 6914 2493 146 10 114 0 bash Sep 15 00:21:48 localhost kernel: [75484.108088] [ 6926] 2048681 6926 43810 14435 89 966 0 cinder-schedule Sep 15 00:21:48 localhost kernel: [75484.108089] [ 6938] 2048681 6938 2493 154 10 107 0 bash Sep 15 00:21:48 localhost kernel: [75484.108091] [ 6959] 2048681 6959 91609 26507 178 36623 0 cinder-volume Sep 15 00:21:48 localhost kernel: [75484.108092] [11669] 2048681 11669 54280 23915 109 378 0 nova-scheduler Sep 15 00:21:48 localhost kernel: [75484.108093] [11679] 2048681 11679 51495 18839 103 2631 0 nova-conductor Sep 15 00:21:48 localhost kernel: [75484.108095] [11688] 2048681 11688 56924 26726 110 205 0 nova-conductor Sep 15 00:21:48 localhost kernel: [75484.108096] [11689] 2048681 11689 57080 26844 110 232 0 nova-conductor Sep 15 00:21:48 localhost kernel: [75484.108097] [11690] 2048681 11690 57187 26912 110 217 0 nova-conductor Sep 15 00:21:48 localhost kernel: [75484.108099] [11691] 2048681 11691 57947 27635 111 225 0 nova-conductor Sep 15 00:21:48 localhost kernel: [75484.108100] [15025] 2048681 15025 195345 30920 158 100 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108102] [15026] 2048681 15026 195409 31214 159 72 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108103] [15027] 2048681 15027 194511 29773 155 763 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108104] [15028] 2048681 15028 84303 14341 106 654 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108105] [15029] 2048681 15029 79605 13989 96 84 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108107] [15030] 2048681 15030 79605 13679 96 292 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108108] [15031] 2048681 15031 84239 14062 106 841 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108109] [15032] 2048681 15032 84303 14920 106 159 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108110] [15033] 2048681 15033 84623 15159 107 180 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108112] [15034] 2048681 15034 84495 15133 107 125 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108113] [15035] 2048681 15035 84623 15290 107 66 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108115] [15036] 2048681 15036 84559 15245 107 119 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108116] [15037] 2048681 15037 84559 14627 107 731 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108117] [15038] 33 15038 501914 2736 97 139 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108118] [15039] 33 15039 501870 2751 98 81 0 apache2 Sep 15 00:21:48 localhost kernel: [75484.108120] [27088] 2048681 27088 93555 9669 185 60262 0 glance-api Sep 15 00:21:48 localhost kernel: [75484.108121] [28480] 0 28480 36615 546 74 0 0 sshd Sep 15 00:21:48 localhost kernel: [75484.108123] [28530] 2048681 28530 36718 553 72 0 0 sshd Sep 15 00:21:48 localhost kernel: [75484.108124] [28531] 2048681 28531 3865 860 12 121 0 bash Sep 15 00:21:48 localhost kernel: [75484.108125] [28855] 0 28855 36615 588 74 0 0 sshd Sep 15 00:21:48 localhost kernel: [75484.108127] [28899] 2048681 28899 36680 479 71 0 0 sshd Sep 15 00:21:48 localhost kernel: [75484.108128] [28900] 2048681 28900 4044 1160 12 0 0 bash Sep 15 00:21:48 localhost kernel: [75484.108129] [30389] 2048681 30389 101598 77258 201 2427 0 python Sep 15 00:21:48 localhost kernel: [75484.108131] [30793] 2048681 30793 100484 20237 197 56573 0 glance-api Sep 15 00:21:48 localhost kernel: [75484.108132] [31082] 2048681 31082 90830 18322 179 50565 0 python Sep 15 00:21:48 localhost kernel: [75484.108133] [31097] 2048681 31097 140765 110517 273 1706 0 cinder-volume Sep 15 00:21:48 localhost kernel: [75484.108135] [31876] 2048681 31876 7309 203 19 0 0 screen Sep 15 00:21:48 localhost kernel: [75484.108136] [32080] 2048681 32080 109761 78193 218 0 0 nova-compute Sep 15 00:21:48 localhost kernel: [75484.108138] [32340] 0 32340 30089 393 60 0 0 sudo Sep 15 00:21:48 localhost kernel: [75484.108139] [32341] 0 32341 1808 131 10 0 0 tail Sep 15 00:21:48 localhost kernel: [75484.108140] Out of memory: Kill process 31097 (cinder-volume) score 88 or sacrifice child Sep 15 00:21:48 localhost kernel: [75484.109010] Killed process 31097 (cinder-volume) total-vm:563060kB, anon-rss:441260kB, file-rss:808kB Sep 15 00:21:53 localhost kernel: [75489.000509] mysqld invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 Sep 15 00:21:53 localhost kernel: [75489.000513] mysqld cpuset=/ mems_allowed=0 Sep 15 00:21:53 localhost kernel: [75489.000516] CPU: 0 PID: 3354 Comm: mysqld Not tainted 3.13.0-24-generic #47-Ubuntu Sep 15 00:21:53 localhost kernel: [75489.000517] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014 Sep 15 00:21:53 localhost kernel: [75489.000519] 0000000000000000 ffff880138d13998 ffffffff81715ac4 ffff8800b4d85fc0 Sep 15 00:21:53 localhost kernel: [75489.000521] ffff880138d13a20 ffffffff817103ff 0000000000000000 0000000000000000 Sep 15 00:21:53 localhost kernel: [75489.000523] 0000000000000000 0000000000000000 0000000000000000 0000000000000000 Sep 15 00:21:53 localhost kernel: [75489.000526] Call Trace: Sep 15 00:21:53 localhost kernel: [75489.000532] [] dump_stack+0x45/0x56 Sep 15 00:21:53 localhost kernel: [75489.000535] [] dump_header+0x7f/0x1f1 Sep 15 00:21:53 localhost kernel: [75489.000538] [] oom_kill_process+0x1ce/0x330 Sep 15 00:21:53 localhost kernel: [75489.000542] [] ? security_capable_noaudit+0x15/0x20 Sep 15 00:21:53 localhost kernel: [75489.000544] [] out_of_memory+0x414/0x450 Sep 15 00:21:53 localhost kernel: [75489.000547] [] __alloc_pages_nodemask+0x983/0xa20 Sep 15 00:21:53 localhost kernel: [75489.000549] [] alloc_pages_current+0xa3/0x160 Sep 15 00:21:53 localhost kernel: [75489.000552] [] __page_cache_alloc+0x97/0xc0 Sep 15 00:21:53 localhost kernel: [75489.000554] [] filemap_fault+0x185/0x410 Sep 15 00:21:53 localhost kernel: [75489.000557] [] __do_fault+0x6f/0x530 Sep 15 00:21:53 localhost kernel: [75489.000560] [] ? native_sched_clock+0x13/0x80 Sep 15 00:21:53 localhost kernel: [75489.000562] [] ? sched_clock+0x9/0x10 Sep 15 00:21:53 localhost kernel: [75489.000564] [] handle_mm_fault+0x492/0xf10 Sep 15 00:21:53 localhost kernel: [75489.000567] [] ? vtime_common_task_switch+0x3d/0x40 Sep 15 00:21:53 localhost kernel: [75489.000570] [] ? finish_task_switch+0x128/0x170 Sep 15 00:21:53 localhost kernel: [75489.000573] [] ? mutex_lock+0x12/0x2f Sep 15 00:21:53 localhost kernel: [75489.000576] [] ? lock_hrtimer_base.isra.19+0x25/0x50 Sep 15 00:21:53 localhost kernel: [75489.000578] [] __do_page_fault+0x184/0x560 Sep 15 00:21:53 localhost kernel: [75489.000582] [] ? acct_account_cputime+0x1c/0x20 Sep 15 00:21:53 localhost kernel: [75489.000584] [] ? account_user_time+0x8b/0xa0 Sep 15 00:21:53 localhost kernel: [75489.000586] [] ? vtime_account_user+0x54/0x60 Sep 15 00:21:53 localhost kernel: [75489.000587] [] do_page_fault+0x1a/0x70 Sep 15 00:21:53 localhost kernel: [75489.000589] [] page_fault+0x28/0x30 Sep 15 00:21:53 localhost kernel: [75489.000591] Mem-Info: Sep 15 00:21:53 localhost kernel: [75489.000592] Node 0 DMA per-cpu: Sep 15 00:21:53 localhost kernel: [75489.000594] CPU 0: hi: 0, btch: 1 usd: 0 Sep 15 00:21:53 localhost kernel: [75489.000595] CPU 1: hi: 0, btch: 1 usd: 0 Sep 15 00:21:53 localhost kernel: [75489.000596] CPU 2: hi: 0, btch: 1 usd: 0 Sep 15 00:21:53 localhost kernel: [75489.000596] CPU 3: hi: 0, btch: 1 usd: 0 Sep 15 00:21:53 localhost kernel: [75489.000597] Node 0 DMA32 per-cpu: Sep 15 00:21:53 localhost kernel: [75489.000599] CPU 0: hi: 186, btch: 31 usd: 30 Sep 15 00:21:53 localhost kernel: [75489.000600] CPU 1: hi: 186, btch: 31 usd: 2 Sep 15 00:21:53 localhost kernel: [75489.000600] CPU 2: hi: 186, btch: 31 usd: 140 Sep 15 00:21:53 localhost kernel: [75489.000601] CPU 3: hi: 186, btch: 31 usd: 26 Sep 15 00:21:53 localhost kernel: [75489.000602] Node 0 Normal per-cpu: Sep 15 00:21:53 localhost kernel: [75489.000603] CPU 0: hi: 186, btch: 31 usd: 0 Sep 15 00:21:53 localhost kernel: [75489.000604] CPU 1: hi: 186, btch: 31 usd: 29 Sep 15 00:21:53 localhost kernel: [75489.000605] CPU 2: hi: 186, btch: 31 usd: 170 Sep 15 00:21:53 localhost kernel: [75489.000605] CPU 3: hi: 186, btch: 31 usd: 30 Sep 15 00:21:53 localhost kernel: [75489.000608] active_anon:657962 inactive_anon:290060 isolated_anon:0 Sep 15 00:21:53 localhost kernel: [75489.000608] active_file:116 inactive_file:33 isolated_file:24 Sep 15 00:21:53 localhost kernel: [75489.000608] unevictable:926 dirty:0 writeback:0 unstable:0 Sep 15 00:21:53 localhost kernel: [75489.000608] free:21717 slab_reclaimable:4431 slab_unreclaimable:7564 Sep 15 00:21:53 localhost kernel: [75489.000608] mapped:754 shmem:24 pagetables:7718 bounce:0 Sep 15 00:21:53 localhost kernel: [75489.000608] free_cma:0 Sep 15 00:21:53 localhost kernel: [75489.000610] Node 0 DMA free:15908kB min:264kB low:328kB high:396kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 15 00:21:53 localhost kernel: [75489.000614] lowmem_reserve[]: 0 2976 3934 3934 Sep 15 00:21:53 localhost kernel: [75489.000618] Node 0 DMA32 free:54616kB min:50912kB low:63640kB high:76368kB active_anon:2183200kB inactive_anon:711452kB active_file:280kB inactive_file:60kB unevictable:3260kB isolated(anon):0kB isolated(file):0kB present:3129280kB managed:3050360kB mlocked:3260kB dirty:0kB writeback:0kB mapped:2488kB shmem:72kB slab_reclaimable:9628kB slab_unreclaimable:18288kB kernel_stack:3000kB pagetables:23572kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:526 all_unreclaimable? yes Sep 15 00:21:53 localhost kernel: [75489.000622] lowmem_reserve[]: 0 0 958 958 Sep 15 00:21:53 localhost kernel: [75489.000625] Node 0 Normal free:16344kB min:16400kB low:20500kB high:24600kB active_anon:448648kB inactive_anon:448788kB active_file:184kB inactive_file:72kB unevictable:444kB isolated(anon):0kB isolated(file):96kB present:1048576kB managed:981812kB mlocked:444kB dirty:0kB writeback:0kB mapped:528kB shmem:24kB slab_reclaimable:8096kB slab_unreclaimable:11968kB kernel_stack:1000kB pagetables:7300kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:411 all_unreclaimable? yes Sep 15 00:21:53 localhost kernel: [75489.000629] lowmem_reserve[]: 0 0 0 0 Sep 15 00:21:53 localhost kernel: [75489.000632] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15908kB Sep 15 00:21:53 localhost kernel: [75489.000642] Node 0 DMA32: 1090*4kB (UEMR) 1587*8kB (UER) 869*16kB (UEMR) 304*32kB (UEMR) 134*64kB (UEMR) 20*128kB (UEMR) 7*256kB (UEM) 0*512kB 1*1024kB (R) 0*2048kB 0*4096kB = 54640kB Sep 15 00:21:53 localhost kernel: [75489.000649] Node 0 Normal: 329*4kB (UEM) 520*8kB (UEM) 227*16kB (UE) 98*32kB (UEM) 54*64kB (UEM) 5*128kB (EM) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 16340kB Sep 15 00:21:53 localhost kernel: [75489.000657] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Sep 15 00:21:53 localhost kernel: [75489.000658] 93623 total pagecache pages Sep 15 00:21:53 localhost kernel: [75489.000659] 92698 pages in swap cache Sep 15 00:21:53 localhost kernel: [75489.000660] Swap cache stats: add 1239631, delete 1146933, find 4194910/4320708 Sep 15 00:21:53 localhost kernel: [75489.000661] Free swap = 0kB Sep 15 00:21:53 localhost kernel: [75489.000662] Total swap = 1044476kB Sep 15 00:21:53 localhost kernel: [75489.000663] 1048462 pages RAM Sep 15 00:21:53 localhost kernel: [75489.000663] 0 pages HighMem/MovableOnly Sep 15 00:21:53 localhost kernel: [75489.000664] 16691 pages reserved Sep 15 00:21:53 localhost kernel: [75489.000665] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name Sep 15 00:21:53 localhost kernel: [75489.000670] [ 588] 0 588 4868 80 13 61 0 upstart-udev-br Sep 15 00:21:53 localhost kernel: [75489.000672] [ 595] 0 595 12867 149 27 174 -1000 systemd-udevd Sep 15 00:21:53 localhost kernel: [75489.000673] [ 965] 0 965 5858 155 17 42 0 rpcbind Sep 15 00:21:53 localhost kernel: [75489.000675] [ 1006] 106 1006 5385 154 16 115 0 rpc.statd Sep 15 00:21:53 localhost kernel: [75489.000677] [ 1022] 0 1022 2557 46 7 568 0 dhclient Sep 15 00:21:53 localhost kernel: [75489.000678] [ 1110] 0 1110 3911 45 11 125 0 upstart-socket- Sep 15 00:21:53 localhost kernel: [75489.000679] [ 1175] 0 1175 5894 74 15 93 0 rpc.idmapd Sep 15 00:21:53 localhost kernel: [75489.000681] [ 1176] 102 1176 10334 128 25 52 0 dbus-daemon Sep 15 00:21:53 localhost kernel: [75489.000682] [ 1183] 101 1183 64485 189 28 106 0 rsyslogd Sep 15 00:21:53 localhost kernel: [75489.000684] [ 1296] 0 1296 4823 79 14 69 0 bluetoothd Sep 15 00:21:53 localhost kernel: [75489.000685] [ 1348] 0 1348 8807 218 22 46 0 systemd-logind Sep 15 00:21:53 localhost kernel: [75489.000686] [ 1552] 0 1552 4161 43 13 383 0 upstart-file-br Sep 15 00:21:53 localhost kernel: [75489.000688] [ 1659] 0 1659 2557 126 8 518 0 dhclient Sep 15 00:21:53 localhost kernel: [75489.000689] [ 2026] 0 2026 1315 108 8 16 0 iscsid Sep 15 00:21:53 localhost kernel: [75489.000691] [ 2028] 0 2028 1440 890 8 0 -17 iscsid Sep 15 00:21:53 localhost kernel: [75489.000692] [ 2140] 0 2140 3954 169 13 37 0 getty Sep 15 00:21:53 localhost kernel: [75489.000694] [ 2144] 0 2144 3954 169 14 40 0 getty Sep 15 00:21:53 localhost kernel: [75489.000695] [ 2153] 0 2153 3954 169 13 37 0 getty Sep 15 00:21:53 localhost kernel: [75489.000696] [ 2154] 0 2154 3954 169 13 38 0 getty Sep 15 00:21:53 localhost kernel: [75489.000698] [ 2157] 0 2157 3954 169 13 36 0 getty Sep 15 00:21:53 localhost kernel: [75489.000699] [ 2184] 0 2184 15341 186 32 141 -1000 sshd Sep 15 00:21:53 localhost kernel: [75489.000700] [ 2196] 0 2196 4784 24 13 40 0 atd Sep 15 00:21:53 localhost kernel: [75489.000701] [ 2197] 0 2197 5913 149 18 30 0 cron Sep 15 00:21:53 localhost kernel: [75489.000703] [ 2215] 0 2215 1091 123 7 36 0 acpid Sep 15 00:21:53 localhost kernel: [75489.000704] [ 2218] 0 2218 4824 139 13 30 0 irqbalance Sep 15 00:21:53 localhost kernel: [75489.000705] [ 2223] 0 2223 4960 110 14 64 0 automount Sep 15 00:21:53 localhost kernel: [75489.000707] [ 2289] 110 2289 81348 136 23 122 0 memcached Sep 15 00:21:53 localhost kernel: [75489.000708] [ 2307] 0 2307 192894 169 42 27 0 nscd Sep 15 00:21:53 localhost kernel: [75489.000710] [ 2358] 108 2358 113216 350 53 127 0 nslcd Sep 15 00:21:53 localhost kernel: [75489.000711] [ 2449] 0 2449 20885 288 40 123 0 vmtoolsd Sep 15 00:21:53 localhost kernel: [75489.000712] [ 2960] 0 2960 1899 162 9 27 0 getty Sep 15 00:21:53 localhost kernel: [75489.000714] [29646] 111 29646 1873 38 9 17 0 epmd Sep 15 00:21:53 localhost kernel: [75489.000715] [29675] 111 29675 1110 115 7 24 0 rabbitmq-server Sep 15 00:21:53 localhost kernel: [75489.000717] [29684] 111 29684 568485 11550 133 1649 0 beam.smp Sep 15 00:21:53 localhost kernel: [75489.000718] [29815] 111 29815 1864 62 9 24 0 inet_gethost Sep 15 00:21:53 localhost kernel: [75489.000719] [29816] 111 29816 1917 62 9 30 0 inet_gethost Sep 15 00:21:53 localhost kernel: [75489.000721] [ 3348] 2050913 3348 593563 17130 124 5377 0 mysqld Sep 15 00:21:53 localhost kernel: [75489.000722] [ 3419] 2048681 3419 7734 1137 19 149 0 screen Sep 15 00:21:53 localhost kernel: [75489.000724] [ 3424] 2048681 3424 2493 144 10 117 0 bash Sep 15 00:21:53 localhost kernel: [75489.000725] [ 3501] 2048681 3501 2493 146 10 116 0 bash Sep 15 00:21:53 localhost kernel: [75489.000726] [ 3513] 2048681 3513 8597 868 21 642 0 dstat Sep 15 00:21:53 localhost kernel: [75489.000727] [ 3725] 2048681 3725 2493 146 10 116 0 bash Sep 15 00:21:53 localhost kernel: [75489.000729] [ 3738] 0 3738 29376 183 59 204 0 sudo Sep 15 00:21:53 localhost kernel: [75489.000730] [ 3746] 0 3746 1095 102 7 18 0 tail Sep 15 00:21:53 localhost kernel: [75489.000731] [ 3748] 2048681 3748 2493 146 9 116 0 bash Sep 15 00:21:53 localhost kernel: [75489.000733] [ 3760] 0 3760 29376 183 60 204 0 sudo Sep 15 00:21:53 localhost kernel: [75489.000734] [ 3764] 0 3764 1095 104 8 17 0 tail Sep 15 00:21:53 localhost kernel: [75489.000735] [ 4921] 0 4921 19126 448 41 51 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000737] [ 5071] 2048681 5071 2493 146 10 117 0 bash Sep 15 00:21:53 localhost kernel: [75489.000738] [ 5088] 0 5088 29376 183 59 205 0 sudo Sep 15 00:21:53 localhost kernel: [75489.000739] [ 5093] 0 5093 1095 93 8 23 0 tail Sep 15 00:21:53 localhost kernel: [75489.000741] [ 6275] 2048681 6275 2493 146 10 117 0 bash Sep 15 00:21:53 localhost kernel: [75489.000742] [ 6287] 2048681 6287 35183 614 72 15805 0 glance-registry Sep 15 00:21:53 localhost kernel: [75489.000743] [ 6296] 2048681 6296 2493 146 9 117 0 bash Sep 15 00:21:53 localhost kernel: [75489.000745] [ 6310] 2048681 6310 36975 12926 76 5238 0 glance-registry Sep 15 00:21:53 localhost kernel: [75489.000746] [ 6311] 2048681 6311 36975 13348 76 4900 0 glance-registry Sep 15 00:21:53 localhost kernel: [75489.000748] [ 6317] 2048681 6317 91159 1865 181 65740 0 glance-api Sep 15 00:21:53 localhost kernel: [75489.000749] [ 6418] 2048681 6418 2493 146 10 116 0 bash Sep 15 00:21:53 localhost kernel: [75489.000750] [ 6430] 2048681 6430 61696 9521 124 20289 0 nova-api Sep 15 00:21:53 localhost kernel: [75489.000752] [ 6447] 2048681 6447 54978 7357 107 15825 0 nova-api Sep 15 00:21:53 localhost kernel: [75489.000753] [ 6448] 2048681 6448 54978 7390 107 15825 0 nova-api Sep 15 00:21:53 localhost kernel: [75489.000755] [ 6451] 2048681 6451 71525 38361 141 1509 0 nova-api Sep 15 00:21:53 localhost kernel: [75489.000756] [ 6452] 2048681 6452 72309 39360 142 1307 0 nova-api Sep 15 00:21:53 localhost kernel: [75489.000757] [ 6462] 2048681 6462 61760 11427 120 18470 0 nova-api Sep 15 00:21:53 localhost kernel: [75489.000758] [ 6463] 2048681 6463 61696 9430 120 20197 0 nova-api Sep 15 00:21:53 localhost kernel: [75489.000760] [ 6484] 2048681 6484 2493 146 10 116 0 bash Sep 15 00:21:53 localhost kernel: [75489.000761] [ 6496] 2048681 6496 85703 4875 170 58993 0 python Sep 15 00:21:53 localhost kernel: [75489.000762] [ 6590] 2048681 6590 2494 200 10 79 0 bash Sep 15 00:21:53 localhost kernel: [75489.000763] [ 6617] 2048681 6617 2493 146 10 117 0 bash Sep 15 00:21:53 localhost kernel: [75489.000765] [ 6645] 2048681 6645 53848 21861 107 1890 0 nova-cert Sep 15 00:21:53 localhost kernel: [75489.000766] [ 6652] 2048681 6652 2493 146 10 116 0 bash Sep 15 00:21:53 localhost kernel: [75489.000767] [ 6681] 2048681 6681 52153 20210 106 1964 0 nova-network Sep 15 00:21:53 localhost kernel: [75489.000769] [ 6689] 2048681 6689 2494 219 11 61 0 bash Sep 15 00:21:53 localhost kernel: [75489.000770] [ 6731] 2048681 6731 2493 146 11 117 0 bash Sep 15 00:21:53 localhost kernel: [75489.000771] [ 6751] 2048681 6751 37345 1718 77 7420 0 nova-novncproxy Sep 15 00:21:53 localhost kernel: [75489.000772] [ 6766] 2048681 6766 2493 155 11 106 0 bash Sep 15 00:21:53 localhost kernel: [75489.000774] [ 6788] 2048681 6788 53708 22308 109 1410 0 nova-consoleaut Sep 15 00:21:53 localhost kernel: [75489.000775] [ 6810] 2048681 6810 2570 292 9 57 0 bash Sep 15 00:21:53 localhost kernel: [75489.000777] [ 6850] 0 6850 16188 135 31 128 -1000 tgtd Sep 15 00:21:53 localhost kernel: [75489.000778] [ 6852] 0 6852 3281 48 11 35 0 tgtd Sep 15 00:21:53 localhost kernel: [75489.000779] [ 6870] 2048681 6870 2493 146 10 116 0 bash Sep 15 00:21:53 localhost kernel: [75489.000781] [ 6887] 2048681 6887 44298 6678 90 9177 0 cinder-api Sep 15 00:21:53 localhost kernel: [75489.000782] [ 6900] 2048681 6900 49246 20015 96 695 0 cinder-api Sep 15 00:21:53 localhost kernel: [75489.000783] [ 6901] 2048681 6901 47991 17454 94 1916 0 cinder-api Sep 15 00:21:53 localhost kernel: [75489.000784] [ 6914] 2048681 6914 2493 146 10 114 0 bash Sep 15 00:21:53 localhost kernel: [75489.000786] [ 6926] 2048681 6926 43810 14433 89 966 0 cinder-schedule Sep 15 00:21:53 localhost kernel: [75489.000787] [ 6938] 2048681 6938 2493 154 10 107 0 bash Sep 15 00:21:53 localhost kernel: [75489.000789] [ 6959] 2048681 6959 91609 26551 178 36620 0 cinder-volume Sep 15 00:21:53 localhost kernel: [75489.000790] [11669] 2048681 11669 54280 23914 109 378 0 nova-scheduler Sep 15 00:21:53 localhost kernel: [75489.000792] [11679] 2048681 11679 51495 18830 103 2631 0 nova-conductor Sep 15 00:21:53 localhost kernel: [75489.000793] [11688] 2048681 11688 56988 26736 110 205 0 nova-conductor Sep 15 00:21:53 localhost kernel: [75489.000795] [11689] 2048681 11689 57080 26823 110 232 0 nova-conductor Sep 15 00:21:53 localhost kernel: [75489.000796] [11690] 2048681 11690 57187 26887 110 217 0 nova-conductor Sep 15 00:21:53 localhost kernel: [75489.000797] [11691] 2048681 11691 57947 27612 111 225 0 nova-conductor Sep 15 00:21:53 localhost kernel: [75489.000799] [15025] 2048681 15025 195345 30950 158 100 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000800] [15026] 2048681 15026 195409 31194 159 72 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000801] [15027] 2048681 15027 194511 29761 156 763 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000803] [15028] 2048681 15028 84303 14321 106 654 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000804] [15029] 2048681 15029 79605 13970 96 84 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000805] [15030] 2048681 15030 79605 13659 96 292 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000807] [15031] 2048681 15031 84239 14043 106 841 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000808] [15032] 2048681 15032 84303 14899 106 159 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000809] [15033] 2048681 15033 84623 15199 107 180 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000811] [15034] 2048681 15034 84495 15165 107 125 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000812] [15035] 2048681 15035 84623 15263 107 66 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000813] [15036] 2048681 15036 84559 15225 107 119 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000815] [15037] 2048681 15037 84559 14606 107 731 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000816] [15038] 33 15038 501914 2736 97 139 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000817] [15039] 33 15039 501870 2751 98 81 0 apache2 Sep 15 00:21:53 localhost kernel: [75489.000819] [27088] 2048681 27088 93555 9638 185 60262 0 glance-api Sep 15 00:21:53 localhost kernel: [75489.000820] [28480] 0 28480 36615 546 74 0 0 sshd Sep 15 00:21:53 localhost kernel: [75489.000821] [28530] 2048681 28530 36718 517 72 0 0 sshd Sep 15 00:21:53 localhost kernel: [75489.000823] [28531] 2048681 28531 3865 860 12 121 0 bash Sep 15 00:21:53 localhost kernel: [75489.000824] [28855] 0 28855 36615 588 74 0 0 sshd Sep 15 00:21:53 localhost kernel: [75489.000825] [28899] 2048681 28899 36680 492 71 0 0 sshd Sep 15 00:21:53 localhost kernel: [75489.000827] [28900] 2048681 28900 4044 1160 12 0 0 bash Sep 15 00:21:53 localhost kernel: [75489.000828] [30389] 2048681 30389 101598 77263 201 2427 0 python Sep 15 00:21:53 localhost kernel: [75489.000829] [30793] 2048681 30793 100484 36235 197 40570 0 glance-api Sep 15 00:21:53 localhost kernel: [75489.000831] [31082] 2048681 31082 90927 18396 179 50564 0 python Sep 15 00:21:53 localhost kernel: [75489.000832] [31876] 2048681 31876 7309 198 19 0 0 screen Sep 15 00:21:53 localhost kernel: [75489.000834] [32080] 2048681 32080 111680 80061 221 0 0 nova-compute Sep 15 00:21:53 localhost kernel: [75489.000835] [32340] 0 32340 30089 393 60 0 0 sudo Sep 15 00:21:53 localhost kernel: [75489.000836] [32341] 0 32341 1808 124 10 0 0 tail Sep 15 00:21:53 localhost kernel: [75489.000838] [32350] 2048681 32350 99537 47996 193 23006 0 cinder-volume Sep 15 00:21:53 localhost kernel: [75489.000839] Out of memory: Kill process 32080 (nova-compute) score 63 or sacrifice child Sep 15 00:21:53 localhost kernel: [75489.001675] Killed process 32080 (nova-compute) total-vm:446720kB, anon-rss:319052kB, file-rss:1192kB Sep 15 00:21:59 localhost kernel: [75495.087540] tgtd invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 Sep 15 00:21:59 localhost kernel: [75495.087544] tgtd cpuset=/ mems_allowed=0 Sep 15 00:21:59 localhost kernel: [75495.087547] CPU: 1 PID: 6852 Comm: tgtd Not tainted 3.13.0-24-generic #47-Ubuntu Sep 15 00:21:59 localhost kernel: [75495.087548] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014 Sep 15 00:21:59 localhost kernel: [75495.087549] 0000000000000000 ffff88001dca5998 ffffffff81715ac4 ffff880137095fc0 Sep 15 00:21:59 localhost kernel: [75495.087552] ffff88001dca5a20 ffffffff817103ff 0000000000000000 0000000000000000 Sep 15 00:21:59 localhost kernel: [75495.087554] 0000000000000000 0000000000000000 0000000000000000 0000000000000000 Sep 15 00:21:59 localhost kernel: [75495.087556] Call Trace: Sep 15 00:21:59 localhost kernel: [75495.087562] [] dump_stack+0x45/0x56 Sep 15 00:21:59 localhost kernel: [75495.087566] [] dump_header+0x7f/0x1f1 Sep 15 00:21:59 localhost kernel: [75495.087569] [] oom_kill_process+0x1ce/0x330 Sep 15 00:21:59 localhost kernel: [75495.087572] [] ? security_capable_noaudit+0x15/0x20 Sep 15 00:21:59 localhost kernel: [75495.087574] [] out_of_memory+0x414/0x450 Sep 15 00:21:59 localhost kernel: [75495.087576] [] __alloc_pages_nodemask+0x983/0xa20 Sep 15 00:21:59 localhost kernel: [75495.087579] [] alloc_pages_current+0xa3/0x160 Sep 15 00:21:59 localhost kernel: [75495.087582] [] __page_cache_alloc+0x97/0xc0 Sep 15 00:21:59 localhost kernel: [75495.087584] [] filemap_fault+0x185/0x410 Sep 15 00:21:59 localhost kernel: [75495.087587] [] __do_fault+0x6f/0x530 Sep 15 00:21:59 localhost kernel: [75495.087590] [] ? update_rq_clock.part.63+0x18/0x40 Sep 15 00:21:59 localhost kernel: [75495.087592] [] handle_mm_fault+0x492/0xf10 Sep 15 00:21:59 localhost kernel: [75495.087595] [] ? arch_vtime_task_switch+0x94/0xa0 Sep 15 00:21:59 localhost kernel: [75495.087596] [] ? vtime_common_task_switch+0x3d/0x40 Sep 15 00:21:59 localhost kernel: [75495.087598] [] ? finish_task_switch+0x128/0x170 Sep 15 00:21:59 localhost kernel: [75495.087601] [] __do_page_fault+0x184/0x560 Sep 15 00:21:59 localhost kernel: [75495.087604] [] ? acct_account_cputime+0x1c/0x20 Sep 15 00:21:59 localhost kernel: [75495.087606] [] ? account_user_time+0x8b/0xa0 Sep 15 00:21:59 localhost kernel: [75495.087607] [] ? vtime_account_user+0x54/0x60 Sep 15 00:21:59 localhost kernel: [75495.087609] [] do_page_fault+0x1a/0x70 Sep 15 00:21:59 localhost kernel: [75495.087612] [] page_fault+0x28/0x30 Sep 15 00:21:59 localhost kernel: [75495.087613] Mem-Info: Sep 15 00:21:59 localhost kernel: [75495.087614] Node 0 DMA per-cpu: Sep 15 00:21:59 localhost kernel: [75495.087616] CPU 0: hi: 0, btch: 1 usd: 0 Sep 15 00:21:59 localhost kernel: [75495.087616] CPU 1: hi: 0, btch: 1 usd: 0 Sep 15 00:21:59 localhost kernel: [75495.087617] CPU 2: hi: 0, btch: 1 usd: 0 Sep 15 00:21:59 localhost kernel: [75495.087618] CPU 3: hi: 0, btch: 1 usd: 0 Sep 15 00:21:59 localhost kernel: [75495.087619] Node 0 DMA32 per-cpu: Sep 15 00:21:59 localhost kernel: [75495.087620] CPU 0: hi: 186, btch: 31 usd: 1 Sep 15 00:21:59 localhost kernel: [75495.087621] CPU 1: hi: 186, btch: 31 usd: 1 Sep 15 00:21:59 localhost kernel: [75495.087622] CPU 2: hi: 186, btch: 31 usd: 0 Sep 15 00:21:59 localhost kernel: [75495.087623] CPU 3: hi: 186, btch: 31 usd: 3 Sep 15 00:21:59 localhost kernel: [75495.087623] Node 0 Normal per-cpu: Sep 15 00:21:59 localhost kernel: [75495.087624] CPU 0: hi: 186, btch: 31 usd: 12 Sep 15 00:21:59 localhost kernel: [75495.087625] CPU 1: hi: 186, btch: 31 usd: 30 Sep 15 00:21:59 localhost kernel: [75495.087626] CPU 2: hi: 186, btch: 31 usd: 2 Sep 15 00:21:59 localhost kernel: [75495.087627] CPU 3: hi: 186, btch: 31 usd: 4 Sep 15 00:21:59 localhost kernel: [75495.087629] active_anon:622487 inactive_anon:326015 isolated_anon:0 Sep 15 00:21:59 localhost kernel: [75495.087629] active_file:73 inactive_file:0 isolated_file:0 Sep 15 00:21:59 localhost kernel: [75495.087629] unevictable:926 dirty:0 writeback:0 unstable:0 Sep 15 00:21:59 localhost kernel: [75495.087629] free:21731 slab_reclaimable:4405 slab_unreclaimable:7557 Sep 15 00:21:59 localhost kernel: [75495.087629] mapped:729 shmem:24 pagetables:7525 bounce:0 Sep 15 00:21:59 localhost kernel: [75495.087629] free_cma:0 Sep 15 00:21:59 localhost kernel: [75495.087631] Node 0 DMA free:15908kB min:264kB low:328kB high:396kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 15 00:21:59 localhost kernel: [75495.087635] lowmem_reserve[]: 0 2976 3934 3934 Sep 15 00:21:59 localhost kernel: [75495.087637] Node 0 DMA32 free:54672kB min:50912kB low:63640kB high:76368kB active_anon:2076240kB inactive_anon:819316kB active_file:248kB inactive_file:0kB unevictable:3260kB isolated(anon):0kB isolated(file):0kB present:3129280kB managed:3050360kB mlocked:3260kB dirty:0kB writeback:0kB mapped:2656kB shmem:72kB slab_reclaimable:9548kB slab_unreclaimable:18256kB kernel_stack:3000kB pagetables:22988kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:378 all_unreclaimable? yes Sep 15 00:21:59 localhost kernel: [75495.087640] lowmem_reserve[]: 0 0 958 958 Sep 15 00:21:59 localhost kernel: [75495.087642] Node 0 Normal free:16344kB min:16400kB low:20500kB high:24600kB active_anon:413708kB inactive_anon:484744kB active_file:44kB inactive_file:0kB unevictable:444kB isolated(anon):0kB isolated(file):0kB present:1048576kB managed:981812kB mlocked:444kB dirty:0kB writeback:0kB mapped:260kB shmem:24kB slab_reclaimable:8072kB slab_unreclaimable:11972kB kernel_stack:992kB pagetables:7112kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:73 all_unreclaimable? yes Sep 15 00:21:59 localhost kernel: [75495.087644] lowmem_reserve[]: 0 0 0 0 Sep 15 00:21:59 localhost kernel: [75495.087646] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15908kB Sep 15 00:21:59 localhost kernel: [75495.087654] Node 0 DMA32: 1282*4kB (UEMR) 1590*8kB (UEMR) 871*16kB (UEMR) 286*32kB (UEMR) 136*64kB (UEMR) 20*128kB (UEMR) 7*256kB (UEM) 0*512kB 1*1024kB (R) 0*2048kB 0*4096kB = 55016kB Sep 15 00:21:59 localhost kernel: [75495.087662] Node 0 Normal: 376*4kB (UEM) 526*8kB (UEM) 231*16kB (UEM) 93*32kB (UEM) 54*64kB (UEM) 5*128kB (EM) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 16480kB Sep 15 00:21:59 localhost kernel: [75495.087669] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Sep 15 00:21:59 localhost kernel: [75495.087670] 129670 total pagecache pages Sep 15 00:21:59 localhost kernel: [75495.087671] 128806 pages in swap cache Sep 15 00:21:59 localhost kernel: [75495.087673] Swap cache stats: add 1275769, delete 1146963, find 4234366/4364974 Sep 15 00:21:59 localhost kernel: [75495.087673] Free swap = 0kB Sep 15 00:21:59 localhost kernel: [75495.087674] Total swap = 1044476kB Sep 15 00:21:59 localhost kernel: [75495.087675] 1048462 pages RAM Sep 15 00:21:59 localhost kernel: [75495.087676] 0 pages HighMem/MovableOnly Sep 15 00:21:59 localhost kernel: [75495.087676] 16691 pages reserved Sep 15 00:21:59 localhost kernel: [75495.087677] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name Sep 15 00:21:59 localhost kernel: [75495.087682] [ 588] 0 588 4868 80 13 61 0 upstart-udev-br Sep 15 00:21:59 localhost kernel: [75495.087684] [ 595] 0 595 12867 149 27 174 -1000 systemd-udevd Sep 15 00:21:59 localhost kernel: [75495.087685] [ 965] 0 965 5858 155 17 42 0 rpcbind Sep 15 00:21:59 localhost kernel: [75495.087687] [ 1006] 106 1006 5385 154 16 115 0 rpc.statd Sep 15 00:21:59 localhost kernel: [75495.087688] [ 1022] 0 1022 2557 46 7 568 0 dhclient Sep 15 00:21:59 localhost kernel: [75495.087690] [ 1110] 0 1110 3911 45 11 125 0 upstart-socket- Sep 15 00:21:59 localhost kernel: [75495.087691] [ 1175] 0 1175 5894 74 15 93 0 rpc.idmapd Sep 15 00:21:59 localhost kernel: [75495.087692] [ 1176] 102 1176 10334 128 25 52 0 dbus-daemon Sep 15 00:21:59 localhost kernel: [75495.087694] [ 1183] 101 1183 64485 244 28 106 0 rsyslogd Sep 15 00:21:59 localhost kernel: [75495.087695] [ 1296] 0 1296 4823 79 14 69 0 bluetoothd Sep 15 00:21:59 localhost kernel: [75495.087696] [ 1348] 0 1348 8807 218 22 46 0 systemd-logind Sep 15 00:21:59 localhost kernel: [75495.087698] [ 1552] 0 1552 4161 43 13 383 0 upstart-file-br Sep 15 00:21:59 localhost kernel: [75495.087699] [ 1659] 0 1659 2557 126 8 518 0 dhclient Sep 15 00:21:59 localhost kernel: [75495.087701] [ 2026] 0 2026 1315 108 8 16 0 iscsid Sep 15 00:21:59 localhost kernel: [75495.087702] [ 2028] 0 2028 1440 890 8 0 -17 iscsid Sep 15 00:21:59 localhost kernel: [75495.087703] [ 2140] 0 2140 3954 169 13 37 0 getty Sep 15 00:21:59 localhost kernel: [75495.087705] [ 2144] 0 2144 3954 169 14 40 0 getty Sep 15 00:21:59 localhost kernel: [75495.087706] [ 2153] 0 2153 3954 169 13 37 0 getty Sep 15 00:21:59 localhost kernel: [75495.087707] [ 2154] 0 2154 3954 169 13 38 0 getty Sep 15 00:21:59 localhost kernel: [75495.087708] [ 2157] 0 2157 3954 169 13 36 0 getty Sep 15 00:21:59 localhost kernel: [75495.087710] [ 2184] 0 2184 15341 186 32 141 -1000 sshd Sep 15 00:21:59 localhost kernel: [75495.087711] [ 2196] 0 2196 4784 24 13 40 0 atd Sep 15 00:21:59 localhost kernel: [75495.087712] [ 2197] 0 2197 5913 149 18 30 0 cron Sep 15 00:21:59 localhost kernel: [75495.087714] [ 2215] 0 2215 1091 123 7 36 0 acpid Sep 15 00:21:59 localhost kernel: [75495.087715] [ 2218] 0 2218 4824 139 13 30 0 irqbalance Sep 15 00:21:59 localhost kernel: [75495.087716] [ 2223] 0 2223 4960 110 14 64 0 automount Sep 15 00:21:59 localhost kernel: [75495.087718] [ 2289] 110 2289 81348 150 23 122 0 memcached Sep 15 00:21:59 localhost kernel: [75495.087719] [ 2307] 0 2307 192894 169 42 27 0 nscd Sep 15 00:21:59 localhost kernel: [75495.087720] [ 2358] 108 2358 113216 350 53 127 0 nslcd Sep 15 00:21:59 localhost kernel: [75495.087722] [ 2449] 0 2449 20885 299 40 123 0 vmtoolsd Sep 15 00:21:59 localhost kernel: [75495.087723] [ 2960] 0 2960 1899 162 9 27 0 getty Sep 15 00:21:59 localhost kernel: [75495.087724] [29646] 111 29646 1873 84 9 17 0 epmd Sep 15 00:21:59 localhost kernel: [75495.087726] [29675] 111 29675 1110 115 7 24 0 rabbitmq-server Sep 15 00:21:59 localhost kernel: [75495.087727] [29684] 111 29684 566561 11320 131 1649 0 beam.smp Sep 15 00:21:59 localhost kernel: [75495.087729] [29815] 111 29815 1864 62 9 24 0 inet_gethost Sep 15 00:21:59 localhost kernel: [75495.087730] [29816] 111 29816 1917 62 9 30 0 inet_gethost Sep 15 00:21:59 localhost kernel: [75495.087731] [ 3348] 2050913 3348 593563 17130 124 5377 0 mysqld Sep 15 00:21:59 localhost kernel: [75495.087733] [ 3419] 2048681 3419 7734 1124 19 149 0 screen Sep 15 00:21:59 localhost kernel: [75495.087734] [ 3424] 2048681 3424 2493 144 10 117 0 bash Sep 15 00:21:59 localhost kernel: [75495.087735] [ 3501] 2048681 3501 2493 146 10 116 0 bash Sep 15 00:21:59 localhost kernel: [75495.087736] [ 3513] 2048681 3513 8597 804 21 642 0 dstat Sep 15 00:21:59 localhost kernel: [75495.087738] [ 3725] 2048681 3725 2493 146 10 116 0 bash Sep 15 00:21:59 localhost kernel: [75495.087739] [ 3738] 0 3738 29376 183 59 204 0 sudo Sep 15 00:21:59 localhost kernel: [75495.087740] [ 3746] 0 3746 1095 102 7 18 0 tail Sep 15 00:21:59 localhost kernel: [75495.087742] [ 3748] 2048681 3748 2493 146 9 116 0 bash Sep 15 00:21:59 localhost kernel: [75495.087743] [ 3760] 0 3760 29376 183 60 204 0 sudo Sep 15 00:21:59 localhost kernel: [75495.087744] [ 3764] 0 3764 1095 104 8 17 0 tail Sep 15 00:21:59 localhost kernel: [75495.087746] [ 4921] 0 4921 19126 467 41 51 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087747] [ 5071] 2048681 5071 2493 146 10 117 0 bash Sep 15 00:21:59 localhost kernel: [75495.087748] [ 5088] 0 5088 29376 183 59 205 0 sudo Sep 15 00:21:59 localhost kernel: [75495.087749] [ 5093] 0 5093 1095 93 8 23 0 tail Sep 15 00:21:59 localhost kernel: [75495.087751] [ 6275] 2048681 6275 2493 146 10 117 0 bash Sep 15 00:21:59 localhost kernel: [75495.087752] [ 6287] 2048681 6287 35183 614 72 15805 0 glance-registry Sep 15 00:21:59 localhost kernel: [75495.087753] [ 6296] 2048681 6296 2493 146 9 117 0 bash Sep 15 00:21:59 localhost kernel: [75495.087755] [ 6310] 2048681 6310 36975 12911 76 5238 0 glance-registry Sep 15 00:21:59 localhost kernel: [75495.087756] [ 6311] 2048681 6311 36975 13287 76 4900 0 glance-registry Sep 15 00:21:59 localhost kernel: [75495.087757] [ 6317] 2048681 6317 91159 1800 181 65740 0 glance-api Sep 15 00:21:59 localhost kernel: [75495.087759] [ 6418] 2048681 6418 2493 146 10 116 0 bash Sep 15 00:21:59 localhost kernel: [75495.087760] [ 6430] 2048681 6430 61696 9493 124 20298 0 nova-api Sep 15 00:21:59 localhost kernel: [75495.087761] [ 6447] 2048681 6447 54978 7332 107 15825 0 nova-api Sep 15 00:21:59 localhost kernel: [75495.087763] [ 6448] 2048681 6448 54978 7365 107 15825 0 nova-api Sep 15 00:21:59 localhost kernel: [75495.087764] [ 6451] 2048681 6451 71525 38348 141 1511 0 nova-api Sep 15 00:21:59 localhost kernel: [75495.087765] [ 6452] 2048681 6452 72309 39388 142 1308 0 nova-api Sep 15 00:21:59 localhost kernel: [75495.087766] [ 6462] 2048681 6462 61760 11393 120 18479 0 nova-api Sep 15 00:21:59 localhost kernel: [75495.087768] [ 6463] 2048681 6463 61696 9396 120 20206 0 nova-api Sep 15 00:21:59 localhost kernel: [75495.087769] [ 6484] 2048681 6484 2493 146 10 116 0 bash Sep 15 00:21:59 localhost kernel: [75495.087770] [ 6496] 2048681 6496 85703 4823 170 58993 0 python Sep 15 00:21:59 localhost kernel: [75495.087772] [ 6590] 2048681 6590 2494 200 10 79 0 bash Sep 15 00:21:59 localhost kernel: [75495.087773] [ 6617] 2048681 6617 2493 146 10 117 0 bash Sep 15 00:21:59 localhost kernel: [75495.087774] [ 6645] 2048681 6645 53848 21874 107 1890 0 nova-cert Sep 15 00:21:59 localhost kernel: [75495.087776] [ 6652] 2048681 6652 2493 146 10 116 0 bash Sep 15 00:21:59 localhost kernel: [75495.087777] [ 6681] 2048681 6681 52153 20206 106 1964 0 nova-network Sep 15 00:21:59 localhost kernel: [75495.087778] [ 6689] 2048681 6689 2494 219 11 61 0 bash Sep 15 00:21:59 localhost kernel: [75495.087779] [ 6731] 2048681 6731 2493 146 11 117 0 bash Sep 15 00:21:59 localhost kernel: [75495.087781] [ 6751] 2048681 6751 37345 1705 77 7420 0 nova-novncproxy Sep 15 00:21:59 localhost kernel: [75495.087782] [ 6766] 2048681 6766 2493 155 11 106 0 bash Sep 15 00:21:59 localhost kernel: [75495.087784] [ 6788] 2048681 6788 53708 22270 109 1409 0 nova-consoleaut Sep 15 00:21:59 localhost kernel: [75495.087785] [ 6810] 2048681 6810 2570 300 9 57 0 bash Sep 15 00:21:59 localhost kernel: [75495.087786] [ 6850] 0 6850 16188 139 31 128 -1000 tgtd Sep 15 00:21:59 localhost kernel: [75495.087788] [ 6852] 0 6852 3281 73 11 35 0 tgtd Sep 15 00:21:59 localhost kernel: [75495.087789] [ 6870] 2048681 6870 2493 146 10 116 0 bash Sep 15 00:21:59 localhost kernel: [75495.087790] [ 6887] 2048681 6887 44298 6656 90 9177 0 cinder-api Sep 15 00:21:59 localhost kernel: [75495.087792] [ 6900] 2048681 6900 49246 19959 96 695 0 cinder-api Sep 15 00:21:59 localhost kernel: [75495.087793] [ 6901] 2048681 6901 47991 17398 94 1916 0 cinder-api Sep 15 00:21:59 localhost kernel: [75495.087794] [ 6914] 2048681 6914 2493 146 10 114 0 bash Sep 15 00:21:59 localhost kernel: [75495.087795] [ 6926] 2048681 6926 43810 14437 89 966 0 cinder-schedule Sep 15 00:21:59 localhost kernel: [75495.087797] [ 6938] 2048681 6938 2493 154 10 107 0 bash Sep 15 00:21:59 localhost kernel: [75495.087798] [ 6959] 2048681 6959 91609 26485 178 36620 0 cinder-volume Sep 15 00:21:59 localhost kernel: [75495.087800] [11669] 2048681 11669 54280 23905 109 378 0 nova-scheduler Sep 15 00:21:59 localhost kernel: [75495.087801] [11679] 2048681 11679 51495 18814 103 2631 0 nova-conductor Sep 15 00:21:59 localhost kernel: [75495.087802] [11688] 2048681 11688 56988 26732 110 205 0 nova-conductor Sep 15 00:21:59 localhost kernel: [75495.087804] [11689] 2048681 11689 57080 26815 110 232 0 nova-conductor Sep 15 00:21:59 localhost kernel: [75495.087805] [11690] 2048681 11690 57187 26925 110 217 0 nova-conductor Sep 15 00:21:59 localhost kernel: [75495.087806] [11691] 2048681 11691 57947 27595 111 225 0 nova-conductor Sep 15 00:21:59 localhost kernel: [75495.087808] [15025] 2048681 15025 195345 30948 158 100 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087809] [15026] 2048681 15026 195409 31207 159 72 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087810] [15027] 2048681 15027 194511 29724 156 745 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087812] [15028] 2048681 15028 84303 14340 106 654 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087813] [15029] 2048681 15029 79605 13992 96 84 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087814] [15030] 2048681 15030 79605 13674 96 292 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087815] [15031] 2048681 15031 84239 14053 106 841 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087817] [15032] 2048681 15032 84303 14913 106 159 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087818] [15033] 2048681 15033 84623 15155 107 180 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087819] [15034] 2048681 15034 84495 15111 107 131 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087820] [15035] 2048681 15035 84623 15272 107 66 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087822] [15036] 2048681 15036 84559 15246 107 119 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087823] [15037] 2048681 15037 84559 14628 107 731 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087824] [15038] 33 15038 501914 2736 97 139 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087825] [15039] 33 15039 501870 2751 98 81 0 apache2 Sep 15 00:21:59 localhost kernel: [75495.087827] [27088] 2048681 27088 93555 9601 185 60262 0 glance-api Sep 15 00:21:59 localhost kernel: [75495.087828] [28480] 0 28480 36615 546 74 0 0 sshd Sep 15 00:21:59 localhost kernel: [75495.087829] [28530] 2048681 28530 36718 543 72 0 0 sshd Sep 15 00:21:59 localhost kernel: [75495.087831] [28531] 2048681 28531 3865 860 12 121 0 bash Sep 15 00:21:59 localhost kernel: [75495.087832] [28855] 0 28855 36615 588 74 0 0 sshd Sep 15 00:21:59 localhost kernel: [75495.087833] [28899] 2048681 28899 36680 516 71 0 0 sshd Sep 15 00:21:59 localhost kernel: [75495.087835] [28900] 2048681 28900 4044 1160 12 0 0 bash Sep 15 00:21:59 localhost kernel: [75495.087836] [30389] 2048681 30389 101598 77197 201 2427 0 python Sep 15 00:21:59 localhost kernel: [75495.087837] [30793] 2048681 30793 100484 54377 197 22375 0 glance-api Sep 15 00:21:59 localhost kernel: [75495.087839] [31082] 2048681 31082 90927 18364 179 50564 0 python Sep 15 00:21:59 localhost kernel: [75495.087840] [31876] 2048681 31876 7309 198 19 0 0 screen Sep 15 00:21:59 localhost kernel: [75495.087841] [32340] 0 32340 30089 393 60 0 0 sudo Sep 15 00:21:59 localhost kernel: [75495.087843] [32341] 0 32341 1808 117 10 0 0 tail Sep 15 00:21:59 localhost kernel: [75495.087844] [32350] 2048681 32350 114836 84166 223 2108 0 cinder-volume Sep 15 00:21:59 localhost kernel: [75495.087845] Out of memory: Kill process 32350 (cinder-volume) score 67 or sacrifice child Sep 15 00:21:59 localhost kernel: [75495.088668] Killed process 32350 (cinder-volume) total-vm:459344kB, anon-rss:335984kB, file-rss:680kB Sep 15 00:22:02 localhost kernel: [75498.073390] mysqld invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0 Sep 15 00:22:02 localhost kernel: [75498.073395] mysqld cpuset=/ mems_allowed=0 Sep 15 00:22:02 localhost kernel: [75498.073397] CPU: 2 PID: 3360 Comm: mysqld Not tainted 3.13.0-24-generic #47-Ubuntu Sep 15 00:22:02 localhost kernel: [75498.073399] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/30/2014 Sep 15 00:22:02 localhost kernel: [75498.073400] 0000000000000000 ffff880137d9f998 ffffffff81715ac4 ffff8801371bc7d0 Sep 15 00:22:02 localhost kernel: [75498.073403] ffff880137d9fa20 ffffffff817103ff 0000000000000000 0000000000000000 Sep 15 00:22:02 localhost kernel: [75498.073405] 0000000000000000 0000000000000000 0000000000000000 0000000000000000 Sep 15 00:22:02 localhost kernel: [75498.073407] Call Trace: Sep 15 00:22:02 localhost kernel: [75498.073413] [] dump_stack+0x45/0x56 Sep 15 00:22:02 localhost kernel: [75498.073417] [] dump_header+0x7f/0x1f1 Sep 15 00:22:02 localhost kernel: [75498.073419] [] oom_kill_process+0x1ce/0x330 Sep 15 00:22:02 localhost kernel: [75498.073423] [] ? security_capable_noaudit+0x15/0x20 Sep 15 00:22:02 localhost kernel: [75498.073425] [] out_of_memory+0x414/0x450 Sep 15 00:22:02 localhost kernel: [75498.073427] [] __alloc_pages_nodemask+0x983/0xa20 Sep 15 00:22:02 localhost kernel: [75498.073430] [] alloc_pages_current+0xa3/0x160 Sep 15 00:22:02 localhost kernel: [75498.073433] [] __page_cache_alloc+0x97/0xc0 Sep 15 00:22:02 localhost kernel: [75498.073435] [] filemap_fault+0x185/0x410 Sep 15 00:22:02 localhost kernel: [75498.073438] [] __do_fault+0x6f/0x530 Sep 15 00:22:02 localhost kernel: [75498.073441] [] ? native_sched_clock+0x13/0x80 Sep 15 00:22:02 localhost kernel: [75498.073443] [] ? sched_clock+0x9/0x10 Sep 15 00:22:02 localhost kernel: [75498.073445] [] handle_mm_fault+0x492/0xf10 Sep 15 00:22:02 localhost kernel: [75498.073448] [] ? vtime_common_task_switch+0x3d/0x40 Sep 15 00:22:02 localhost kernel: [75498.073451] [] ? finish_task_switch+0x128/0x170 Sep 15 00:22:02 localhost kernel: [75498.073454] [] ? mutex_lock+0x12/0x2f Sep 15 00:22:02 localhost kernel: [75498.073456] [] ? lock_hrtimer_base.isra.19+0x25/0x50 Sep 15 00:22:02 localhost kernel: [75498.073459] [] __do_page_fault+0x184/0x560 Sep 15 00:22:02 localhost kernel: [75498.073463] [] ? acct_account_cputime+0x1c/0x20 Sep 15 00:22:02 localhost kernel: [75498.073464] [] ? account_user_time+0x8b/0xa0 Sep 15 00:22:02 localhost kernel: [75498.073466] [] ? vtime_account_user+0x54/0x60 Sep 15 00:22:02 localhost kernel: [75498.073467] [] do_page_fault+0x1a/0x70 Sep 15 00:22:02 localhost kernel: [75498.073470] [] page_fault+0x28/0x30 Sep 15 00:22:02 localhost kernel: [75498.073471] Mem-Info: Sep 15 00:22:02 localhost kernel: [75498.073472] Node 0 DMA per-cpu: Sep 15 00:22:02 localhost kernel: [75498.073474] CPU 0: hi: 0, btch: 1 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073475] CPU 1: hi: 0, btch: 1 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073476] CPU 2: hi: 0, btch: 1 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073476] CPU 3: hi: 0, btch: 1 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073477] Node 0 DMA32 per-cpu: Sep 15 00:22:02 localhost kernel: [75498.073478] CPU 0: hi: 186, btch: 31 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073479] CPU 1: hi: 186, btch: 31 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073480] CPU 2: hi: 186, btch: 31 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073481] CPU 3: hi: 186, btch: 31 usd: 9 Sep 15 00:22:02 localhost kernel: [75498.073481] Node 0 Normal per-cpu: Sep 15 00:22:02 localhost kernel: [75498.073483] CPU 0: hi: 186, btch: 31 usd: 31 Sep 15 00:22:02 localhost kernel: [75498.073484] CPU 1: hi: 186, btch: 31 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073484] CPU 2: hi: 186, btch: 31 usd: 0 Sep 15 00:22:02 localhost kernel: [75498.073485] CPU 3: hi: 186, btch: 31 usd: 49 Sep 15 00:22:02 localhost kernel: [75498.073488] active_anon:619850 inactive_anon:328660 isolated_anon:0 Sep 15 00:22:02 localhost kernel: [75498.073488] active_file:43 inactive_file:0 isolated_file:0 Sep 15 00:22:02 localhost kernel: [75498.073488] unevictable:926 dirty:0 writeback:0 unstable:0 Sep 15 00:22:02 localhost kernel: [75498.073488] free:21721 slab_reclaimable:4429 slab_unreclaimable:7557 Sep 15 00:22:02 localhost kernel: [75498.073488] mapped:773 shmem:24 pagetables:7510 bounce:0 Sep 15 00:22:02 localhost kernel: [75498.073488] free_cma:0 Sep 15 00:22:02 localhost kernel: [75498.073490] Node 0 DMA free:15908kB min:264kB low:328kB high:396kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Sep 15 00:22:02 localhost kernel: [75498.073494] lowmem_reserve[]: 0 2976 3934 3934 Sep 15 00:22:02 localhost kernel: [75498.073496] Node 0 DMA32 free:54684kB min:50912kB low:63640kB high:76368kB active_anon:2058920kB inactive_anon:836536kB active_file:124kB inactive_file:0kB unevictable:3260kB isolated(anon):0kB isolated(file):0kB present:3129280kB managed:3050360kB mlocked:3260kB dirty:0kB writeback:0kB mapped:2776kB shmem:72kB slab_reclaimable:9628kB slab_unreclaimable:18256kB kernel_stack:3000kB pagetables:22924kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:325 all_unreclaimable? yes Sep 15 00:22:02 localhost kernel: [75498.073499] lowmem_reserve[]: 0 0 958 958 Sep 15 00:22:02 localhost kernel: [75498.073501] Node 0 Normal free:16292kB min:16400kB low:20500kB high:24600kB active_anon:420480kB inactive_anon:478104kB active_file:48kB inactive_file:0kB unevictable:444kB isolated(anon):0kB isolated(file):0kB present:1048576kB managed:981812kB mlocked:444kB dirty:0kB writeback:0kB mapped:316kB shmem:24kB slab_reclaimable:8088kB slab_unreclaimable:11972kB kernel_stack:992kB pagetables:7116kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:72 all_unreclaimable? yes Sep 15 00:22:02 localhost kernel: [75498.073504] lowmem_reserve[]: 0 0 0 0 Sep 15 00:22:02 localhost kernel: [75498.073506] Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15908kB Sep 15 00:22:02 localhost kernel: [75498.073514] Node 0 DMA32: 1301*4kB (UEMR) 1589*8kB (UEMR) 858*16kB (UEMR) 289*32kB (UEMR) 136*64kB (UEMR) 20*128kB (UEMR) 7*256kB (UEM) 0*512kB 1*1024kB (R) 0*2048kB 0*4096kB = 54972kB Sep 15 00:22:02 localhost kernel: [75498.073521] Node 0 Normal: 371*4kB (UE) 523*8kB (UE) 229*16kB (UEM) 93*32kB (UEM) 54*64kB (UEM) 5*128kB (EM) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 16404kB Sep 15 00:22:02 localhost kernel: [75498.073528] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Sep 15 00:22:02 localhost kernel: [75498.073529] 134681 total pagecache pages Sep 15 00:22:02 localhost kernel: [75498.073531] 134007 pages in swap cache Sep 15 00:22:02 localhost kernel: [75498.073532] Swap cache stats: add 1280981, delete 1146974, find 4288342/4419607 Sep 15 00:22:02 localhost kernel: [75498.073533] Free swap = 0kB Sep 15 00:22:02 localhost kernel: [75498.073533] Total swap = 1044476kB Sep 15 00:22:02 localhost kernel: [75498.073534] 1048462 pages RAM Sep 15 00:22:02 localhost kernel: [75498.073535] 0 pages HighMem/MovableOnly Sep 15 00:22:02 localhost kernel: [75498.073535] 16691 pages reserved Sep 15 00:22:02 localhost kernel: [75498.073536] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name Sep 15 00:22:02 localhost kernel: [75498.073541] [ 588] 0 588 4868 80 13 61 0 upstart-udev-br Sep 15 00:22:02 localhost kernel: [75498.073543] [ 595] 0 595 12867 149 27 174 -1000 systemd-udevd Sep 15 00:22:02 localhost kernel: [75498.073544] [ 965] 0 965 5858 155 17 42 0 rpcbind Sep 15 00:22:02 localhost kernel: [75498.073546] [ 1006] 106 1006 5385 154 16 115 0 rpc.statd Sep 15 00:22:02 localhost kernel: [75498.073547] [ 1022] 0 1022 2557 46 7 568 0 dhclient Sep 15 00:22:02 localhost kernel: [75498.073549] [ 1110] 0 1110 3911 45 11 125 0 upstart-socket- Sep 15 00:22:02 localhost kernel: [75498.073550] [ 1175] 0 1175 5894 74 15 93 0 rpc.idmapd Sep 15 00:22:02 localhost kernel: [75498.073551] [ 1176] 102 1176 10334 128 25 52 0 dbus-daemon Sep 15 00:22:02 localhost kernel: [75498.073553] [ 1183] 101 1183 64485 181 28 106 0 rsyslogd Sep 15 00:22:02 localhost kernel: [75498.073554] [ 1296] 0 1296 4823 79 14 69 0 bluetoothd Sep 15 00:22:02 localhost kernel: [75498.073555] [ 1348] 0 1348 8807 218 22 46 0 systemd-logind Sep 15 00:22:02 localhost kernel: [75498.073557] [ 1552] 0 1552 4161 43 13 383 0 upstart-file-br Sep 15 00:22:02 localhost kernel: [75498.073558] [ 1659] 0 1659 2557 126 8 518 0 dhclient Sep 15 00:22:02 localhost kernel: [75498.073560] [ 2026] 0 2026 1315 108 8 16 0 iscsid Sep 15 00:22:02 localhost kernel: [75498.073561] [ 2028] 0 2028 1440 890 8 0 -17 iscsid Sep 15 00:22:02 localhost kernel: [75498.073562] [ 2140] 0 2140 3954 169 13 37 0 getty Sep 15 00:22:02 localhost kernel: [75498.073564] [ 2144] 0 2144 3954 169 14 40 0 getty Sep 15 00:22:02 localhost kernel: [75498.073565] [ 2153] 0 2153 3954 169 13 37 0 getty Sep 15 00:22:02 localhost kernel: [75498.073566] [ 2154] 0 2154 3954 169 13 38 0 getty Sep 15 00:22:02 localhost kernel: [75498.073568] [ 2157] 0 2157 3954 169 13 36 0 getty Sep 15 00:22:02 localhost kernel: [75498.073569] [ 2184] 0 2184 15341 186 32 141 -1000 sshd Sep 15 00:22:02 localhost kernel: [75498.073570] [ 2196] 0 2196 4784 24 13 40 0 atd Sep 15 00:22:02 localhost kernel: [75498.073572] [ 2197] 0 2197 5913 144 18 30 0 cron Sep 15 00:22:02 localhost kernel: [75498.073573] [ 2215] 0 2215 1091 123 7 36 0 acpid Sep 15 00:22:02 localhost kernel: [75498.073574] [ 2218] 0 2218 4824 138 13 30 0 irqbalance Sep 15 00:22:02 localhost kernel: [75498.073576] [ 2223] 0 2223 4960 110 14 64 0 automount Sep 15 00:22:02 localhost kernel: [75498.073577] [ 2289] 110 2289 81348 136 23 122 0 memcached Sep 15 00:22:02 localhost kernel: [75498.073578] [ 2307] 0 2307 192894 167 42 27 0 nscd Sep 15 00:22:02 localhost kernel: [75498.073579] [ 2358] 108 2358 113216 348 53 127 0 nslcd Sep 15 00:22:02 localhost kernel: [75498.073581] [ 2449] 0 2449 20885 272 40 123 0 vmtoolsd Sep 15 00:22:02 localhost kernel: [75498.073582] [ 2960] 0 2960 1899 162 9 27 0 getty Sep 15 00:22:02 localhost kernel: [75498.073583] [29646] 111 29646 1873 81 9 17 0 epmd Sep 15 00:22:02 localhost kernel: [75498.073585] [29675] 111 29675 1110 115 7 24 0 rabbitmq-server Sep 15 00:22:02 localhost kernel: [75498.073586] [29684] 111 29684 565919 11329 130 1649 0 beam.smp Sep 15 00:22:02 localhost kernel: [75498.073587] [29815] 111 29815 1864 62 9 24 0 inet_gethost Sep 15 00:22:02 localhost kernel: [75498.073589] [29816] 111 29816 1917 62 9 30 0 inet_gethost Sep 15 00:22:02 localhost kernel: [75498.073591] [ 3348] 2050913 3348 593563 17130 124 5377 0 mysqld Sep 15 00:22:02 localhost kernel: [75498.073592] [ 3419] 2048681 3419 7734 1127 19 149 0 screen Sep 15 00:22:02 localhost kernel: [75498.073593] [ 3424] 2048681 3424 2493 144 10 117 0 bash Sep 15 00:22:02 localhost kernel: [75498.073594] [ 3501] 2048681 3501 2493 146 10 116 0 bash Sep 15 00:22:02 localhost kernel: [75498.073596] [ 3513] 2048681 3513 8597 783 21 642 0 dstat Sep 15 00:22:02 localhost kernel: [75498.073597] [ 3725] 2048681 3725 2493 146 10 116 0 bash Sep 15 00:22:02 localhost kernel: [75498.073598] [ 3738] 0 3738 29376 183 59 204 0 sudo Sep 15 00:22:02 localhost kernel: [75498.073599] [ 3746] 0 3746 1095 102 7 18 0 tail Sep 15 00:22:02 localhost kernel: [75498.073601] [ 3748] 2048681 3748 2493 146 9 116 0 bash Sep 15 00:22:02 localhost kernel: [75498.073602] [ 3760] 0 3760 29376 183 60 204 0 sudo Sep 15 00:22:02 localhost kernel: [75498.073603] [ 3764] 0 3764 1095 104 8 17 0 tail Sep 15 00:22:02 localhost kernel: [75498.073604] [ 4921] 0 4921 19126 454 41 51 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073606] [ 5071] 2048681 5071 2493 146 10 117 0 bash Sep 15 00:22:02 localhost kernel: [75498.073607] [ 5088] 0 5088 29376 183 59 205 0 sudo Sep 15 00:22:02 localhost kernel: [75498.073608] [ 5093] 0 5093 1095 93 8 23 0 tail Sep 15 00:22:02 localhost kernel: [75498.073609] [ 6275] 2048681 6275 2493 146 10 117 0 bash Sep 15 00:22:02 localhost kernel: [75498.073611] [ 6287] 2048681 6287 35183 614 72 15805 0 glance-registry Sep 15 00:22:02 localhost kernel: [75498.073612] [ 6296] 2048681 6296 2493 146 9 117 0 bash Sep 15 00:22:02 localhost kernel: [75498.073614] [ 6310] 2048681 6310 36975 12868 76 5238 0 glance-registry Sep 15 00:22:02 localhost kernel: [75498.073615] [ 6311] 2048681 6311 36975 13246 76 4900 0 glance-registry Sep 15 00:22:02 localhost kernel: [75498.073616] [ 6317] 2048681 6317 91159 1754 181 65740 0 glance-api Sep 15 00:22:02 localhost kernel: [75498.073618] [ 6418] 2048681 6418 2493 146 10 116 0 bash Sep 15 00:22:02 localhost kernel: [75498.073619] [ 6430] 2048681 6430 61696 9439 124 20298 0 nova-api Sep 15 00:22:02 localhost kernel: [75498.073620] [ 6447] 2048681 6447 54978 7310 107 15825 0 nova-api Sep 15 00:22:02 localhost kernel: [75498.073622] [ 6448] 2048681 6448 54978 7343 107 15825 0 nova-api Sep 15 00:22:02 localhost kernel: [75498.073623] [ 6451] 2048681 6451 71525 38260 141 1511 0 nova-api Sep 15 00:22:02 localhost kernel: [75498.073624] [ 6452] 2048681 6452 72309 39248 142 1308 0 nova-api Sep 15 00:22:02 localhost kernel: [75498.073626] [ 6462] 2048681 6462 61760 11383 120 18479 0 nova-api Sep 15 00:22:02 localhost kernel: [75498.073627] [ 6463] 2048681 6463 61696 9374 120 20206 0 nova-api Sep 15 00:22:02 localhost kernel: [75498.073628] [ 6484] 2048681 6484 2493 146 10 116 0 bash Sep 15 00:22:02 localhost kernel: [75498.073629] [ 6496] 2048681 6496 85703 4783 170 58993 0 python Sep 15 00:22:02 localhost kernel: [75498.073630] [ 6590] 2048681 6590 2494 200 10 79 0 bash Sep 15 00:22:02 localhost kernel: [75498.073632] [ 6617] 2048681 6617 2493 146 10 117 0 bash Sep 15 00:22:02 localhost kernel: [75498.073633] [ 6645] 2048681 6645 53848 21811 107 1890 0 nova-cert Sep 15 00:22:02 localhost kernel: [75498.073634] [ 6652] 2048681 6652 2493 146 10 116 0 bash Sep 15 00:22:02 localhost kernel: [75498.073636] [ 6681] 2048681 6681 52153 20067 106 1964 0 nova-network Sep 15 00:22:02 localhost kernel: [75498.073637] [ 6689] 2048681 6689 2494 219 11 61 0 bash Sep 15 00:22:02 localhost kernel: [75498.073638] [ 6731] 2048681 6731 2493 146 11 117 0 bash Sep 15 00:22:02 localhost kernel: [75498.073639] [ 6751] 2048681 6751 37345 1597 77 7420 0 nova-novncproxy Sep 15 00:22:02 localhost kernel: [75498.073641] [ 6766] 2048681 6766 2493 155 11 106 0 bash Sep 15 00:22:02 localhost kernel: [75498.073642] [ 6788] 2048681 6788 53708 22228 109 1409 0 nova-consoleaut Sep 15 00:22:02 localhost kernel: [75498.073643] [ 6810] 2048681 6810 2570 300 9 57 0 bash Sep 15 00:22:02 localhost kernel: [75498.073645] [ 6850] 0 6850 16188 153 31 128 -1000 tgtd Sep 15 00:22:02 localhost kernel: [75498.073646] [ 6852] 0 6852 3281 67 11 35 0 tgtd Sep 15 00:22:02 localhost kernel: [75498.073647] [ 6870] 2048681 6870 2493 146 10 116 0 bash Sep 15 00:22:02 localhost kernel: [75498.073649] [ 6887] 2048681 6887 44298 6536 90 9177 0 cinder-api Sep 15 00:22:02 localhost kernel: [75498.073650] [ 6900] 2048681 6900 49246 19920 96 695 0 cinder-api Sep 15 00:22:02 localhost kernel: [75498.073651] [ 6901] 2048681 6901 47991 17359 94 1916 0 cinder-api Sep 15 00:22:02 localhost kernel: [75498.073652] [ 6914] 2048681 6914 2493 146 10 114 0 bash Sep 15 00:22:02 localhost kernel: [75498.073654] [ 6926] 2048681 6926 43810 14341 89 966 0 cinder-schedule Sep 15 00:22:02 localhost kernel: [75498.073655] [ 6938] 2048681 6938 2493 154 10 107 0 bash Sep 15 00:22:02 localhost kernel: [75498.073656] [ 6959] 2048681 6959 91609 26445 178 36615 0 cinder-volume Sep 15 00:22:02 localhost kernel: [75498.073658] [11669] 2048681 11669 54280 23774 109 378 0 nova-scheduler Sep 15 00:22:02 localhost kernel: [75498.073659] [11679] 2048681 11679 51495 18693 103 2631 0 nova-conductor Sep 15 00:22:02 localhost kernel: [75498.073661] [11688] 2048681 11688 56988 26640 110 205 0 nova-conductor Sep 15 00:22:02 localhost kernel: [75498.073662] [11689] 2048681 11689 57080 26720 110 232 0 nova-conductor Sep 15 00:22:02 localhost kernel: [75498.073663] [11690] 2048681 11690 57187 26824 110 217 0 nova-conductor Sep 15 00:22:02 localhost kernel: [75498.073665] [11691] 2048681 11691 57947 27503 111 225 0 nova-conductor Sep 15 00:22:02 localhost kernel: [75498.073666] [15025] 2048681 15025 195345 30929 158 100 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073667] [15026] 2048681 15026 195409 31173 159 72 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073669] [15027] 2048681 15027 194511 29736 156 745 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073670] [15028] 2048681 15028 84303 14321 106 654 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073671] [15029] 2048681 15029 79605 13973 96 84 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073673] [15030] 2048681 15030 79605 13708 96 292 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073674] [15031] 2048681 15031 84239 14090 106 841 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073675] [15032] 2048681 15032 84303 14942 106 159 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073676] [15033] 2048681 15033 84623 15178 107 180 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073678] [15034] 2048681 15034 84495 15138 107 136 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073679] [15035] 2048681 15035 84623 15300 107 66 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073680] [15036] 2048681 15036 84559 15227 107 119 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073682] [15037] 2048681 15037 84559 14608 107 731 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073683] [15038] 33 15038 501914 2736 97 139 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073684] [15039] 33 15039 501870 2751 98 81 0 apache2 Sep 15 00:22:02 localhost kernel: [75498.073686] [27088] 2048681 27088 93555 9558 185 60262 0 glance-api Sep 15 00:22:02 localhost kernel: [75498.073687] [28480] 0 28480 36615 546 74 0 0 sshd Sep 15 00:22:02 localhost kernel: [75498.073688] [28530] 2048681 28530 36718 508 72 0 0 sshd Sep 15 00:22:02 localhost kernel: [75498.073689] [28531] 2048681 28531 3865 860 12 121 0 bash Sep 15 00:22:02 localhost kernel: [75498.073691] [28855] 0 28855 36615 588 74 0 0 sshd Sep 15 00:22:02 localhost kernel: [75498.073692] [28899] 2048681 28899 36680 475 71 0 0 sshd Sep 15 00:22:02 localhost kernel: [75498.073693] [28900] 2048681 28900 4044 1160 12 0 0 bash Sep 15 00:22:02 localhost kernel: [75498.073695] [30389] 2048681 30389 101598 77154 201 2427 0 python Sep 15 00:22:02 localhost kernel: [75498.073696] [30793] 2048681 30793 100979 74361 199 2807 0 glance-api Sep 15 00:22:02 localhost kernel: [75498.073697] [31082] 2048681 31082 90927 18324 179 50564 0 python Sep 15 00:22:02 localhost kernel: [75498.073699] [31876] 2048681 31876 7309 198 19 0 0 screen Sep 15 00:22:02 localhost kernel: [75498.073700] [32340] 0 32340 30089 392 60 0 0 sudo Sep 15 00:22:02 localhost kernel: [75498.073701] [32341] 0 32341 1808 110 10 0 0 tail Sep 15 00:22:02 localhost kernel: [75498.073703] [32356] 2048681 32356 107120 76374 207 2120 0 cinder-volume Sep 15 00:22:02 localhost kernel: [75498.073704] Out of memory: Kill process 30389 (python) score 62 or sacrifice child Sep 15 00:22:02 localhost kernel: [75498.074535] Killed process 30389 (python) total-vm:406392kB, anon-rss:308300kB, file-rss:316kB