oneiric LAMP server: Apache consumes 100% of cpu in QEMU
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| apache2 (Ubuntu) |
Undecided
|
Unassigned |
Bug Description
I'm running ubuntu server 11.10 in a kvm environmnet as a production server for php applications.
How ever, my apache installation is very standard, except for virtual hosts of course.
These are apache2.conf vbles:
Timeout 2
KeepAlive On
MaxKeepAliveReq
KeepAliveTimeout 2
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 15
ServerLimit 256
MaxClients 256
#50
MaxRequests
</IfModule>
When Internet http clients are sending requests every single apache2 process almost immediately consumes 100% cpu until the request is done and falls back to < 1% then. The access logs don't show anything except my requests. Error log is empty.
$ sudo lsb_release -rd
Description: Ubuntu 11.10
Release: 11.10
$ sudo apt-cache policy apache2
apache2:
Installed: 2.2.20-1ubuntu1.2
Candidate: 2.2.20-1ubuntu1.2
Version table:
*** 2.2.20-1ubuntu1.2 0
500 http://
500 http://
100 /var/lib/
2.
500 http://
$ sudo apt-cache policy libapache2-mod-php5
libapache2-
Installed: 5.3.10-1~oneiric+2
Candidate: 5.3.10-1~oneiric+2
Version table:
*** 5.3.10-1~oneiric+2 0
100 /var/lib/
5.
500 http://
500 http://
5.
500 http://
$ ls -la /etc/apache2/
total 24
drwxr-xr-x 2 root root 4096 2012-03-11 13:13 .
drwxr-xr-x 7 root root 4096 2012-03-11 13:10 ..
-rw-r--r-- 1 root root 269 2011-09-06 20:38 charset
-rw-r--r-- 1 root root 3296 2011-09-06 20:38 localized-
-rw-r--r-- 1 root root 143 2011-09-06 20:38 other-vhosts-
-rw-r--r-- 1 root root 1424 2011-09-06 20:38 security
$ ls -la /etc/apache2/
total 8
drwxr-xr-x 2 root root 4096 2012-01-15 22:27 .
drwxr-xr-x 7 root root 4096 2012-03-11 13:10 ..
lrwxrwxrwx 1 root root 28 2012-01-15 18:57 alias.conf -> ../mods-
lrwxrwxrwx 1 root root 28 2012-01-15 18:57 alias.load -> ../mods-
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 auth_basic.load -> ../mods-
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 authn_file.load -> ../mods-
lrwxrwxrwx 1 root root 36 2012-01-15 18:57 authz_default.load -> ../mods-
lrwxrwxrwx 1 root root 38 2012-01-15 18:57 authz_groupfile
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 authz_host.load -> ../mods-
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 authz_user.load -> ../mods-
lrwxrwxrwx 1 root root 32 2012-01-15 18:57 autoindex.conf -> ../mods-
lrwxrwxrwx 1 root root 32 2012-01-15 18:57 autoindex.load -> ../mods-
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 cgi.load -> ../mods-
lrwxrwxrwx 1 root root 30 2012-01-15 18:57 deflate.conf -> ../mods-
lrwxrwxrwx 1 root root 30 2012-01-15 18:57 deflate.load -> ../mods-
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 dir.conf -> ../mods-
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 dir.load -> ../mods-
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 env.load -> ../mods-
lrwxrwxrwx 1 root root 30 2012-01-15 21:06 headers.load -> ../mods-
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 mime.conf -> ../mods-
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 mime.load -> ../mods-
lrwxrwxrwx 1 root root 34 2012-01-15 18:57 negotiation.conf -> ../mods-
lrwxrwxrwx 1 root root 34 2012-01-15 18:57 negotiation.load -> ../mods-
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 php5.conf -> ../mods-
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 php5.load -> ../mods-
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 reqtimeout.conf -> ../mods-
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 reqtimeout.load -> ../mods-
lrwxrwxrwx 1 root root 30 2012-01-15 22:27 rewrite.load -> ../mods-
lrwxrwxrwx 1 root root 31 2012-01-15 18:57 setenvif.conf -> ../mods-
lrwxrwxrwx 1 root root 31 2012-01-15 18:57 setenvif.load -> ../mods-
lrwxrwxrwx 1 root root 26 2012-01-15 21:02 ssl.conf -> ../mods-
lrwxrwxrwx 1 root root 26 2012-01-15 21:02 ssl.load -> ../mods-
lrwxrwxrwx 1 root root 29 2012-01-15 18:57 status.conf -> ../mods-
lrwxrwxrwx 1 root root 29 2012-01-15 18:57 status.load -> ../mods-
Virtualhost conf: (hiding just IP, domainname, directories)
<VirtualHost 111.111.111.111:80>
ServerName www.domain.com
ErrorLog /directory/
CustomLog /directory/
Loglevel warn
<Directory /directory/
</VirtualHost>
$ sudo uname -a
Linux xxxxxxxxxxxxx 3.0.0-16-server #29-Ubuntu SMP Tue Feb 14 13:08:12 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
$ sudo free
total used free shared buffers cached
Mem: 8193376 3243356 4950020 0 503084 1733488
-/+ buffers/cache: 1006784 7186592
Swap: 4928508 9912 4918596
$ more /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
macada (macadanet) wrote : | #2 |
More information to understand this bug:
$ sudo lsof|grep apache2|awk '{print $7" "$9}'|sort -n -r|uniq|egrep -v "(log)"|grep linux|less
2943032 /usr/lib/
1749000 /lib/x86_
1685816 /lib/x86_
1518928 /usr/lib/
1277312 /usr/lib/
1003848 /lib/x86_
824024 /usr/lib/
688328 /usr/lib/
620776 /usr/lib/
538928 /lib/x86_
516056 /lib/x86_
373224 /usr/lib/
332400 /lib/x86_
310816 /usr/lib/
252592 /usr/lib/
243800 /lib/x86_
220896 /usr/lib/
215520 /usr/lib/
207312 /usr/lib/
178424 /usr/lib/
169992 /lib/x86_
162776 /lib/x86_
162248 /usr/lib/
150144 /usr/lib/
141088 /lib/x86_
135500 /lib/x86_
112992 /usr/lib/
109264 /usr/lib/
101552 /usr/lib/
101192 /lib/x86_
100472 /usr/lib/
97256 /lib/x86_
96816 /lib/x86_
88384 /lib/x86_
77696 /usr/lib/
67856 /usr/lib/
55888 /usr/lib/
55504 /usr/lib/
51736 /lib/x86_
47696 /lib/x86_
43296 /lib/x86_
35712 /lib/x86_
31752 /lib/x86_
31184 /usr/lib/
27032 /lib/x86_
26776 /usr/lib/
26066 /usr/lib/
22496 /usr/lib/
18888 /lib/x86_
14768 /lib/x86_
14544 /lib/x86_
14472 /usr/lib/
$ sudo lsof|grep apache2|awk '{print $7" "$9}'|sort -n -r|uniq|egrep -v "(log|linux)"|less
8340728 /usr/lib/
6172480 /usr/lib/
2866608 /usr/lib/
2348016 /usr/lib/
2140240 /usr/lib/
1516546 /usr/lib/
1417304 /usr/lib/
1101136 /usr/lib/
983328 /usr/lib/
676952 /usr/lib/
617456...
macada (macadanet) wrote : | #3 |
Informatión from libphp5:
$ sudo php -i
................
................
................
................
PHP API => 20090626
PHP Extension => 20090626
Zend Extension => 220090626
Zend Extension Build => API220090626,NTS
PHP Extension Build => API20090626,NTS
Debug Build => no
Thread Safety => disabled
Zend Memory Manager => enabled
Zend Multibyte Support => disabled
IPv6 Support => enabled
Registered PHP Streams => https, ftps, compress.zlib, compress.bzip2, php, file, glob, data, http, ftp, zip, phar
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, tls
Registered Stream Filters => zlib.*, bzip2.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, mcrypt.*, mdecrypt.*
................
................
................
................
apc
APC Support => disabled
Version => 3.1.7
APC Debugging => Disabled
MMAP Support => Enabled
MMAP File Mask =>
Locking type => pthread mutex Locks
Serialization Support => broken
Revision => $Revision: 307215 $
Build Date => May 2 2011 19:00:42
Directive => Local Value => Master Value
apc.cache_
apc.canonicalize => On => On
apc.coredump_unmap => Off => Off
apc.enable_cli => Off => Off
apc.enabled => On => On
apc.file_md5 => Off => Off
apc.file_
apc.filters => no value => no value
apc.gc_ttl => 3600 => 3600
apc.include_
apc.lazy_classes => Off => Off
apc.lazy_functions => Off => Off
apc.max_file_size => 1M => 1M
apc.mmap_file_mask => no value => no value
apc.num_files_hint => 1000 => 1000
apc.preload_path => no value => no value
apc.report_
apc.rfc1867 => Off => Off
apc.rfc1867_freq => 0 => 0
apc.rfc1867_name => APC_UPLOAD_PROGRESS => APC_UPLOAD_PROGRESS
apc.rfc1867_prefix => upload_ => upload_
apc.rfc1867_ttl => 3600 => 3600
apc.serializer => default => default
apc.shm_segments => 1 => 1
apc.shm_size => 128 => 128
apc.slam_defense => On => On
apc.stat => On => On
apc.stat_ctime => Off => Off
apc.ttl => 0 => 0
apc.use_
apc.user_
apc.user_ttl => 0 => 0
apc.write_lock => On => On
................
................
................
................
Core
PHP Version => 5.3.10-1~oneiric+2
Directive => Local Value => Master Value
allow_call_
allow_url_fopen => On => On
allow_url_include => Off => Off
always_
arg_separator.input => & => &
arg_separator.
asp_tags => Off => Off
auto_append_file => no value => no value
auto_globals_jit => On => On
auto_prepend_file => no value => no value
browscap => no value => no value
default_charset => no value => no value
default_mimetype => text/html => text/html
define_
disable_classes => no value => no value
disable_functions => no value => no value
display_errors => Off => Off
display_
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => Off => Off
error_append_string => no value => no value
error_log => no value => no value
err...
Status changed to 'Confirmed' because the bug affects multiple users.