oneiric LAMP server: Apache consumes 100% of cpu in QEMU

Bug #952199 reported by macada on 2012-03-11
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
apache2 (Ubuntu)
Undecided
Unassigned

Bug Description

I'm running ubuntu server 11.10 in a kvm environmnet as a production server for php applications.
How ever, my apache installation is very standard, except for virtual hosts of course.

These are apache2.conf vbles:

Timeout 2
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 2

<IfModule mpm_prefork_module>
    StartServers 5
    MinSpareServers 5
    MaxSpareServers 15
    ServerLimit 256
    MaxClients 256
    #50
    MaxRequestsPerChild 1000
</IfModule>

When Internet http clients are sending requests every single apache2 process almost immediately consumes 100% cpu until the request is done and falls back to < 1% then. The access logs don't show anything except my requests. Error log is empty.

$ sudo lsb_release -rd
Description: Ubuntu 11.10
Release: 11.10

$ sudo apt-cache policy apache2
apache2:
  Installed: 2.2.20-1ubuntu1.2
  Candidate: 2.2.20-1ubuntu1.2
  Version table:
 *** 2.2.20-1ubuntu1.2 0
        500 http://archive.ubuntu.com/ubuntu/ oneiric-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ oneiric-security/main amd64 Packages
        100 /var/lib/dpkg/status
     2.2.20-1ubuntu1 0
        500 http://archive.ubuntu.com/ubuntu/ oneiric/main amd64 Packages

$ sudo apt-cache policy libapache2-mod-php5
libapache2-mod-php5:
  Installed: 5.3.10-1~oneiric+2
  Candidate: 5.3.10-1~oneiric+2
  Version table:
 *** 5.3.10-1~oneiric+2 0
        100 /var/lib/dpkg/status
     5.3.6-13ubuntu3.6 0
        500 http://archive.ubuntu.com/ubuntu/ oneiric-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ oneiric-security/main amd64 Packages
     5.3.6-13ubuntu3.1 0
        500 http://archive.ubuntu.com/ubuntu/ oneiric/main amd64 Packages

$ ls -la /etc/apache2/conf.d/
total 24
drwxr-xr-x 2 root root 4096 2012-03-11 13:13 .
drwxr-xr-x 7 root root 4096 2012-03-11 13:10 ..
-rw-r--r-- 1 root root 269 2011-09-06 20:38 charset
-rw-r--r-- 1 root root 3296 2011-09-06 20:38 localized-error-pages
-rw-r--r-- 1 root root 143 2011-09-06 20:38 other-vhosts-access-log
-rw-r--r-- 1 root root 1424 2011-09-06 20:38 security

$ ls -la /etc/apache2/mods-enabled/
total 8
drwxr-xr-x 2 root root 4096 2012-01-15 22:27 .
drwxr-xr-x 7 root root 4096 2012-03-11 13:10 ..
lrwxrwxrwx 1 root root 28 2012-01-15 18:57 alias.conf -> ../mods-available/alias.conf
lrwxrwxrwx 1 root root 28 2012-01-15 18:57 alias.load -> ../mods-available/alias.load
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 auth_basic.load -> ../mods-available/auth_basic.load
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 authn_file.load -> ../mods-available/authn_file.load
lrwxrwxrwx 1 root root 36 2012-01-15 18:57 authz_default.load -> ../mods-available/authz_default.load
lrwxrwxrwx 1 root root 38 2012-01-15 18:57 authz_groupfile.load -> ../mods-available/authz_groupfile.load
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 authz_host.load -> ../mods-available/authz_host.load
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 authz_user.load -> ../mods-available/authz_user.load
lrwxrwxrwx 1 root root 32 2012-01-15 18:57 autoindex.conf -> ../mods-available/autoindex.conf
lrwxrwxrwx 1 root root 32 2012-01-15 18:57 autoindex.load -> ../mods-available/autoindex.load
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 cgi.load -> ../mods-available/cgi.load
lrwxrwxrwx 1 root root 30 2012-01-15 18:57 deflate.conf -> ../mods-available/deflate.conf
lrwxrwxrwx 1 root root 30 2012-01-15 18:57 deflate.load -> ../mods-available/deflate.load
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 dir.conf -> ../mods-available/dir.conf
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 dir.load -> ../mods-available/dir.load
lrwxrwxrwx 1 root root 26 2012-01-15 18:57 env.load -> ../mods-available/env.load
lrwxrwxrwx 1 root root 30 2012-01-15 21:06 headers.load -> ../mods-available/headers.load
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 mime.conf -> ../mods-available/mime.conf
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 mime.load -> ../mods-available/mime.load
lrwxrwxrwx 1 root root 34 2012-01-15 18:57 negotiation.conf -> ../mods-available/negotiation.conf
lrwxrwxrwx 1 root root 34 2012-01-15 18:57 negotiation.load -> ../mods-available/negotiation.load
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 php5.conf -> ../mods-available/php5.conf
lrwxrwxrwx 1 root root 27 2012-01-15 18:57 php5.load -> ../mods-available/php5.load
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 reqtimeout.conf -> ../mods-available/reqtimeout.conf
lrwxrwxrwx 1 root root 33 2012-01-15 18:57 reqtimeout.load -> ../mods-available/reqtimeout.load
lrwxrwxrwx 1 root root 30 2012-01-15 22:27 rewrite.load -> ../mods-available/rewrite.load
lrwxrwxrwx 1 root root 31 2012-01-15 18:57 setenvif.conf -> ../mods-available/setenvif.conf
lrwxrwxrwx 1 root root 31 2012-01-15 18:57 setenvif.load -> ../mods-available/setenvif.load
lrwxrwxrwx 1 root root 26 2012-01-15 21:02 ssl.conf -> ../mods-available/ssl.conf
lrwxrwxrwx 1 root root 26 2012-01-15 21:02 ssl.load -> ../mods-available/ssl.load
lrwxrwxrwx 1 root root 29 2012-01-15 18:57 status.conf -> ../mods-available/status.conf
lrwxrwxrwx 1 root root 29 2012-01-15 18:57 status.load -> ../mods-available/status.load

Virtualhost conf: (hiding just IP, domainname, directories)
<VirtualHost 111.111.111.111:80>
        ServerName www.domain.com
        DocumentRoot /directory/directory/directory/directory/

        ErrorLog /directory/directory/directory/tmp/logs/error.log
        CustomLog /directory/directory/directory/tmp/logs/access.log combined
        Loglevel warn
        DirectoryIndex index.html index.htm index.php index.php4 index.php5
        ServerSignature Off

        <Directory /directory/directory/directory/directory/>
                Options -Indexes IncludesNOEXEC FollowSymLinks
                Allow from all
                AllowOverride All
        </Directory>

</VirtualHost>

$ sudo uname -a
Linux xxxxxxxxxxxxx 3.0.0-16-server #29-Ubuntu SMP Tue Feb 14 13:08:12 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

$ sudo free
             total used free shared buffers cached
Mem: 8193376 3243356 4950020 0 503084 1733488
-/+ buffers/cache: 1006784 7186592
Swap: 4928508 9912 4918596

$ more /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 2
model name : QEMU Virtual CPU version 0.14.1
stepping : 3
cpu MHz : 2793.574
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 4
wp : yes
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl pni cx16 popcnt hypervisor lahf_lm
bogomips : 5587.14
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apache2 (Ubuntu):
status: New → Confirmed
macada (macadanet) wrote :
Download full text (4.5 KiB)

More information to understand this bug:

$ sudo lsof|grep apache2|awk '{print $7" "$9}'|sort -n -r|uniq|egrep -v "(log)"|grep linux|less

2943032 /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2.0.8
1749000 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
1685816 /lib/x86_64-linux-gnu/libc-2.13.so
1518928 /usr/lib/x86_64-linux-gnu/libdb-5.1.so
1277312 /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0
1003848 /lib/x86_64-linux-gnu/libglib-2.0.so.0.3000.0
824024 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
688328 /usr/lib/x86_64-linux-gnu/libgnutls.so.26.16.14
620776 /usr/lib/x86_64-linux-gnu/libfreetype.so.6.6.2
538928 /lib/x86_64-linux-gnu/libm-2.13.so
516056 /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
373224 /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
332400 /lib/x86_64-linux-gnu/libssl.so.1.0.0
310816 /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.7.0
252592 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
243800 /lib/x86_64-linux-gnu/libpcre.so.3.12.1
220896 /usr/lib/x86_64-linux-gnu/libfontconfig.so.1.4.4
215520 /usr/lib/x86_64-linux-gnu/liblcms.so.1.0.19
207312 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.5
178424 /usr/lib/x86_64-linux-gnu/libvorbis.so.0.4.5
169992 /lib/x86_64-linux-gnu/libexpat.so.1.5.2
162776 /lib/x86_64-linux-gnu/libpng12.so.0.46.0
162248 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
150144 /usr/lib/x86_64-linux-gnu/libjpeg.so.62.0.0
141088 /lib/x86_64-linux-gnu/ld-2.13.so
135500 /lib/x86_64-linux-gnu/libpthread-2.13.so
112992 /usr/lib/x86_64-linux-gnu/libxcb.so.1.1.0
109264 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.24
101552 /usr/lib/x86_64-linux-gnu/librtmp.so.0
101192 /lib/x86_64-linux-gnu/libresolv-2.13.so
100472 /usr/lib/x86_64-linux-gnu/libspeex.so.1.5.0
97256 /lib/x86_64-linux-gnu/libnsl-2.13.so
96816 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
88384 /lib/x86_64-linux-gnu/libgcc_s.so.1
77696 /usr/lib/x86_64-linux-gnu/libXext.so.6.4.0
67856 /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.11
55888 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
55504 /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.7.0
51736 /lib/x86_64-linux-gnu/libnss_files-2.13.so
47696 /lib/x86_64-linux-gnu/libnss_nis-2.13.so
43296 /lib/x86_64-linux-gnu/libcrypt-2.13.so
35712 /lib/x86_64-linux-gnu/libnss_compat-2.13.so
31752 /lib/x86_64-linux-gnu/librt-2.13.so
31184 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
27032 /lib/x86_64-linux-gnu/libnss_dns-2.13.so
26776 /usr/lib/x86_64-linux-gnu/libogg.so.0.7.1
26066 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
22496 /usr/lib/x86_64-linux-gnu/libXdmcp.so.6.0.0
18888 /lib/x86_64-linux-gnu/libuuid.so.1.3.0
14768 /lib/x86_64-linux-gnu/libdl-2.13.so
14544 /lib/x86_64-linux-gnu/libcom_err.so.2.1
14472 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.3000.0

$ sudo lsof|grep apache2|awk '{print $7" "$9}'|sort -n -r|uniq|egrep -v "(log|linux)"|less

8340728 /usr/lib/apache2/modules/libphp5.so
6172480 /usr/lib/libavcodec.so.53.6.0
2866608 /usr/lib/locale/locale-archive
2348016 /usr/lib/libMagickCore.so.3.0.0
2140240 /usr/lib/libmysqlclient_r.so.16.0.0
1516546 /usr/lib/apache2/mpm-prefork/apache2
1417304 /usr/lib/libxml2.so.2.7.8
1101136 /usr/lib/libMagickWand.so.3.0.0
983328 /usr/lib/libavformat.so.53.3.0
676952 /usr/lib/libschroedinger-1.0.so.0.10.0
617456...

Read more...

macada (macadanet) wrote :
Download full text (7.4 KiB)

Informatión from libphp5:

$ sudo php -i
................
................
................
................
PHP API => 20090626
PHP Extension => 20090626
Zend Extension => 220090626
Zend Extension Build => API220090626,NTS
PHP Extension Build => API20090626,NTS
Debug Build => no
Thread Safety => disabled
Zend Memory Manager => enabled
Zend Multibyte Support => disabled
IPv6 Support => enabled
Registered PHP Streams => https, ftps, compress.zlib, compress.bzip2, php, file, glob, data, http, ftp, zip, phar
Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, tls
Registered Stream Filters => zlib.*, bzip2.*, convert.iconv.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, mcrypt.*, mdecrypt.*
................
................
................
................

apc

APC Support => disabled
Version => 3.1.7
APC Debugging => Disabled
MMAP Support => Enabled
MMAP File Mask =>
Locking type => pthread mutex Locks
Serialization Support => broken
Revision => $Revision: 307215 $
Build Date => May 2 2011 19:00:42

Directive => Local Value => Master Value
apc.cache_by_default => On => On
apc.canonicalize => On => On
apc.coredump_unmap => Off => Off
apc.enable_cli => Off => Off
apc.enabled => On => On
apc.file_md5 => Off => Off
apc.file_update_protection => 2 => 2
apc.filters => no value => no value
apc.gc_ttl => 3600 => 3600
apc.include_once_override => Off => Off
apc.lazy_classes => Off => Off
apc.lazy_functions => Off => Off
apc.max_file_size => 1M => 1M
apc.mmap_file_mask => no value => no value
apc.num_files_hint => 1000 => 1000
apc.preload_path => no value => no value
apc.report_autofilter => Off => Off
apc.rfc1867 => Off => Off
apc.rfc1867_freq => 0 => 0
apc.rfc1867_name => APC_UPLOAD_PROGRESS => APC_UPLOAD_PROGRESS
apc.rfc1867_prefix => upload_ => upload_
apc.rfc1867_ttl => 3600 => 3600
apc.serializer => default => default
apc.shm_segments => 1 => 1
apc.shm_size => 128 => 128
apc.slam_defense => On => On
apc.stat => On => On
apc.stat_ctime => Off => Off
apc.ttl => 0 => 0
apc.use_request_time => On => On
apc.user_entries_hint => 4096 => 4096
apc.user_ttl => 0 => 0
apc.write_lock => On => On
................
................
................
................

Core

PHP Version => 5.3.10-1~oneiric+2

Directive => Local Value => Master Value
allow_call_time_pass_reference => Off => Off
allow_url_fopen => On => On
allow_url_include => Off => Off
always_populate_raw_post_data => Off => Off
arg_separator.input => & => &
arg_separator.output => & => &
asp_tags => Off => Off
auto_append_file => no value => no value
auto_globals_jit => On => On
auto_prepend_file => no value => no value
browscap => no value => no value
default_charset => no value => no value
default_mimetype => text/html => text/html
define_syslog_variables => Off => Off
disable_classes => no value => no value
disable_functions => no value => no value
display_errors => Off => Off
display_startup_errors => Off => Off
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => Off => Off
error_append_string => no value => no value
error_log => no value => no value
err...

Read more...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers