Comment 9 for bug 234927

Revision history for this message
steph (acidcortex) wrote : Re: delay php script execution on cli

This is an addition this bug.
My webserver is acting normal in the first 24 hours (after start/restart) then becomes very sluggish.
If I try to open 30-40 urls in 30-40 seconds, they all take 100% cpu and lots of memory, and then hang.
At first they eat 100% CPU, then they eat-up all the memory and swap space, and then change to wait state (hang).

Test case:
I forked around 50 apache2/php5s in 60 seconds, and left them for half an hour.
The application is var_dump() of a large object containing few inherited classes, and few objects;
In normal conditions, this causes load average < 0.5.

here is a Top copy with around 50 active apache2/php5 forks after 10 minutes:
top - 10:46:15 up 1 day, 42 min, 2 users, load average: 60.40, 59.84, 40.41
Tasks: 143 total, 4 running, 139 sleeping, 0 stopped, 0 zombie
Cpu0 : 6.5%us, 14.3%sy, 0.0%ni, 5.1%id, 72.8%wa, 0.1%hi, 1.2%si, 0.0%st
Mem: 1018856k total, 997360k used, 21496k free, 14252k buffers
Swap: 1959912k total, 1959364k used, 548k free, 16936k cached

Didn't catch ps tho to take exact count of forks.
When I restart apache or machine everything works fine for approx 1 day, then starts bugging all over again.
When I disable php5-clamavlib everything works fine and stays fine.
I have also this issue with php-cli and 99% load, and it also works fine when php5-clamavlib is disabled.

I presumed that this has something to do with clamav-freshclam daemon, and tried to repeat this problem by doing $freshclam update manually, but nothing happened.
My biggest concern is that kernel doesn't get ENOMEM error or ignores it or something very unusual happens since dmesg, /var/log/messages and /var/log/apache2/error.log are all clean, and kernel isn't killing any forks. They just deadlock and stay like that forever (30 minutes, and none of them was killed so I presumed they will stay in wait state forever)

hardware:
Sempron64 2800+, 1Gig RAM, 2Gig swap, 2x160Gig RAID1 hdd
distro:
ubuntu server hardy
kernel:
Linux srv1.intranea 2.6.24-19-server #1 SMP Wed Jun 4 17:16:58 UTC 2008 i686 GNU/Linux
apache:
Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.1 with Suhosin-Patch mod_ssl/2.2.8 OpenSSL/0.9.8g
apache modules:
core mod_log_config mod_logio prefork http_core mod_so mod_alias mod_auth_basic mod_authn_file mod_authz_default mod_authz_groupfile mod_authz_host mod_authz_user mod_autoindex mod_cgi mod_dir mod_env mod_mime mod_negotiation mod_php5 mod_rewrite mod_setenvif mod_ssl mod_status mod_userdir
php get_loaded_extensions says:
zip xmlwriter libxml xml wddx tokenizer sysvshm sysvsem sysvmsg session SimpleXML sockets soap SPL shmop standard Reflection posix mime_magic mbstring json iconv hash gettext ftp filter exif dom dba date ctype calendar bz2 bcmath zlib pcre openssl xmlreader apache2handler ADOdb clamav curl ffmpeg gd ldap mcrypt mhash mysql mysqli odbc PDO pdo_mysql PDO_ODBC pdo_pgsql pdo_sqlite pgsql SQLite xmlrpc