Backport Apache fix "Honor ProxyTimeout" (54973)

[Robin Kluth]

As suggested by Manfred Hampl, I now open a bug report from this question:

I use Apache 2.4.7 with mod_proxy_fcgi which is affected of this bug: https://bz.apache.org/bugzilla/show_bug.cgi?id=54973 .

Is it possible to backport this patch to 2.4.7 (fixed in 2.4.8, see link @ bottom page)? Would be nice to use timeouts higher than 30 seconds ;)

My system is an up-to-date Ubuntu Server 14.04.3 LTS with Apache 2.4.7-1ubuntu4.8

Thanks in advance!

[Bruno-canet (bruno-canet)]

mod_proxy_fcgi is configured to proxy request to php-fpm as follow:
ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:9000/opt/httpd/htdocs/app/$1

If a php script takes more than 30 seconds to execute, a HTTP RC 500 is returned to the client.

php max_execution_time is set to 300 seconds

I used the following script to validate that the error was not due to php max_execution_time issue.

<?php
    error_log("executing script... ");
    $time = time();
    for ($t = 0; $t <= 15; $t++) {
        error_log("Logging: $t (".(time()-$time)." seconds)");
        sleep(5);
    }
    error_log("execution done (".(time()-$time)." seconds)");
?>
after 30 seconds, the HTTP 500 is returned to the client, but in the php error_log the script continue its execution.

According to the documentation, TimeOut is set to 60 seconds by default, so it should not be an issue, but in case, the following explicit definition were tested without improvement:

1. ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:9000/opt/httpd/htdocs/app/$1 timeout=300

2. in virtual host: ProxyTimeout 300

3 in server config: TimeOut 300

The only workaround found was to hardcode the timeout in the mod_proxy_fcgi.c:
--- ./modules/proxy/mod_proxy_fcgi.c.orig 2013-04-16 16:09:25.970332062 +0200
+++ ./modules/proxy/mod_proxy_fcgi.c 2013-04-16 16:09:56.311088966 +0200
@@ -575,7 +575,7 @@
         /* We need SOME kind of timeout here, or virtually anything will
          * cause timeout errors. */
         if (! conn->worker->s->timeout_set) {
- timeout = apr_time_from_sec(30);
+ timeout = apr_time_from_sec(300);
         }

         rv = apr_poll(&pfd, 1, &n, timeout);

[Apacheorg (apacheorg)]

we confirm this behavior with:
mod_proxy_fcgi
mod_proxy
apache event mpm
on latest stable apache 2.4.6

we are using:

(we tried both inside and outside virtualhost: )
<Proxy fcgi://socket=%2fdev%2fshm%2ffpm-php.sock>
  ProxySet timeout=3600
  ProxySet connectiontimeout=3600
</Proxy>

ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://socket=%2fdev%2fshm%2ffpm-php.sock/mobilnet.sk/$1 timeout=3600 connectiontimeout=3600

OR
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^/?(.*\.php)$ fcgi://socket=\%2fdev\%2fshm\%2ffpm-php.sock/path/$1 [P,L]

but none of them accepts the timeout:
[proxy_fcgi:error] (70007)The timeout specified has expired: [...] AH01075: Error dispatching request to:

[Jan Kaluza (jkaluza)]

Created attachment 30828
proposed patch

This patch removes hardcoded 30 second timeout from mod_proxy_fcgi and replaces it with ProxyTimeout/Timeout.

[Jan Kaluza (jkaluza)]

Created attachment 30829
proposed patch v2

Better patch respecting previous "conn->worker->s->timeout" timeout if set.

[Jan Kaluza (jkaluza)]

The fact that you were not able to set "timeout=300" is caused by PR 43513.

[Jan Kaluza (jkaluza)]

Created attachment 30833
proposed patch v3

Even better patch. Now uses apr_socket_timeout_get.

[Trawick-3 (trawick-3)]

Fixed in 2.4.8:

 *) mod_proxy_fcgi: Use apr_socket_timeout_get instead of hard-coded
     30 seconds timeout. [Jan Kaluza]

[Robie Basak (racb)]

Hi Robin,

Thank you for taking the time to report this bug and helping to make Ubuntu better.

This request looks reasonable to me. I'll add it to the queue. To help process it faster, we will need:

Identification of the upstream patch in upstream VCS (commit ID and preferably a link to the commit in a web view of the VCS).

Manual verification by a developer that the patch is still applicable to the version in 14.04.

A test case with step by step instructions on how to reproduce the issue on 14.04.

The full procedure is documented at: https://wiki.ubuntu.com/StableReleaseUpdates#Procedure

Thanks

[Robin Kluth (commifreak)]

I can help you to identify the upstream patch in upstream VCS for now ;)

The revision: http://svn.apache.org/viewvc?view=revision&amp;revision=1524368

The path: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/mod_proxy_fcgi.c?r1=1524368&r2=1524367&pathrev=1524368&view=patch

Regards,

[Robin Kluth (commifreak)]

Eh.. Did someone changed something?

I've tested a few tips to set the Timeout. Now it works as expected? In the same way as it not worked earlier. Weird.

For the log:

php5-fpm 5.5.9+dfsg-1ubuntu4.1
apache2 2.4.7-1ubuntu4.9

And the config:

<VirtualHost 192.168.205.69:80>
 ServerName site
 ServerAdmin mail
 DocumentRoot "/var/www/site/"
 LogLevel warn
 ErrorLog ${APACHE_LOG_DIR}/site.error.log
 CustomLog ${APACHE_LOG_DIR}/site.log combined
 ProxyTimeout 600

 <Directory "/var/www/site/">
  Options -Indexes

  Require all granted
  AllowOverride All
 </Directory>

 <LocationMatch "^/(.*\.php(/.*)?)$">
  ProxyPass fcgi://127.0.0.1:9091/var/www/site//$1
 </LocationMatch>

 DirectoryIndex index.html index.php
</VirtualHost>

FPM got same timeout (max_execution_timeout).

This is working. Dont ask me why. I didnt found any changelog entry for this.

[Christian Ehrhardt  (paelzer)]

Based on the examples here and the referred Apache bug I recreated this and confirms that it works.
There were some proxy fixes recently, maybe one of them affected this as well without knowing about it.

Steps to reproduce:
trusty container
apt-get install php5-fpm apache2 libapache2-mod-fcgid

config for the site is:
cat /etc/apache2/sites-available/test-fcgi-proxy-pass.conf
<VirtualHost 10.0.4.126:80>
  DocumentRoot /var/www/html
  ProxyTimeout 600

  <Directory "/var/www/html/">
   Options -Indexes
   Require all granted
   AllowOverride All
  </Directory>

  <LocationMatch "^/(.*\.php(/.*)?)$">
    ProxyPass fcgi://127.0.0.1:9000/var/www/html/$1
  </LocationMatch>
</VirtualHost>

enable site and enable proxy
   ln -s /etc/apache2/sites-available/test-fcgi-proxy-pass.conf /etc/apache2/sites-enabled/
   ln -s /etc/apache2/mods-available/proxy* /etc/apache2/mods-enabled/
   ln -s /etc/apache2/mods-available/xml2enc.load /etc/apache2/mods-enabled/
   ln -s /etc/apache2/mods-available/slotmem_shm.load /etc/apache2/mods-enabled/

Change the default local socket to match apache config in /etc/php5/fpm/pool.d/www.conf
listen = 9000

Now place a long duration script as /var/www/html/foo.php
<?php
    error_log("executing script... ");
    $time = time();
    for ($t = 0; $t <= 290; $t+=5) {
        error_log("Logging: $t (".(time()-$time)." seconds)");
        print("Logging: $t (".(time()-$time)." seconds)");
        sleep(5);
    }
    error_log("execution done (".(time()-$time)." seconds)");
?>

then access your servers apache to the php file, gets redirected and takes a while
http://10.0.4.126/foo.php

It would now be up to the initial reporter to check if it works for them as well.
Setting to incomplete until we hear again.

[Christian Ehrhardt  (paelzer)]

I can indeed confirm that the referred fix is in 2.4.7-1ubuntu4.11.
Looking back in versions when it got added I found it was already in 2.4.7-1ubuntu4.

If this needs something else to trigger or if it was meant to be a precise bug please let us know.