php-fpm should be reloaded on upgrade

[Mark McKinstry]

If you're using php-fpm, you need to reload php-fpm when php is updated, otherwise all the members of the php-fpm pool will continue to run on the older version of PHP. This can be a quite old version depending on when the last time you did a php-fpm reload.

You can check this by upgrading a server and checking the phpinfo of a site, it will still show the old version. If you do a 'lsofp -p $PID | grep deleted' on the php-fpm process, you'll see it is running off the older deleted php-fpm binary. When you do a reload of php-fpm, it will fix it.

Fedora now does a reload of php-fpm on upgrade: http://pkgs.fedoraproject.org/cgit/php.git/plain/php.spec

[booi (booi)]

I believe we should follow RHEL's convention on this IMHO. Operating a high volume website, we stage an update on many machines before reloading them simultaneously. This is especially true if we are adding/removing additional PECL modules or if there is a code change necessary. Not reloading automatically gives us the ability to do this.

I don't know of any daemons that are automatically reloaded or restarted on upgrade in RHEL actually.

[Mark McKinstry (mmckinst)]

Brandon,

It's actually pretty common for RHEL to include a condrestart or reload in the RPMs for services since you need to restart the service so it will use the new binary.

I checked a few random RPMs using `rpm -q --scripts some_rpm` and found the following include a condrestart:

* openssh-server
* rsyslog
* cronie
* ntp
* mysql-server
* mysql51-server from IUS
* mysql55-server from IUS

For the people who don't know they're supposed to manually reload php-fpm, they're potentially running a very old version of PHP on their websites which probably isn't what they intended when they started using the IUS repo.

A few notes about the change:

* I did a reload instead of a condrestart. reload sends a SIGUSR2 signal which does a 'graceful reload of all workers + reload of fpm conf/binary'. I did this instead of a condrestart since the condrestart does full stop followed by a start. A reload will not start the php-fpm service if it was previously turned off.

* When upgrading RPMs, the postun section is run by the old RPM being removed. So if this was released as php53u-5.3.20-2 , the reload won't happen until php53u-5.3.20-2 is upgraded by php53u-5.3.20-3 or php53u-5.3.21-1 . This is documented at http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Scriptlet_Ordering . All the RPMs and documentation I checked for Fedora and RHEL have the condrestart in the postun section, I couldn't find a reason why it is done this way but decided to keep it consistent.

[bharper (bharper)]

Hello Mark,

Thanks for taking time to contact us about this feature request and supplying spec files. Personally, I have some reservations regarding this change, but we must proceed based on whats best for the community.

IUS strives to follow Fedora's packaging guide. The packaging guide does cover the topic of restarting services:

https://fedoraproject.org/wiki/Packaging:SysVInitScript?rd=Packaging/SysVInitScript#InitscriptScriptlets

The guide out lines how to restart services within spec file, but also notes that it may not be appropriate for all services. So now we need to determine if it is appropriate for php-fpm to restart or reload when its updated. I agree with booi, that we should take into account RHEL's conventions. I use too many distributions to recall RHEL's conventions off the top of my head, so research is needed.

-Ben

[bharper (bharper)]

Mark,

Looks like we were updating this ticket at the same time. I will look into this further and let you know.

-Ben

[booi (booi)]

Hi Mark,

I looked at the scripts you mentioned and none of those actually do a
restart on an update, only uninstall (probably the right thing for
uninstalls). I just tested this by updating mysql55-server and no such
restart happened.

Brandon

On Thu, Jan 10, 2013 at 2:51 PM, bharper <email address hidden> wrote:

> Mark,
>
> Looks like we were updating this ticket at the same time. I will look
> into this further and let you know.
>
> -Ben
>
> --
> You received this bug notification because you are subscribed to IUS
> Community Project.
> https://bugs.launchpad.net/bugs/1098319
>
> Title:
> php-fpm should be reloaded on upgrade
>
> Status in IUS Community Project:
> New
>
> Bug description:
> If you're using php-fpm, you need to reload php-fpm when php is
> updated, otherwise all the members of the php-fpm pool will continue
> to run on the older version of PHP. This can be a quite old version
> depending on when the last time you did a php-fpm reload.
>
> You can check this by upgrading a server and checking the phpinfo of a
> site, it will still show the old version. If you do a 'lsofp -p $PID |
> grep deleted' on the php-fpm process, you'll see it is running off the
> older deleted php-fpm binary. When you do a reload of php-fpm, it will
> fix it.
>
> Fedora now does a reload of php-fpm on upgrade:
> http://pkgs.fedoraproject.org/cgit/php.git/plain/php.spec
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ius/+bug/1098319/+subscriptions
>

[Mark McKinstry (mmckinst)]

Brandon,

The %postun section is also run run when upgrading, it will be run by the older version of the RPM when it is removed which is what restarts the service. http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Scriptlet_Ordering talks about all the sections and what order they're run in when an RPM is upgraded from the old RPM to the new RPM. To further complicate things, there's a variable named $1 which is used to determine if the RPM is being installed, upgraded, or uninstalled. When you see the conditional testing if $1 is greater than or equal to 1, that is how the RPM knows it is being upgraded.

I'm not sure why mysql55 didn't restart for you. On el6, I installed 5.5.28-2 from the ius-archive repo, upgraded to 5.5.29-1, and mysql was restarted for me. Make sure you're actually doing an upgrade of a package (ie the RPM you're upgrading has a higher version number of one already installed), not running rpm -Uvh of an RPM already installed (ie don't run rpm -Uvh mysql-5.5.29-1 if you already have mysql-5-5.29-1 installed) since I don't think that will work. Also, check your yum.conf and see if you've set tsflags="noscripts" since that will disable to condrestart/reload on upgrade (as well as other things).

[bharper (bharper)]

Hey Mark,

I have been doing some research into this request. In my testing, I found that php-fpm reload is not very gracefully. Here is the output when php-fpm gets reloaded while a page is still loading:

$ curl testsite.local/sleep_10_echo_date.php
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.2.6</center>
</body>
</html>

I see that there is an open bug with PHP about this issue:

https://bugs.php.net/bug.php?id=60961

I also tested restart, force-reload, condrestart and try-restart, but they all gave the same result.

I think it might be a good idea not to make any changes till php-fpm can get gracefully restarted. When the time has come, we can revisit this request.

-Ben

[bharper (bharper)]

Our php53u packages went EOL some time ago. Looks like we forgot to
close this ticket when that happened.