php7.0-fpm_7.0.33-0ubuntu0.16.04.2 update stop fpm service for 10 minutes
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
php7.0 (Ubuntu) |
Fix Released
|
Undecided
|
Karl Stenerud | ||
Xenial |
Won't Fix
|
Undecided
|
Unassigned |
Bug Description
[Impact]
Since php7.0 still follows the compat level 9 defaults of stopping the service at the start of an upgrade, and then starting it again at the end of the upgrade, there is potentially a long period of the service being stopped when there are many packages to upgrade, leading to unwanted downtime.
[Test Case]
A simple apt install --reinstall will exhibit the behavior in /var/log/syslog:
Command:
apt install --reinstall -y php7.0-fpm
/var/log/syslog:
Mar 12 13:40:17 tester systemd[1]: Reloading.
Mar 12 13:40:17 tester systemd[1]: Stopping The PHP 7.0 FastCGI Process Manager...
Mar 12 13:40:17 tester systemd[1]: Stopped The PHP 7.0 FastCGI Process Manager.
Mar 12 13:40:17 tester systemd[1]: Reloading.
Mar 12 13:40:18 tester systemd[1]: Reloading.
Mar 12 13:40:19 tester systemd[1]: Reloading.
Mar 12 13:40:19 tester systemd[1]: Reloading.
Mar 12 13:40:19 tester systemd[1]: Starting The PHP 7.0 FastCGI Process Manager...
Mar 12 13:40:19 tester systemd[1]: Started The PHP 7.0 FastCGI Process Manager.
php-fpm is down during the entire update process.
When upgrading to the fixed PPA, it will still shut down at the start due to the old script being run.
Command:
add-apt-repository -y ppa:kstenerud/
/var/log/syslog:
Mar 12 13:41:40 tester systemd[1]: Reloading.
Mar 12 13:41:41 tester systemd[1]: Stopping The PHP 7.0 FastCGI Process Manager...
Mar 12 13:41:41 tester systemd[1]: Stopped The PHP 7.0 FastCGI Process Manager.
Mar 12 13:41:41 tester systemd[1]: Reloading.
Mar 12 13:41:43 tester systemd[1]: Reloading.
Mar 12 13:41:52 tester systemd[1]: Reloading.
Mar 12 13:41:52 tester systemd[1]: Reloading.
Mar 12 13:41:52 tester systemd[1]: Stopped The PHP 7.0 FastCGI Process Manager.
Mar 12 13:41:52 tester systemd[1]: Starting The PHP 7.0 FastCGI Process Manager...
Mar 12 13:41:52 tester systemd[1]: Started The PHP 7.0 FastCGI Process Manager.
The extra "Stopped" at 13:41:52 is due to the new behavior calling restart.
With the new package installed, it no longer stops when beginning an upgrade, and does a restart at the end (which php-fpm accomplishes with a stop and start).
Command:
apt install --reinstall -y php7.0-fpm
/var/log/syslog:
Mar 12 13:43:00 tester systemd[1]: Reloading.
Mar 12 13:43:00 tester systemd[1]: Reloading.
Mar 12 13:43:01 tester systemd[1]: Reloading.
Mar 12 13:43:02 tester systemd[1]: Reloading.
Mar 12 13:43:02 tester systemd[1]: Stopping The PHP 7.0 FastCGI Process Manager...
Mar 12 13:43:02 tester systemd[1]: Stopped The PHP 7.0 FastCGI Process Manager.
Mar 12 13:43:02 tester systemd[1]: Starting The PHP 7.0 FastCGI Process Manager...
Mar 12 13:43:02 tester systemd[1]: Started The PHP 7.0 FastCGI Process Manager.
[Regression Potential]
If the service tries to load a file that has been replaced during upgrade, but before it was restarted, it could cause a failure. However, the service will soon be restarted anyway.
If the restart takes too long it could fail.
[Original Description]
The unattended update has launch the update of php7.0-fpm that trigger the stop of php7.0-fpm service (timestamp match that operation). The service does not restart within 10 minutes.
# systemctl status php7.0-fpm
● php7.0-fpm.service - The PHP 7.0 FastCGI Process Manager
Loaded: loaded (/lib/systemd/
Active: inactive (dead) since Thu 2019-03-07 15:21:00 UTC; 8min ago
Main PID: 3505 (code=exited, status=0/SUCCESS)
Status: "Processes active: 0, idle: 32, Requests: 0, slow: 0, Traffic: 0req/sec"
Mar 07 15:14:55 xxx systemd[1]: Starting The PHP 7.0 FastCGI Process Manager...
Mar 07 15:14:55 xxx systemd[1]: Started The PHP 7.0 FastCGI Process Manager.
Mar 07 15:21:00 xxx systemd[1]: Stopping The PHP 7.0 FastCGI Process Manager...
Mar 07 15:21:00 xxx systemd[1]: Stopped The PHP 7.0 FastCGI Process Manager.
# cat /var/log/dpkg.log | grep php | grep fpm
[...]
2019-03-07 15:20:58 status triggers-pending php7.0-fpm:amd64 7.0.30-
2019-03-07 15:21:00 upgrade php7.0-fpm:amd64 7.0.30-
2019-03-07 15:21:00 status half-configured php7.0-fpm:amd64 7.0.30-
2019-03-07 15:21:00 status unpacked php7.0-fpm:amd64 7.0.30-
2019-03-07 15:21:00 status half-installed php7.0-fpm:amd64 7.0.30-
2019-03-07 15:21:00 status half-installed php7.0-fpm:amd64 7.0.30-
2019-03-07 15:21:00 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:21:00 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 configure php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status unpacked php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:31 status half-configured php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:32 status installed php7.0-fpm:amd64 7.0.33-
2019-03-07 15:31:32 status triggers-pending php7.0-fpm:amd64 7.0.33-
$ lsb_release -rd
Description: Ubuntu 16.04.4 LTS
Release: 16.04
$ apt-cache policy php7.0-fpm
php7.0-fpm:
Installed: 7.0.33-
Candidate: 7.0.33-
Version table:
*** 7.0.33-
500 http://
500 http://
100 /var/lib/
7.0.4-7ubuntu2 500
500 http://
Related branches
- Christian Ehrhardt (community): Approve
- Canonical Server: Pending requested
-
Diff: 33 lines (+14/-0)2 files modifieddebian/changelog (+8/-0)
debian/rules (+6/-0)
Odd, in a quick repro I haven't seen any delay.
There are prerm/postrm/ postinst steps in this package.
In addition as we have seen with other packages it has sysV and systemd units which sometimes can cause double actions in maintainer scripts.
Prerm of the old version will stop it (sysV)
invoke-rc.d php7.0-fpm stop
Postrm will do nothind on upgrade
Postinst of the new version will then start it
invoke-rc.d php7.0-fpm start
I checked - even 7.2.15 in Disco fixed it as it correctly adds after-upgrade after-upgrade
dh_installinit --restart-
dh_systemd_start --restart-
Which will make the stop in prerm only happen if you are removing the package.
Without that if the upgrades take long (for whatever reason overload/ slow-disk/ ...) the described issue can happen. That was the main point when introducing --restart- after-upgrade.
Although there has to be taken extra care as the behaviro of dh_installinit and co changed in different compat levels.
That said - the fix seems clear, lets check when it was added [1].
Hmm that is pre-xenial for sure 7.0.6-13~2 to be specific.
And latter 7.1 / 7.2 hold it as well.
What happened to our 7.0? after-upgrade for both dh_installinit and
E.g. the version we had in Artful had this for sure
+php7.0 (7.0.6-12) unstable; urgency=medium
+
+ * Enable --restart-
+ dh_systemd_start to minimize downtimes
Xenial was originally on 7.0.4-7ubuntu2 and since then got only security updates.
That means the fix from 7.0.6-12 isn't in there.
IMHO One would need to pick and package [2] for Xenial.
[1]: https:/ /salsa. debian. org/php- team/php/ commit/ 8e34b064ccdcb97 36668dec2e184a5 77a202880f /git.launchpad. net/ubuntu/ +source/ php7.0/ diff/debian/ rules?h= import/ 7.0.6-12& id=ee74f97a368a 45eb8254ae9a26c b001732b85b6f
[2]: https:/