php5-fpm upstart job is not compatible with precise upstart

Bug #1272788 reported by David Planella on 2014-01-26
26
This bug affects 4 people
Affects Status Importance Assigned to Milestone
php5 (Ubuntu)
High
Dimitri John Ledkov
Trusty
High
Dimitri John Ledkov

Bug Description

After upgrading a server from 12.04 to trusty, I noticed the php-fpm service would not start.

Running sudo service php-fpm, the following message was shown:

“Unknown job: php5-fpm”

Following the workaround here [1] and commenting out the "reload signal USR2" line in /etc/init/php5-fpm.conf fixed the issue.

[1]: https://github.com/gplessis/dotdeb-php5/issues/45

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: php5-fpm (not installed)
ProcVersionSignature: Ubuntu 3.13.0-5.20-generic 3.13.0
Uname: Linux 3.13.0-5-generic x86_64
ApportVersion: 2.13.1-0ubuntu2
Architecture: amd64
CurrentDesktop: Unity
Date: Sun Jan 26 00:58:53 2014
InstallationDate: Installed on 2013-09-21 (126 days ago)
InstallationMedia: Ubuntu 13.10 "Saucy Salamander" - Alpha amd64 (20130921)
SourcePackage: php5
UpgradeStatus: Upgraded to trusty on 2013-12-18 (38 days ago)

David Planella (dpm) wrote :
description: updated
Dimitri John Ledkov (xnox) wrote :

Was the machine rebooted after the upgrade, to actually pick up the new upstart?

Robie Basak (racb) wrote :

Confirmed. Reproduced on both LXC and KVM (in order to eliminate any overlayfs issue). Rebooting does seem to fix the issue. It seems to me that upstart doesn't pick up /etc/init.d/php5-fpm.conf. Perhaps this is related to the conversion from a SysV init script to upstart job in the upgrade? Is the php5-fpm packaging doing something wrong here?

Dimitri, even if a reboot fixes the issue, it shouldn't be required, right?

Changed in php5 (Ubuntu):
status: New → Confirmed
importance: Undecided → Medium
summary: - php-fpm service won't start: unknown job
+ php-fpm service does not automatically restart after upgrade

Following a conversation and some debugging with Stéphane and James, the conclusion is that this is a bug in upstart which is failing to re-exec/reload itself and so doesn't see php-fpm upstart job.

affects: php5 (Ubuntu) → upstart (Ubuntu)
Changed in upstart (Ubuntu):
importance: Medium → High
status: Confirmed → Triaged
summary: - php-fpm service does not automatically restart after upgrade
+ New upstart jobs are not detected in a P to T upgrade

The conclusion is incorrect =)
upstart ignore jobs with unknown stanzas.
php-fpm specifies a stanza which is not available / unknown to upstart in precise.
also precise upstart doesn't know how to perform a stateful re-exec, hence a reboot is required to get the new init.

If it is desired for the upstart job to work, before rebooting one of the following are possible solutions:
* maintainer scripts fallback to init.d script
* make upstart job precise compatible, e.g. remove "reload signal" stanza
* (possibly) ship reload signal stanza in the .override file (precise upstart will ignore override, trusty upstart will use it)

Changed in upstart (Ubuntu):
assignee: nobody → Dimitri John Ledkov (xnox)
summary: - New upstart jobs are not detected in a P to T upgrade
+ php5-fpm upstart job is not compatible with precise upstart
affects: upstart (Ubuntu) → php5 (Ubuntu)
Changed in php5 (Ubuntu):
assignee: Dimitri John Ledkov (xnox) → nobody
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in php5 (Ubuntu Trusty):
status: Triaged → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package php5 - 5.5.9+dfsg-1ubuntu4

---------------
php5 (5.5.9+dfsg-1ubuntu4) trusty; urgency=medium

  * Comment out "reload signal USR2" stanza from php5-fpm to make the job
    compatible with Precise upstart, when it's still running as pid1
    during upgrade to trusty and before the restart. We'd rather support
    shorter down-time then reload interface. (LP: #1272788)
 -- Dimitri John Ledkov <email address hidden> Wed, 09 Apr 2014 16:23:30 +0100

Changed in php5 (Ubuntu Trusty):
status: Fix Committed → Fix Released
Renan Gonçalves (renan-saddam) wrote :

The package 5.5.9+dfsg-1ubuntu3 have the correct upstart script with "reload signal USR2", which works great on Ubuntu Trusty since it uses upstart >= 1.10.0

The fix for this bug introduces a new package, called "5.5.9+dfsg-1ubuntu4", which comments out "reload signal USR2".
Making it work on Precise but breaking on Trusty.

Installing a new Ubuntu machine with the latest version and latest PHP should be the priority to work out of the box. No adjustments should be needed.
If anyone wants to use Precise with the latest PHP version, adjustments to the upstart script are okay.

Hence, IMO this fix should be reversed.

@Renan

I believe Dimitri made this change to avoid breaking php5-fpm on upgrade
from Precise to Trusty, because the previous version of upstart needs to
have support for the new feature before we can use it.

AFAICT, this means that we can fix this in Trusty+1, but it will remain
like this in Trusty. The workaround is trivial, right? Just remove the
comment?

Dimitri John Ledkov (xnox) wrote :

On 18 April 2014 11:58, Renan Gonçalves <email address hidden> wrote:
> The package 5.5.9+dfsg-1ubuntu3 have the correct upstart script with
> "reload signal USR2", which works great on Ubuntu Trusty since it uses
> upstart >= 1.10.0
>
> The fix for this bug introduces a new package, called "5.5.9+dfsg-1ubuntu4", which comments out "reload signal USR2".
> Making it work on Precise but breaking on Trusty.
>
> Installing a new Ubuntu machine with the latest version and latest PHP should be the priority to work out of the box. No adjustments should be needed.
> If anyone wants to use Precise with the latest PHP version, adjustments to the upstart script are okay.
>
> Hence, IMO this fix should be reversed.
>

We fully understand the drawbacks of all combinations, and there is no
win-win situation at the moment.
At release time, it was decided that smooth upgrades are more
important than supporting reload mechanism in this single job.
Independently, i'm working on patches to "service" and "invoke-rc.d"
commands to properly support this case.
Once those get accepted into Debian, and subsequently SRUed into
trusty, this change can be reverted and the service will be correctly
started using sysvinit upon upgrades from precise and will be managed
by upstart after reboot into trusty's upstart.

If you require reload functionality, you can uncoment that line in the
upstart job or add "reload signal USR2" into .override file next to
the job file.

--
Regards,

Dimitri.

Ruben Laban (r-laban) wrote :

This decision has some REALLY annoying side effects on Trusty, which took us quite some time to figure out.

With the commented reload stanza, the reload function is broken in a horrible way:

It sends SIGHUP to the php5-fpm master process causing it to terminate! Without any warning or indication that this is in fact happening. Based on the output the user thinks the reload worked just fine. However, after having done such a "reload", any further reloads, restarts or stops will fail, because the master process has been terminated.

James Qiang (shijialee) wrote :

I concurred with Ruben Laban. It is annoying and I didn't expect to find this behavior from the bug list. After all, Trusty is LTS..

Dimitri's third proposal seems to be perfect to me - no bug for upgrading from precise. and neither for the trusty user.

* (possibly) ship reload signal stanza in the .override file (precise upstart will ignore override, trusty upstart will use it)

Robie Basak (racb) wrote :

James, Ruben,

Your concerns are completely valid, but it isn't effective to track them here as this bug was about the upgrade path which is fixed, so this bug is marked Fix Released and thus isn't on anyone's radar. I understand that the fix regressed fpm reload behaviour so we should track this regression in a separate bug and I don't see a relevant one. If you can't find one then please could you file one and note that bug number here?

James Qiang (shijialee) wrote :

Hi, Robie

FYI, here is the content in Trusty /etc/init/php5-fpm.conf file,

# Precise upstart does not support reload signal, and thus rejects the
# job. We'd rather start the daemon, instead of forcing users to
# reboot https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1272788
#
# reload signal USR2

There is another bug report that you participated back in July https://bugs.launchpad.net/ubuntu/+source/php5/+bug/1242376

I have tested the above solution: adding 'reload signal USR2' in php5-fpm.override solve the problem.

Robie Basak (racb) wrote :

Thanks. Bug 1242376 looks like the bug we want to track for the regression James and Ruben described. Please mark yourself as affected in that bug ("Does this bug affect you?" link near the top).

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments