systemd unable to detect running apache if invoked via "apache2ctl graceful"
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| | apache2 (Debian) |
New
|
Unknown
|
||
| | apache2 (Ubuntu) |
High
|
Unassigned | ||
Bug Description
When starting or restarting Apache via the /usr/sbin/
Description of issue and impact:
Despite this issue, Apache itself generally runs fine, so this is not an absolutely critical issue, however as both "systemctl status apache2.service" and "service apache2 status" become unable to detect its running state, and as some people may be scripting Apache service checks using those commands, they may experience server outages when they migrate from 14.04 LTS to 16.04 LTS or 18.04 LTS, so this could potentially have serious consequences. I suspect this caused or at least contributed to at least one outage of a web server I am responsible for, as the web file backup script was unable to restart the apache process after stopping it.
This issue affects Ubuntu 16.04 LTS when using "apache2ctl start" or "apache2ctl graceful", and Ubuntu 18.04 LTS just when using "apache2ctl graceful".
Fix:
The fix is made somewhat obvious due to the fact that the issue was fixed for "apache2ctl start" in 18.04 LTS , by replacing the regular call which resolves to "/usr/sbin/apache2 -k start" with a check for presence of systemd, followed by invoking "systemctl start apache2.service" if that check is affirmative, or falling back to the old start command if that check is negative.
Attached is a diff of the file before and after I copied the fixed invocation from the "star"t subsection to the "graceful" subsection in 18.04 LTS. A fix for 16.04 LTS would require copying that block both into the "start" and "graceful" subsections.
Additional information:
lsb_release -rd :
Description: Ubuntu 18.04.2 LTS
Release: 18.04
apt-cache policy apache2:
apache2:
Installed: 2.4.29-1ubuntu4.6
Candidate: 2.4.29-1ubuntu4.6
| RedScourge (redscourge) wrote : | #2 |
| RedScourge (redscourge) wrote : | #3 |
(if someone could redact the domain names and file paths from the above comment that would be much appreciated)
The attachment "diff of /usr/sbin/
[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]
| tags: | added: patch |
| Andreas Hasenack (ahasenack) wrote : | #5 |
Thanks for the detailed report. It looks like it's the same issue as https:/
| Changed in apache2 (Ubuntu): | |
| status: | New → Triaged |
| tags: | added: server-next |
| Changed in apache2 (Ubuntu): | |
| importance: | Undecided → Low |
| Changed in apache2 (Debian): | |
| status: | Unknown → New |
| Andreas Hasenack (ahasenack) wrote : | #6 |
Acknowledging that the bug still exists and is in our queue, but not at high priority.
| RedScourge (redscourge) wrote : | #7 |
UPDATE: I just had a server experience downtime due to this bug. An update of the apache2 package was automatically triggered on a client's 16.04 LTS server on Aug 31 via the unattended-upgrades cron script, and apparently before this update, the last time the Apache service had been restarted was done via "apachectl graceful", without an appropriately modified /usr/sbin/apachectl as described in my initial report, so the usual unattended-upgrades stop-and-start which follows an Apache update failed to perform as expected, leaving the service in the stopped state.
Therefore, it appears there is now a definite case where a user invoking "apachectl graceful" leads to downtime. To mitigate this, you can either disable/uninstall unattended-
I have deleted the old patch file and replaced it with an actual correct one based on the apache2ctl file on my client's server after the latest apache2 package update, now that I know how diff patch files are generated
| Bryce Harrington (bryce) wrote : | #8 |
Bumping importance on this bug, since it is causing downtime on production systems. It is already on our server-next queue and known in debian BTS.
| Changed in apache2 (Ubuntu): | |
| importance: | Low → High |
| Changed in apache2 (Ubuntu): | |
| status: | Triaged → Fix Committed |
| Changed in apache2 (Ubuntu): | |
| status: | Fix Committed → Fix Released |
| Bryce Harrington (bryce) wrote : | #10 |
@faisal, please don't change status on bugs without an explanation.
| Changed in apache2 (Ubuntu): | |
| status: | Fix Released → Triaged |
Output showing how to reproduce the issue:
root@server:
root@server:
httpd not running, trying to start
root@server:
? apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/
Drop-In: /lib/systemd/
Active: inactive (dead) since Mon 2019-06-10 01:04:13 MDT; 17s ago
Process: 27370 ExecStop=
Process: 11462 ExecReload=
Main PID: 1849 (code=exited, status=0/SUCCESS)
Jun 08 19:11:24 server.
Jun 08 19:11:24 server.
Jun 08 19:11:24 server.
Jun 08 19:11:24 server.
Jun 08 19:11:24 server.
Jun 08 19:11:24 server.
Jun 08 19:11:24 server.
Jun 09 06:25:04 server.
Jun 09 06:25:04 server.
Jun 10 01:04:13 server.
Output after customizing the apache2ctl script and restarting it:
root@server:
Invoking 'systemctl start apache2'.
Use 'systemctl status apache2' for more info.
root@server:
? apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/
Drop-In: /lib/systemd/
Active: active (running) since Mon 2019-06-10 01:04:50 MDT; 4s ago
Process: 27370 ExecStop=
Process: 11462 ExecReload=
Process: 27432 ExecStart=
Main PID: 27444 (apache2)
Tasks: 6 (limit: 2318)
CGroup: /system.
+-27444 /usr/sbin/apache2 -k start
+-27448 /usr/sbin/apache2 -k start
+-27449 /usr/sbin/apache2 -k start
+-27451 /usr/sbin/apache2 -k start
+-27454 /usr/sbin/apache2 -k start
+-27455 /usr/sbin/apache2 -k start
Jun 10 01:04:50 server.
Jun 10 01:04:50 server.