systemd unable to detect running apache if invoked via "apache2ctl graceful"

[RedScourge]

When starting or restarting Apache via the /usr/sbin/apache2ctl script, systemd becomes unaware of the state of Apache, causing "systemctl status apache2.service" to report "Active: inactive (dead)". Below I describe the issue and a fix; attached is the output of the diff command after I fixed it on my own 18.04 LTS system.

Description of issue and impact:

Despite this issue, Apache itself generally runs fine, so this is not an absolutely critical issue, however as both "systemctl status apache2.service" and "service apache2 status" become unable to detect its running state, and as some people may be scripting Apache service checks using those commands, they may experience server outages when they migrate from 14.04 LTS to 16.04 LTS or 18.04 LTS, so this could potentially have serious consequences. I suspect this caused or at least contributed to at least one outage of a web server I am responsible for, as the web file backup script was unable to restart the apache process after stopping it.

This issue affects Ubuntu 16.04 LTS when using "apache2ctl start" or "apache2ctl graceful", and Ubuntu 18.04 LTS just when using "apache2ctl graceful".

Fix:

The fix is made somewhat obvious due to the fact that the issue was fixed for "apache2ctl start" in 18.04 LTS , by replacing the regular call which resolves to "/usr/sbin/apache2 -k start" with a check for presence of systemd, followed by invoking "systemctl start apache2.service" if that check is affirmative, or falling back to the old start command if that check is negative.

Attached is a diff of the file before and after I copied the fixed invocation from the "star"t subsection to the "graceful" subsection in 18.04 LTS. A fix for 16.04 LTS would require copying that block both into the "start" and "graceful" subsections.

Additional information:

lsb_release -rd :

Description: Ubuntu 18.04.2 LTS
Release: 18.04

apt-cache policy apache2:

apache2:
  Installed: 2.4.29-1ubuntu4.6
  Candidate: 2.4.29-1ubuntu4.6

[RedScourge (redscourge)]

Output showing how to reproduce the issue:

root@server:/usr/local/sbin# apache2ctl stop
root@server:/usr/local/sbin# apache2ctl graceful
httpd not running, trying to start
root@server:/usr/local/sbin# systemctl status apache2.service
? apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           +-apache2-systemd.conf
   Active: inactive (dead) since Mon 2019-06-10 01:04:13 MDT; 17s ago
  Process: 27370 ExecStop=/usr/sbin/apachectl stop (code=exited, status=0/SUCCESS)
  Process: 11462 ExecReload=/usr/sbin/apachectl graceful (code=exited, status=0/SUCCESS)
 Main PID: 1849 (code=exited, status=0/SUCCESS)

Jun 08 19:11:24 server.wasteofcache.com apachectl[1408]: AH00112: Warning: DocumentRoot [/srv/apache2/www/russianfake
Jun 08 19:11:24 server.wasteofcache.com apachectl[1408]: AH00112: Warning: DocumentRoot [/srv/apache2/www/russianfake
Jun 08 19:11:24 server.wasteofcache.com apachectl[1408]: AH00112: Warning: DocumentRoot [/srv/apache2/www/stratmeme.c
Jun 08 19:11:24 server.wasteofcache.com apachectl[1408]: AH00112: Warning: DocumentRoot [/srv/apache2/www/stratmeme.c
Jun 08 19:11:24 server.wasteofcache.com apachectl[1408]: AH00112: Warning: DocumentRoot [/srv/apache2/www/wasteofcach
Jun 08 19:11:24 server.wasteofcache.com apachectl[1408]: AH00112: Warning: DocumentRoot [/srv/apache2/www/wasteofcach
Jun 08 19:11:24 server.wasteofcache.com systemd[1]: Started The Apache HTTP Server.
Jun 09 06:25:04 server.wasteofcache.com systemd[1]: Reloading The Apache HTTP Server.
Jun 09 06:25:04 server.wasteofcache.com systemd[1]: Reloaded The Apache HTTP Server.
Jun 10 01:04:13 server.wasteofcache.com apachectl[27370]: httpd (no pid file) not running

Output after customizing the apache2ctl script and restarting it:

root@server:/usr/local/sbin# apache2ctl_custom graceful
Invoking 'systemctl start apache2'.
Use 'systemctl status apache2' for more info.
root@server:/usr/local/sbin# systemctl status apache2.service
? apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           +-apache2-systemd.conf
   Active: active (running) since Mon 2019-06-10 01:04:50 MDT; 4s ago
  Process: 27370 ExecStop=/usr/sbin/apachectl stop (code=exited, status=0/SUCCESS)
  Process: 11462 ExecReload=/usr/sbin/apachectl graceful (code=exited, status=0/SUCCESS)
  Process: 27432 ExecStart=/usr/sbin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 27444 (apache2)
    Tasks: 6 (limit: 2318)
   CGroup: /system.slice/apache2.service
           +-27444 /usr/sbin/apache2 -k start
           +-27448 /usr/sbin/apache2 -k start
           +-27449 /usr/sbin/apache2 -k start
           +-27451 /usr/sbin/apache2 -k start
           +-27454 /usr/sbin/apache2 -k start
           +-27455 /usr/sbin/apache2 -k start

Jun 10 01:04:50 server.wasteofcache.com systemd[1]: Starting The Apache HTTP Server...
Jun 10 01:04:50 server.wasteofcache.com systemd[1]: Started The Apache HTTP Server.

[RedScourge (redscourge)]

(if someone could redact the domain names and file paths from the above comment that would be much appreciated)

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "diff of /usr/sbin/apache2ctl after correcting issue" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Andreas Hasenack (ahasenack)]

Thanks for the detailed report. It looks like it's the same issue as https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=927302, so I'm linking them both together.