Apache doesn't start if /var/log/apache2 is missing

[Removed by request]

I'm using Ubuntu 12.04 dev with apache2 2.2.22-1ubuntu1. If the directory /var/log/apache2 doesn't exist the apache server can't start:

sworddragon@ubuntu-vm:~$ sudo /etc/init.d/apache2 start
 * Starting web server apache2
apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.1.101 for ServerName
(2)No such file or directory: apache2: could not open error log file /var/log/apache2/error.log.
Unable to open logs
Action 'start' failed.
The Apache error log may have more information.

I think /etc/init.d/apache2 should check for this directory and create it if it is missing.

[Davidf-a (davidf-a)]

If httpd is configured to log to a file in a non-existing directory, it refuses
to start up, e.g.:
[Fri Sep 28 03:58:13 2007] [notice] caught SIGTERM, shutting down
(2)No such file or directory: httpd: could not open error log file
/etc/httpd/logs/mailman/error_log.
Unable to open logs

It's useful when restarting Apache to check if any configuration changes will
prevent the restart from being successful (and thus result in down time when the
restart command is issued). However the configuration syntax check doesn't check
if log file directories exist and so doesn't catch this problem

Other non-existing directories (for serving files out of etc) don't prevent the
entire process from starting up and so aren't such critical errors.

It would be great if either
1) the inability to create a log file did not prevent httpd from starting
2) the config test (or another test) checked for this condition (and possible
other config options that would prevent startup)

[Davidf-a (davidf-a)]

Tried to code a patch for this but couldn't find where ap_run_test_config is
defined :-(

[Rahul-g-nair (rahul-g-nair)]

Created attachment 22281
use test_config to check if errorlog can be opened

A tiny patch to see if errorlog can be opened during httpd -t
Since test_config does not let me return a value, This will still return
Syntax OK after printing the warning.

[Rahul-g-nair (rahul-g-nair)]

Note: The above patch assumes that mod_log_config is loaded.

[Davidf-a (davidf-a)]

Created attachment 23280
Checks error logs and custom logs for the main host

Thanks for the patch rahul! Got me started...

The new patch I'm attaching now also checks files mentioned in CustomLog or TransferLog directives.

Problems that remain:
 * The attempt is made relative to the current directory, not the eventual ServerRoot
 * Virtual Host configurations are not checked

[Davidf-a (davidf-a)]

Created attachment 23281
Checks error logs and custom logs with relative paths for the main host and virtual hosts

Adjusted patch to address the problems above: it now iterates over the virtual hosts, and relative paths are used too.

It also checks whether the error log is even defined, and adjusts the language to indicate which config file it is looking at.

Sample output:
[root@kulma httpd]# /etc/init.d/httpd configtest
(2)No such file or directory: Could not open error log file /etc/httpd/logs2/error_log for server defined in main config file.
(2)No such file or directory: Could not open log file /etc/httpd/logs2/access_log for server defined in main config file.
(2)No such file or directory: Could not open log file /etc/httpd/logs2/access_log for server defined in /etc/httpd/conf.d/virtual.d/zimbra.conf.
(2)No such file or directory: Could not open error log file /etc/httpd/logs2/trac/error_log for server defined in /etc/httpd/conf.d/virtual.d/trac.conf.
(2)No such file or directory: Could not open log file /etc/httpd/logs2/trac/access_log for server defined in /etc/httpd/conf.d/virtual.d/trac.conf.
(2)No such file or directory: Could not open error log file /etc/httpd/logs2/error_log for server defined in /etc/httpd/conf.d/virtual.d/test.conf.
(2)No such file or directory: Could not open error log file /etc/httpd/logs2/error_log for server defined in /etc/httpd/conf.d/virtual.d/staging.conf.
(2)No such file or directory: Could not open log file /etc/httpd/logs2/access_log for server defined in /etc/httpd/conf.d/virtual.d/old.conf.
(2)No such file or directory: Could not open log file /etc/httpd/logs2/access_log for server defined in /etc/httpd/conf/httpd.conf.
Syntax OK

The patch is against 2.2.8, and is complete from my side.

[Davidf-a (davidf-a)]

I'm not familiar with Apache procedures, so not sure where to go from here...

[R-nick-q (r-nick-q)]

(In reply to comment #6)
> I'm not familiar with Apache procedures, so not sure where to go from here...
>

Posting a patch and marking the bug report PatchAvailable is a good start. If you're lucky, someone notices and reviews the patch. If nothing happens, you can raise it on the dev list.

First at-a-glance comment on this patch: how does it work with piped logs?

[Jorton-9 (jorton-9)]

It seems pretty odd to do this. If you have some process which is randomly making your log directory unwritable outside of your control (or knowledge), who is to say that the log directory isn't going to become unwritable between when the testconfig hook runs, and startup completes?

[Davidf-a (davidf-a)]

(In reply to comment #8)
> It seems pretty odd to do this. If you have some process which is randomly
> making your log directory unwritable outside of your control (or knowledge),
> who is to say that the log directory isn't going to become unwritable between
> when the testconfig hook runs, and startup completes?
>

The original use case for this was not someone making the log directory unwriteable, but someone (myself!) forgetting to add the directory referred to in the ErrorLog directive. I would then run configtest and since it said everything looked OK, thought I was good to go - but restarting the server would actually stop it dead.

At least printing out a warning gives you a flag that that needs to be fixed. It needn't actually try writing to the files - just checking that their directories exist would be OK for me...

[Davidf-a (davidf-a)]

(In reply to comment #7)
> (In reply to comment #6)
> > I'm not familiar with Apache procedures, so not sure where to go from here...
> >
>
> Posting a patch and marking the bug report PatchAvailable is a good start. If
> you're lucky, someone notices and reviews the patch. If nothing happens, you
> can raise it on the dev list.
Great, thanks
> First at-a-glance comment on this patch: how does it work with piped logs?
It didn't, thanks :-)
Will attach a new version...

[Davidf-a (davidf-a)]

Created attachment 23282
Checks error logs and custom logs with relative paths for the main host and virtual hosts, ignoring pipes

Attached new patch ignoring piped logs

[Martin-smith-1 (martin-smith-1)]

Apache also doesn't bother checking if log directories exist when you reload it with /etc/init.d/httpd reload and just fails silently, meaning your sites go down because you added a new vhost and forgot to make the log directory.

It also would be nice if the reload init script printed something to stderr if something went wrong.

[Daniel Manrique (roadmr)]

There's an upstream bug here:

https://issues.apache.org/bugzilla/show_bug.cgi?id=43502

[Julius-4 (julius-4)]

Seriously, we're 5 years onwards and this still has not made it into the stable release?

[Aeoris (aeoris)]

Almost 7 years and counting. This is still an issue in most recent versions. What the heck, Apache?

[Anthony Geoghegan (anthony-geoghegan)]

Over the past few years, I've been bitten by this issue on a number of occasions where the *Error Log* for a virtual host configuration had its directory moved / renamed but the `configtest` still reports that everything is OK.

I created an account to report this as a bug / feature request only to discover that a patch for this issue was already submitted over 7 years ago. As a user, I'd love to see this patch (or a revised version of it) added to the production code.

[Jason Wood (sandain)]

This requires systemd 235, which is in the Bionic Beaver, but a good fix for this issue would be to simply modify the apache2.service file to include the LogsDirectory directive.

[Jason Wood (sandain)]

I should note that I hit this issue only because I use tmpfs to mount /var/log on one of my systems where I don't care about logs surviving a restart. This is obviously non-standard, so it is of low priority, but it would be nice if the above patch could make it into the next version of Ubuntu.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "patch.diff" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Christian Ehrhardt  (paelzer)]

Hi Jason,
I wanted to thank you for your participation.
But on the suggested change I have a few comments.

First of all if you install apache2 it will create /var/log/apache2 by debian/apache2.dirs in the packaging telling it to do so.
I see that if you run on tmpfs you'll loose that directory on e.g. reboots and then run into this issue.
But then it is kind of a special configuration that has to take care.

I like that "just" adding this single line would solve a very old apache bug so trivially.
But there are a few things to be considered.
First of all the dir is configurable in /etc/apache2/envvars and then reused.
See:
 $ grep -Hrn APACHE_LOG_DIR /etc/

So the change to the service file (if not denied by some other implications systemd makes on it).
Maybe with
Environment=/etc/apache2/envvars
LogsDirectory=${APACHE_LOG_DIR}

To be clear I like it, but I think you should address this upstream instead of in a deviation by Ubuntu.
But maybe you can give some live to the old bug if you would suggest that there (feel free to incooperate my suggestions)?

[Rainer Jung (rainer-jung-kippdata)]

Undo spam change

[Vladimir Kondratyev (vladimirkondratyev)]

Just another victim of this bug. I have a Docker container with Apache. All was OK until I mounted container's /var/log of to host's ~/log.

According to Docker documentation content of non-empty directory is obscured by bind mount, so I've got empty /var/log after container start.

The problem became even worse, because due to lack of /var/log/apache2, the httpd process was not able to log error about missing log. As the result the container started, but Apache did not.