Comment 1 for bug 26607

Message-Id: <email address hidden>
Date: Mon, 05 Dec 2005 20:47:54 +0200
From: Fotos Georgiadis <email address hidden>
To: Debian Bug Tracking System <email address hidden>
Subject: apache2: Apache2 weird forks (owned by root)

Package: apache2
Version: 2.0.54-5
Severity: grave
Tags: security
Justification: renders package unusable

(What follow is the original messege I posted first on
the <email address hidden>, but no-one replied after a few days.
So now I report this as a Debian bug...)

Hello!

A strange problem occurs in our apache 2 installation. I have the
prefork MPM where the expected (and documented) behavior is one root
process open for managing the privileges ports (etc.) that spawns
children, with the privileges of the User and Group directives
specified (www-data in my case), in order to serve the requests.

Well, in our system what happens in that the children also have root
privileges despite the User www-data option. But what is more strange
is that this doesn't happen all the time. Sometimes the server starts
with 3 children owned by www-data and the rest 2 (out of a
MinSpareServers 5) are owned by root. The number varies between
restarts from all owned by root to all owned by www-data. (Don't
mention the security implications of the situation).

Processes owned by root are not serving pages (nor that I would want
the root user to serve pages...), and that means the capabilities of
the server are reduced. When all processes are owned by root, and a
client opens a connection it hangs there indefinitely. When only 1 or
2 processes are owned by www-data the server is really slow,
otherwise it behaves nicely under a moderate load.

Also issuing a reload (apache2ctl graceful) seems to zombie the child
processes and only SIGKILL can make them rest in peace. The problem
possibly lies in the forking section of apache 2. Killing the root-
owned children spawns new, sometimes owned by root, sometimes owned
by www-data. Killing enough root processes eventually allows us to
have all processes owned by www-data!

Now that I explained the situation and the problem, details about the
system follow:
Debian Sarge (3.1) up to date

Standard debian package for apache 2:
% apache2ctl -V:
Server version: Apache/2.0.54
Server built: Sep 5 2005 11:15:09
Server's Module Magic Number: 20020903:9
Architecture: 32-bit
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D HTTPD_ROOT=""
-D SUEXEC_BIN="/usr/lib/apache2/suexec2"
-D DEFAULT_PIDLOG="/var/run/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="/var/run/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types"
-D SERVER_CONFIG_FILE="/etc/apache2/apache2.conf"

Using the prefork MPM as already mentioned.

Server signature and other modules:
Apache/2.0.54 (Debian GNU/Linux)
PHP/4.3.10-16
mod_ssl/2.0.54
OpenSSL/0.9.7e Server

The configuration is a mixed IP-Based (using ip aliasing) and Name-
Based Virtual Hosting, with SSL enabled. Details (and full
configuration) can be given as requested. The configuration is not
the problem as the problem occurs even with no sites enabled. Also
the server works as advertised, excluding the problem I mentioned.

The only "custom" thing on the server is the kernel which is Linux
2.4.30 (being a Dell server it wouldn't boot with a debian vanilla
kernel). But I fail to see how the kernel can be a problem in this
situation (since every other service works perfectly (and there are a
lot)). It look like a race condition problem.

This on a SMP machine (Xeon @ 3.2Ghz, with 1GB ram), otherwise stable
as a rock.

I already searched the archives and couldn't find anything similar.
Perhaps I should also post a bug report here (oneline):
http://bugs.debian.org/cgi-bin/pkgreport.cgi? \
which=pkg&data=apache2&archive=no&version=&dist=stable

but I don't know if this is debian specific or an apache 2 problem.

That's all I can think of. Has anybody seen something similar?
Any developer comments? How can I help you trace this (ugly) bug?

Thanks in advance for reading all this (*wow* thats big)!
-fot

PS. On a side-note, as a joke we started placing bets on how many www-
data owned processes there will be on the next apache 2 restart! :) :)

-- System Information:
Debian Release: 3.1
Architecture: i386 (i686)
Kernel: Linux 2.4.30
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

Versions of packages apache2 depends on:
ii apache2-mpm-prefork 2.0.54-5 traditional model for Apache2

-- no debconf information