libapache2-mod-perl2 writes wrong process name in /proc/$$/status

[Asgard]

# uname -a
Linux ---.com 3.2.0-24-generic #39-Ubuntu SMP Mon May 21 16:52:17 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/issue
Ubuntu 12.04 LTS \n \l

# dpkg-query -l|grep apache
ii apache2 2.2.22-1ubuntu1 Apache HTTP Server metapackage
ii apache2-mpm-itk 2.2.22-1ubuntu1 multiuser MPM for Apache 2.2
ii apache2-suexec 2.2.22-1ubuntu1 Standard suexec program for Apache 2 mod_suexec
ii apache2-utils 2.2.22-1ubuntu1 utility programs for webservers
ii apache2.2-bin 2.2.22-1ubuntu1 Apache HTTP Server common binary files
ii apache2.2-common 2.2.22-1ubuntu1 Apache HTTP Server common files
ii libapache2-mod-perl2 2.0.5-5ubuntu1 Integration of perl with the Apache2 web server
ii libapache2-mod-php5 5.3.10-1ubuntu3.1 server-side, HTML-embedded scripting language (Apache 2 module)
ii libapache2-mod-rpaf 0.6-2 module for Apache2 which takes the last IP from the 'X-Forwarded-For' header
ii libapache2-reload-perl 0.11-2 module for reloading Perl modules when changed on disk

# apache2 -v
Server version: Apache/2.2.22 (Ubuntu)
Server built: Feb 13 2012 01:51:55

# ps axu|grep apache
root 9070 0.0 0.3 526708 99456 ? S 12:04 0:00 /usr/sbin/apache2 -k start
....

# cat /proc/9070/status
Name: /usr/sbin/apach
State: S (sleeping)
Tgid: 9070
...

The "Name:" must be "apache2". I have troubles in searching processes by name with some unpleasant consequences.
If prefork version of apache is installed, it shows correct process name.

[Panos (panos-redhat-bugs)]

Description of problem: An configuration entry to display the number of httpd processes running does not work on recent SNMP servers.

Version-Release number of selected component (if applicable): net-snmp-5.7.1-2.fc16.x86_64

How reproducible: Enable public SNMP community and add the following entry in /etc/snmp/snmpd.conf:
proc httpd 30 1

Steps to Reproduce:
1. Edit /etc/snmp/snmpd.conf
2. Enable public SNMP community and add "proc httpd 30 1"
3. Restart snmpd by issuing "systemctl restart snmpd.service" as root
4. Run "snmpwalk -v 1 localhost -c public .1.3.6.1.4.1.2021.2"

Actual results:
UCD-SNMP-MIB::prCount.1 = INTEGER: 0
UCD-SNMP-MIB::prErrorFlag.1 = INTEGER: error(1)
UCD-SNMP-MIB::prErrMessage.1 = STRING: No httpd process running

Expected results:
UCD-SNMP-MIB::prCount.1 = INTEGER: 9
UCD-SNMP-MIB::prErrorFlag.1 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrMessage.1 = STRING:

Additional info:
The number of the httpd processes currently run in the system:

# ps -ef | grep -i httpd
root 1067 1 0 Jan16 ? 00:00:02 /usr/sbin/httpd -k start
apache 1078 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start
apache 1079 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start
apache 1080 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start
apache 1081 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start
apache 1082 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start
apache 1083 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start
apache 1084 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start
apache 1085 1067 0 Jan16 ? 00:00:00 /usr/sbin/httpd -k start

The full snmpd.conf configuration is also attached for your convenience.

[Panos (panos-redhat-bugs)]

Created attachment 555720
SNMPD configuration

Configuration file /etc/snmp/snmpd.conf

[Panos (panos-redhat-bugs)]

The rest configured processes (smbd & sshd) work fine:

# snmpwalk -v 1 localhost -c public .1.3.6.1.4.1.2021.2
UCD-SNMP-MIB::prIndex.1 = INTEGER: 1
UCD-SNMP-MIB::prIndex.2 = INTEGER: 2
UCD-SNMP-MIB::prIndex.3 = INTEGER: 3
UCD-SNMP-MIB::prNames.1 = STRING: httpd
UCD-SNMP-MIB::prNames.2 = STRING: smbd
UCD-SNMP-MIB::prNames.3 = STRING: sshd
UCD-SNMP-MIB::prMin.1 = INTEGER: 1
UCD-SNMP-MIB::prMin.2 = INTEGER: 1
UCD-SNMP-MIB::prMin.3 = INTEGER: 1
UCD-SNMP-MIB::prMax.1 = INTEGER: 30
UCD-SNMP-MIB::prMax.2 = INTEGER: 25
UCD-SNMP-MIB::prMax.3 = INTEGER: 20
UCD-SNMP-MIB::prCount.1 = INTEGER: 0
UCD-SNMP-MIB::prCount.2 = INTEGER: 2
UCD-SNMP-MIB::prCount.3 = INTEGER: 1
UCD-SNMP-MIB::prErrorFlag.1 = INTEGER: error(1)
UCD-SNMP-MIB::prErrorFlag.2 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrorFlag.3 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrMessage.1 = STRING: No httpd process running
UCD-SNMP-MIB::prErrMessage.2 = STRING:
UCD-SNMP-MIB::prErrMessage.3 = STRING:
UCD-SNMP-MIB::prErrFix.1 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrFix.2 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrFix.3 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrFixCmd.1 = STRING:
UCD-SNMP-MIB::prErrFixCmd.2 = STRING:
UCD-SNMP-MIB::prErrFixCmd.3 = STRING:

[Jan (jan-redhat-bugs)]

In my test environment I can see all httpd daemons in prTable. Please note that
net-snmp-5.7.1 uses caches with 30 second timeout to count processes, i.e. the prTable is updated every 30 seconds with new/exited processes. Did you wait at least half a minute after httpd service start/stop?

BTW, I don't know how this 30 second timeout was chosen, maybe it's too long.

[Panos (panos-redhat-bugs)]

Hello Jan,

Something very weird is happening. It now works:

# snmpwalk -v 1 localhost -c public .1.3.6.1.4.1.2021.2
UCD-SNMP-MIB::prIndex.1 = INTEGER: 1
UCD-SNMP-MIB::prIndex.2 = INTEGER: 2
UCD-SNMP-MIB::prIndex.3 = INTEGER: 3
UCD-SNMP-MIB::prNames.1 = STRING: httpd
UCD-SNMP-MIB::prNames.2 = STRING: smbd
UCD-SNMP-MIB::prNames.3 = STRING: sshd
UCD-SNMP-MIB::prMin.1 = INTEGER: 1
UCD-SNMP-MIB::prMin.2 = INTEGER: 1
UCD-SNMP-MIB::prMin.3 = INTEGER: 1
UCD-SNMP-MIB::prMax.1 = INTEGER: 30
UCD-SNMP-MIB::prMax.2 = INTEGER: 25
UCD-SNMP-MIB::prMax.3 = INTEGER: 20
UCD-SNMP-MIB::prCount.1 = INTEGER: 4
UCD-SNMP-MIB::prCount.2 = INTEGER: 2
UCD-SNMP-MIB::prCount.3 = INTEGER: 7
UCD-SNMP-MIB::prErrorFlag.1 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrorFlag.2 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrorFlag.3 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrMessage.1 = STRING:
UCD-SNMP-MIB::prErrMessage.2 = STRING:
UCD-SNMP-MIB::prErrMessage.3 = STRING:
UCD-SNMP-MIB::prErrFix.1 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrFix.2 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrFix.3 = INTEGER: noError(0)
UCD-SNMP-MIB::prErrFixCmd.1 = STRING:
UCD-SNMP-MIB::prErrFixCmd.2 = STRING:
UCD-SNMP-MIB::prErrFixCmd.3 = STRING:

but only for an Apache started as normal user having 4 processes. The system processes are not calculated:

# ps -ef | grep -i httpd
root 1044 1 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1047 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1048 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1049 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1050 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1051 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1052 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1053 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
apache 1054 1044 0 10:39 ? 00:00:00 /usr/sbin/httpd -k start
n-lex 6274 1 0 12:24 ? 00:00:00 /home/n-lex/tools/bin/httpd -f /home/n-lex/natlex_utils/Nat-Lex_perlsoap/conf/soap_httpd.conf
n-lex 6275 6274 0 12:24 ? 00:00:00 /home/n-lex/tools/bin/httpd -f /home/n-lex/natlex_utils/Nat-Lex_perlsoap/conf/soap_httpd.conf
n-lex 6276 6274 0 12:24 ? 00:00:00 /home/n-lex/tools/bin/httpd -f /home/n-lex/natlex_utils/Nat-Lex_perlsoap/conf/soap_httpd.conf
n-lex 6277 6274 0 12:24 ? 00:00:00 /home/n-lex/tools/bin/httpd -f /home/n-lex/natlex_utils/Nat-Lex_perlsoap/conf/soap_httpd.conf

# id apache
uid=48(apache) gid=48(apache) groups=48(apache)
# id n-lex
uid=503(n-lex) gid=100(users) groups=100(users)

The system's apache server is run at boot time, my PC is always turned on 24/7 and only restarts may occur. As I can see from my MRTG graphs for httpd processes run every 5 minutes, it was unable to count ever the system's httpd processes, but only that development httpd.

[Panos (panos-redhat-bugs)]

Created attachment 556266
Latest daily MRTG graph for httpd processes

[Panos (panos-redhat-bugs)]

Created attachment 556267
Latest weekly MRTG graph for httpd processes

[Panos (panos-redhat-bugs)]

Created attachment 556269
Latest yearly MRTG graph for httpd processes

As you can see from this graph, the problem occurred on early December. It used to work fine before by displaying 9 processes.

[Jan (jan-redhat-bugs)]

That's very strange. Can you see your httpd processes in hrSWRunTable? It shares the same data with prTable.

snmptable localhost hrSWRunTable | grep httpd

If you don't see any httpd there, I need strace of snmpd. Please run following commands (in different terminals) and send me their output. I need to see how snmpd goes through /proc/* and reads /proc/*/cmdline of every process and also what it finds there (= the snmpwalk output).

$ service snmpd stop; strace -s 9999 snmpd -f -Lo 2>&1 | tee strace.out

(in different terminal)
$ (snmpwalk -v2c -c public localhost hrSWRunTable ; snmpwalk -v2c -c public localhost prTable) | tee walk.out

Also, do you remember anything interesting happening in beginning of December, like updating the machine (see /var/log/yum.log) or so?

[Panos (panos-redhat-bugs)]

Hello Jan,

I confirm that hrSWRunTable displays my system httpd processes:

# snmptable -c public -v 1 localhost hrSWRunTable | grep httpd
         1051 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1060 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1061 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1062 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1063 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1064 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1065 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1066 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         1067 "/usr/sbin/httpd" SNMPv2-SMI::zeroDotZero "/usr/sbin/httpd" "-k start" application runnable
         6371 "grep" SNMPv2-SMI::zeroDotZero

Please, tell me if you still need the strace, because I have understood that you need it in case that no httpd process is displayed.

I'm also attaching the yum update log. A major system event was the upgrade from F15 to F16 on Nov 22, 2012.

[Panos (panos-redhat-bugs)]

Created attachment 556485
Yum update log of previous year

[Panos (panos-redhat-bugs)]

Created attachment 556487
Yum update log of previous year

Attaching the correct yum log

[Jan (jan-redhat-bugs)]

Sorry for late response...

I can see a difference in my and your setup. My hrSWRunTable short "httpd" as name of the httpd process, your shows "/usr/sbin/httpd"

Try using "proc /usr/sbin/httpd 30 1" in your snmpd.conf as a workaround.

And yes, please provide the strace, I want to see where the difference comes from.

[Panos (panos-redhat-bugs)]

Created attachment 566293
Strace and snmwalk output

Hello Jan,

Please, find attached strace-walk.zip file containing both requested outputs.

[Jan (jan-redhat-bugs)]

Thanks a lot!

In the strace I can see snmpd reads /proc/1048/status. snmpd interprets the Name: line as name of the process and your kernel reports full path of the binary "Name: /usr/sbin/httpd" here, while e.g. /proc/971/status shows only name of the binary: "Name: bluetoothd".

That's why snmpd does not match "httpd" process and reports UCD-SNMP-MIB::prCount.1 = 0.

My kernel (kernel-3.2.5-3.fc16.x86_64) shows "Name: httpd" in /proc/XYZ/status of httpd processes.

Try to use full path in your snmpd.conf as suggested in comment #12. I have no idea why kernel reports full path name to the binary, you can ask kernel guys (i.e. reassign this bug to them). Definitely, there is no bug in snmpd.

[Panos (panos-redhat-bugs)]

Hello Jan,

Thank you very much. I have also verified that only for httpd process the full path is given in the Name field of status file in /proc/<PID>/status.

Re-assigning the bug to kernel folks. My kernel is the latest kernel-3.2.7-1.fc16.x86_64. Their feedback could be very helpful. The question is why HTTPD process is reported with full path /usr/sbin/httpd instead of httpd?

[Josh (josh-redhat-bugs)]

This isn't a kernel bug.

A process can call prctl(PR_SET_NAME) to set it's task name (task->comm) in the kernel, as long as the name is under 16 characters. If you have mod_perl included in httpd, it calls prctl and sets the name to '/usr/sbin/httpd'.

Why it does this, I have absolutely no clue. But it does.

[jwboyer@hansolo ~]$ sudo gdb /usr/sbin/httpd
GNU gdb (GDB) Fedora (7.3.50.20110722-10.fc16)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/httpd...Reading symbols from /usr/lib/debug/usr/sbin/httpd.debug...done.
done.
(gdb) set args -k start
(gdb) break prctl
Breakpoint 1 at 0x162f0
(gdb) r
Starting program: /usr/sbin/httpd -k start
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, prctl () at ../sysdeps/unix/syscall-template.S:82
82 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
Missing separate debuginfos, use: debuginfo-install avahi-libs-0.6.30-4.fc16.x86_64 cyrus-sasl-lib-2.1.23-27.fc16.x86_64 dbus-libs-1.4.10-3.fc16.x86_64 keyutils-libs-1.5.2-1.fc16.x86_64 krb5-libs-1.9.2-6.fc16.x86_64 libcom_err-1.41.14-2.fc15.x86_64 libselinux-2.1.6-6.fc16.x86_64 libuuid-2.20.1-2.2.fc16.x86_64 mod_dnssd-0.6-4.fc15.x86_64 mod_perl-2.0.5-7.fc16.x86_64 mod_python-3.3.1-16.fc16.x86_64 mod_wsgi-3.3-1.fc16.x86_64 nspr-4.8.9-2.fc16.x86_64 nss-3.13.1-11.fc16.x86_64 nss-util-3.13.1-3.fc16.x86_64 openldap-2.4.26-6.fc16.x86_64 openssl-1.0.0g-1.fc16.x86_64 python-libs-2.7.2-5.2.fc16.x86_64
(gdb) bt
#0 prctl () at ../sysdeps/unix/syscall-template.S:82
#1 0x00007fffed120abc in Perl_magic_set (my_perl=0x55555594c1d0,
    sv=<optimized out>, mg=<optimized out>) at mg.c:2979
#2 0x00007fffed11c942 in Perl_mg_set (my_perl=0x55555594c1d0,
    sv=0x55555596c9e0) at mg.c:302
#3 0x00007fffed412079 in modperl_startup ()
   from /etc/httpd/modules/mod_perl.so
#4 0x00007fffed411fa0 in modperl_startup ()
   from /etc/httpd/modules/mod_perl.so
#5 0x00007fffed4123f5 in modperl_init () from /etc/httpd/modules/mod_perl.so
#6 0x00007fffed41254b in modperl_hook_init ()
   from /etc/httpd/modules/mod_perl.so
#7 0x0000555555580511 in ap_run_open_logs (pconf=0x5555557b4138,
    plog=0x5555557e62c8, ptemp=0x5555557e82d8, s=0x5555557de3d0)
    at /usr/src/debug/httpd-2.2.22/server/config.c:151
#8 0x000055555556b5cc in main (argc=3, argv=0x7fffffffe618)
    at /usr/src/debug/httpd-2.2.22/server/main.c:680
(gdb) up
#1 0x00007fffed120abc in Perl_magic_set (my_perl=0x55555594c1d0,
    sv=<optimized out>, mg=<optimized out>) at mg.c:2979
2979 if (prctl(PR_SET_NAME, (unsigned long)s, 0, 0, 0) != 0) {
(gdb) list
2974 PL_origargv[0][PL_origalen-1] = 0;
2975 for (i = 1; i < PL_origargc; i++)
2976 PL_origargv[i] = 0;
2977 #ifdef HAS_PRCTL_SET_NAME
2978 /* ...

[Panos (panos-redhat-bugs)]

Hello Josh,

Thank you very much. I have also confirmed that disabling mod_perl allows HTTPD to set the name correctly.

Re-assigning the bug to mod_perl experts.

[Jan (jan-redhat-bugs)]

Panos,

when snmpd/MRTG was working perfectly, did you have mod_perl installed? I'm trying to find, if it is regression in mod_perl or already old mod_perl bug, which just got visible when you enabled it.

There was perl update on Nov 22 (4:perl-5.14.2-190.fc16.x86_64), could this be the guilty one?

[Jan (jan-redhat-bugs)]

Also mod_perl was updated at the end of November, this could be another source of regressions:
Nov 30 14:21:42 Installed: mod_perl-2.0.5-6.fc16.x86_64

[Panos (panos-redhat-bugs)]

Hello Jan,

I confirm that mod_perl wasn't installed before the problem appeared:

root@bb229:[205] ~ # grep -i mod_perl /var/log/yum.log*
/var/log/yum.log:Jan 12 08:49:38 Updated: mod_perl-2.0.5-7.fc16.x86_64
/var/log/yum.log:Jan 12 08:50:01 Updated: mod_perl-devel-2.0.5-7.fc16.x86_64
/var/log/yum.log-20120101:Nov 30 14:21:42 Installed: mod_perl-2.0.5-6.fc16.x86_64
/var/log/yum.log-20120101:Nov 30 14:21:43 Installed: mod_perl-devel-2.0.5-6.fc16.x86_64

Please, also note that Nov 22 was an upgrade date from F15 to F16.

[Jan (jan-redhat-bugs)]

From the first try, attached mod_perl patch fixed the problem for me. But I will better check it twice and find out what exactly is the meaning of that short_name variable.

[Jan (jan-redhat-bugs)]

Created attachment 567625
proposed patch

[Fedora (fedora-redhat-bugs)]

mod_perl-2.0.5-8.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/mod_perl-2.0.5-8.fc17

[Fedora (fedora-redhat-bugs)]

mod_perl-2.0.5-8.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/mod_perl-2.0.5-8.fc16

[Fedora (fedora-redhat-bugs)]

Package mod_perl-2.0.5-8.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing mod_perl-2.0.5-8.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-3185/mod_perl-2.0.5-8.fc17
then log in and leave karma (feedback).

[Fedora (fedora-redhat-bugs)]

mod_perl-2.0.5-8.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.

[Fedora (fedora-redhat-bugs)]

mod_perl-2.0.5-8.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1008385/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Albéric de Pertat (alberic)]

This also happens with apache2-mpm-worker

[Vladimir Kulev (me-lightoze)]

As well as apache2-mpm-prefork

[Robie Basak (racb)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.

I was unable to reproduce this using 2.2.22-1ubuntu1.2 - neither on mpm-prefork, nor mpm-worker, nor mpm-itk.

A difference that might be relevant is that I tried it on a virtual machine with a -virtual 3.2.0-23 kernel, as opposed to -generic.

It appears that this particular VM won't boot with a -generic kernel, so I can't easily go further right now.

[Dennis S. (dennix-ger)]

Ubuntu 12.04.1 is not affected by this bug. I reproduced this bug only with Ubuntu 12.04.2:

root@server:~# cat /proc/22315/status
Name: /usr/sbin/apach

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Albéric de Pertat (alberic)]

I confirm this bug on a freshly installed Ubuntu 12.04.2. However, it does not appear on an apparently similarly configured (yet older, probably installed as a 12.04.1) machine.

root@okserver:~# cat /proc/2761/stat
2761 (apache2) S 16432 16432 16432 0 -1 4202816 62931 0 23 0 446 35 0 0 39 19 1 0 15483478 392368128 10767 18446744073709551615 1 1 0 0 0 0 0 4096 201344747 18446744073709551615 0 0 17 0 0 0 25 0 0

root@misbehavingserver:~# cat /proc/13556/stat
13556 (/usr/sbin/apach) S 1 13556 13556 0 -1 4202560 40031 0 0 0 76 16 0 0 20 0 1 0 26449769 302252032 36195 18446744073709551615 140602045796352 140602046243212 140737419513488 140737419512584 140602031456275 0 0 4096 134235883 18446744071580458505 0 0 17 0 0 0 0 0 0

The kernels are the same :

$ uname -a
Linux XXXX 3.2.0-49-virtual #75-Ubuntu SMP Tue Jun 18 17:59:38 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

The point release are the same :

$ cat /etc/issue
Ubuntu 12.04.2 LTS \n \l

Apache is installed with the same packages :

root@okserver:~# dpkg -l | grep apache
ii apache2 2.2.22-1ubuntu1.4 Apache HTTP Server metapackage
ii apache2-mpm-prefork 2.2.22-1ubuntu1.4 Apache HTTP Server - traditional non-threaded model
ii apache2-utils 2.2.22-1ubuntu1.4 utility programs for webservers
ii apache2.2-bin 2.2.22-1ubuntu1.4 Apache HTTP Server common binary files
ii apache2.2-common 2.2.22-1ubuntu1.4 Apache HTTP Server common files
ii libapache2-mod-php5 5.3.10-1ubuntu3.7 server-side, HTML-embedded scripting language (Apache 2 module)

root@misbehavingserver:~# dpkg -l | grep apache
ii apache2 2.2.22-1ubuntu1.4 Apache HTTP Server metapackage
ii apache2-mpm-worker 2.2.22-1ubuntu1.4 Apache HTTP Server - high speed threaded model
ii apache2-utils 2.2.22-1ubuntu1.4 utility programs for webservers
ii apache2.2-bin 2.2.22-1ubuntu1.4 Apache HTTP Server common binary files
ii apache2.2-common 2.2.22-1ubuntu1.4 Apache HTTP Server common files
ii libapache-dbi-perl 1.11-1 interface connecting apache server to database via perl's DBI
ii libapache2-mod-perl2 2.0.5-5ubuntu1 Integration of perl with the Apache2 web server
ii libapache2-reload-perl 0.11-2 module for reloading Perl modules when changed on disk

[Ruben Laban (r-laban)]

Really curious as to what could be used as a fix or workaround. Currently it's "breaking" the New Relic agent on quite a few of our servers (it fails to detect the Apache2 processes properly).

[Ruben Laban (r-laban)]

Just did an interesting find:

When starting Apache using the usual "apache2ctl start", the process name is mangled.
When starting Apache using the not-so-usual ". /etc/apache2/envvars && apache2 -k start", the process name isn't mangled.

Investigation continues...

[Ruben Laban (r-laban)]

Turns out apache2 doesn't strip off the path properly:
"/usr/sbin/apache2 -k start" yields "/usr/sbin/apach"
"apache2 -k start" yields "apache2"

[Ruben Laban (r-laban)]

Unfortunately this isn't part of the actual solution. On "good" servers, invoking it through /usr/sbin/apache2 works fine as well.

[Albin Hubert (hubert-albin)]

Experienced this bug also on 12.04. Seems to come from mod-perl.

# cat /etc/issue
Ubuntu 12.04.2 LTS \n \l

# uname -a
Linux --- 3.2.0-51-virtual #77-Ubuntu SMP Wed Jul 24 20:38:32 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

# dpkg-query -l | grep apache
ii apache2-mpm-prefork 2.2.22-1ubuntu1.4 Apache HTTP Server - traditional non-threaded model
ii apache2-utils 2.2.22-1ubuntu1.4 utility programs for webservers
ii apache2.2-bin 2.2.22-1ubuntu1.4 Apache HTTP Server common binary files
ii apache2.2-common 2.2.22-1ubuntu1.4 Apache HTTP Server common files
ii libapache2-mod-auth-cas 1.0.8-3build1 CAS authentication module for Apache2
ii libapache2-mod-perl2 2.0.5-5ubuntu1 Integration of perl with the Apache2 web server
ii libapache2-reload-perl 0.11-2 module for reloading Perl modules when changed on disk

I resolved the bug by removing libapache2-mod-perl2. In /proc/<pid>/status the process name is showing again "apache2".

# aptitude remove libapache2-mod-perl2

[Ruben Laban (r-laban)]

I can confirm that removing libapache2-mod-perl2 "fixes" this. Question remains: why does libapache2-mod-perl2 break it in the first place?

[Jonathan Wiltshire (tiger-jmw)]

A patch for this issue is on the Fedora bug tracker:

https://bugzilla.redhat.com/attachment.cgi?id=567625&action=diff

[Robie Basak (racb)]

This patch doesn't appear to have been committed upstream. What's the submission or upstream inclusion status of this, please?

Removing the mpm-itk task as it seems clear that this bug is in mod-perl2. If this is wrong, please speak up.

[Petr (petr-redhat-bugs)]

*** Bug 1233262 has been marked as a duplicate of this bug. ***

[Petr (petr-redhat-bugs)]

The fix was dropped by an accident in mod_perl-2.0.7-12.20130221svn1448242.fc18. We need to put it back.

[Petr (petr-redhat-bugs)]

Well, the patch breaks current mod_perl tests. I will have to look why is that a problem:

APACHE_TEST_APXS= APACHE_TEST_GROUP= APACHE_TEST_HTTPD= APACHE_TEST_PORT= APACHE_TEST_USER= \
"/usr/bin/perl" -Iblib/arch -Iblib/lib \
t/TEST -bugreport -verbose=1
/usr/sbin/httpd -d /home/test/fedora/mod_perl/mod_perl-2.0.10/t -f /home/test/fedora/mod_perl/mod_perl-2.0.10/t/conf/httpd.conf -D APACHE2 -D APACHE2_4 -D PERL_USEITHREADS
using Apache/2.4.27 (event MPM)

waiting 300 seconds for server to start: .[Wed Jul 26 13:33:35.181948 2017] [env:warn] [pid 18091:tid 139855501953280] AH01506: PassEnv variable LD_LIBRARY_PATH was undefined
[Wed Jul 26 13:33:35.205891 2017] [perl:info] [pid 18091:tid 139855501953280] 6 Apache2:: modules loaded
[Wed Jul 26 13:33:35.205911 2017] [perl:info] [pid 18091:tid 139855501953280] 0 APR:: modules loaded
[Wed Jul 26 13:33:35.205947 2017] [perl:info] [pid 18091:tid 139855501953280] base server + 28 vhosts ready to run tests
[Wed Jul 26 13:33:35.351835 2017] [perl:error] [pid 18091:tid 139855501953280] $s->add_config() has failed: Cannot find current script 'httpd' at /usr/share/perl5/FindBin.pm line 166.\nBEGIN failed--compilation aborted at /usr/share/perl5/FindBin.pm line 166.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestSmoke.pm line 31.\nBEGIN failed--compilation aborted at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestSmoke.pm line 31.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestMM.pm line 24.\nBEGIN failed--compilation aborted at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestMM.pm line 24.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestRun.pm line 22.\nBEGIN failed--compilation aborted at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestRun.pm line 22.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestServer.pm line 26.\nBEGIN failed--compilation aborted at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestServer.pm line 26.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestConfig.pm line 56.\nBEGIN failed--compilation aborted at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestConfig.pm line 56.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/Test.pm line 23.\nBEGIN failed--compilation aborted at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/Test.pm line 23.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestHandler.pm line 21.\nBEGIN failed--compilation aborted at /home/test/fedora/mod_perl/mod_perl-2.0.10/Apache-Test/lib/Apache/TestHandler.pm line 21.\nCompilation failed in require at (eval 3) line 1.\nCompilation failed in require at /home/test/fedora/mod_perl/mod_perl-2.0.10/t/conf/modperl_startup.pl line 18.\n\t...propagated at /home/test/fedora/mod_perl/mod_perl-2.0.10/t/conf/modperl_startup.pl l...

[Fedora (fedora-redhat-bugs)]

Fedora 24 changed to end-of-life (EOL) status on 2017-08-08. Fedora 24 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.