Segmentation fault in libapache2-mod-php5 when calling methods

[Kurt Huwig]

Binary package hint: php5

I have these 2 php files:

----- 8< ---- parta.php
<?php
require_once('partb.php');

$fieldlist[0]='A';
$fieldlist[1]='B';
while (list($x, $fieldbez) = each($fieldlist)) {
        $a = myfunc('a',$$fieldbez);
}
?>
----- 8< ----

----- 8< ---- partb.php
<?php
function myfunc() {
        return 'huhu';
}
?>
----- 8< ----

When calling this, I get

[Wed Jul 07 13:58:07 2010] [notice] child pid 8589 exit signal Segmentation fault (11)

I reduced the code to the bare minimum. Having the function in the same php file works fine, as well as having just one entry in the array or one parameter to the function call.

With slightly different code, I got

[Wed Jul 07 13:06:38 2010] [notice] child pid 2362 exit signal Bus error (7)

I created a core dump and this is the backtrace (2MB packed, available upon request).

#0 0x00007f8fc2810343 in ?? () from /usr/lib/apache2/modules/libphp5.so
#1 0x00007f8fc27ef93d in _zval_dtor_func () from /usr/lib/apache2/modules/libphp5.so
#2 0x00007f8fc27e39bd in _zval_ptr_dtor () from /usr/lib/apache2/modules/libphp5.so
#3 0x00007f8fc27fc642 in ?? () from /usr/lib/apache2/modules/libphp5.so
#4 0x00007f8fc27fc8c8 in zend_hash_graceful_reverse_destroy () from /usr/lib/apache2/modules/libphp5.so
#5 0x00007f8fc27e3f6e in ?? () from /usr/lib/apache2/modules/libphp5.so
#6 0x00007f8fc27f06d2 in ?? () from /usr/lib/apache2/modules/libphp5.so
#7 0x00007f8fc279c385 in php_request_shutdown () from /usr/lib/apache2/modules/libphp5.so
#8 0x00007f8fc287cdc7 in ?? () from /usr/lib/apache2/modules/libphp5.so
#9 0x00007f8fc6640140 in ap_run_handler (r=0x7f8fc7c27098) at /build/buildd/apache2-2.2.14/server/config.c:159
#10 0x00007f8fc6643aa8 in ap_invoke_handler (r=0x7f8fc7c27098) at /build/buildd/apache2-2.2.14/server/config.c:373
#11 0x00007f8fc665149c in ap_internal_redirect (new_uri=<value optimized out>, r=<value optimized out>) at /build/buildd/apache2-2.2.14/modules/http/http_request.c:501
#12 0x00007f8fbfe38c95 in ?? () from /usr/lib/apache2/modules/mod_rewrite.so
#13 0x00007f8fc6640140 in ap_run_handler (r=0x7f8fc7c1dcb8) at /build/buildd/apache2-2.2.14/server/config.c:159
#14 0x00007f8fc6643aa8 in ap_invoke_handler (r=0x7f8fc7c1dcb8) at /build/buildd/apache2-2.2.14/server/config.c:373
#15 0x00007f8fc6651678 in ap_process_request (r=0x7f8fc7c1dcb8) at /build/buildd/apache2-2.2.14/modules/http/http_request.c:282
#16 0x00007f8fc664e528 in ap_process_http_connection (c=0x7f8fc7c17bd8) at /build/buildd/apache2-2.2.14/modules/http/http_core.c:190
#17 0x00007f8fc6647cf8 in ap_run_process_connection (c=0x7f8fc7c17bd8) at /build/buildd/apache2-2.2.14/server/connection.c:43
#18 0x00007f8fc6656037 in child_main (child_num_arg=<value optimized out>) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:662
#19 0x00007f8fc665634a in make_child (s=0x7f8fc7831938, slot=41) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:758
#20 0x00007f8fc665667b in startup_children (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:776
#21 ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:997
#22 0x00007f8fc662c350 in main (argc=3, argv=0x7fff695fc958) at /build/buildd/apache2-2.2.14/server/main.c:742

The system is Ubuntu 10.04 Lucid with all updates installed.

ii apache2 2.2.14-5ubuntu8 Apache HTTP Server metapackage
ii apache2-mpm-prefork 2.2.14-5ubuntu8 Apache HTTP Server - traditional non-threade
ii apache2-utils 2.2.14-5ubuntu8 utility programs for webservers
ii apache2.2-bin 2.2.14-5ubuntu8 Apache HTTP Server common binary files
ii apache2.2-common 2.2.14-5ubuntu8 Apache HTTP Server common files
ii libapache2-mod-php5 5.3.2-1ubuntu4.2 server-side, HTML-embedded scripting languag

ii libapache2-mod-php5 5.3.2-1ubuntu4.2 server-side, HTML-embedded scripting languag
ii php5-common 5.3.2-1ubuntu4.2 Common files for packages built from the php
ii php5-gd 5.3.2-1ubuntu4.2 GD module for php5
ii php5-mcrypt 5.3.2-0ubuntu1 MCrypt module for php5
ii php5-mysql 5.3.2-1ubuntu4.2 MySQL module for php5
ii php5-xcache 1.3.0-5ubuntu1 Fast, stable PHP opcode cacher

When I deinstall xcache, the crash still happens. I tried to reproduce it on another machine, but without success. It just crashes on one machine.

[Bilal Akhtar (bilalakhtar)]

Please see if this bug is the same as bug #531150 .

[Kurt Huwig (k-huwig)]

I don't think it is the same as bug #531150:

1. the page is NOT in ~/public_html/ but in a configured virtual host

2. short_tags = on does not help; in php.ini the option

short_open_tag = On

was already active. I added "short_tags" but it still crashes.

Frankly, I don't quite understand some of the comments of #531150.

[Scott Moser (smoser)]

I'm not able to reproduce this on maverick i386 system I tested on. That said, the bug opener stated that he could only recreate on one system. To attempt recreate I tried on a fresh ec2 instance:
$ sudo apt-get install libapache2-mod-php5
$ vi /var/www/parta.php /var/www/partb.php
# as above, but adding an 'echo "Done!\n";' at the end.

$ wget http://localhost/parta.php -O - -q
Done!

If we can't recreate this then it will be difficult to fix.

Could you please try getting a crash with debugging symbols available ?
https://wiki.ubuntu.com/DebuggingProgramCrash

[Kurt Huwig (k-huwig)]

The system is AMD64 (I thought this was clear from the backtrace output).

uname -a
Linux 2.6.32-24-server #38-Ubuntu SMP Mon Jul 5 10:29:32 UTC 2010 x86_64 GNU/Linux

I followed the instructions from the wiki, but there seems to be a missing package:

apt-get install libapache2-mod-php5-dbgsym
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
  libapache2-mod-php5-dbgsym: Depends: libapache2-mod-php5 (= 5.3.2-1ubuntu4) but 5.3.2-1ubuntu4.2 is to be installed
E: Broken packages

[Kurt Huwig (k-huwig)]

I forced the installation of the debug symbols, but they did not match the binary:

warning: the debug information found in "/usr/lib/debug//usr/lib/apache2/modules/libphp5.so" does not match "/usr/lib/apache2/modules/libphp5.so" (CRC mismatch).

warning: the debug information found in "/usr/lib/debug/usr/lib/apache2/modules/libphp5.so" does not match "/usr/lib/apache2/modules/libphp5.so" (CRC mismatch).

[Michael D (macd81)]

Why are there no debug symbols for the current version of libapache2-mod-php5?

[Kurt Huwig (k-huwig)]

I pinned the release to 5.3.2-1ubuntu4 and downgraded. This is the core-dump:

Core was generated by `/usr/sbin/apache2 -k start'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f3168507343 in _zend_mm_free_canary_int (heap=0x7f316e2b8a30, p=0xe781bdf254723a17) at /build/buildd/php5-5.3.2/Zend/zend_alloc_canary.c:2090
2090 /build/buildd/php5-5.3.2/Zend/zend_alloc_canary.c: No such file or directory.
        in /build/buildd/php5-5.3.2/Zend/zend_alloc_canary.c
(gdb) bt
#0 0x00007f3168507343 in _zend_mm_free_canary_int (heap=0x7f316e2b8a30, p=0xe781bdf254723a17) at /build/buildd/php5-5.3.2/Zend/zend_alloc_canary.c:2090
#1 0x00007f31684e693d in _zval_dtor_func (zvalue=0x7f316e4f72a8) at /build/buildd/php5-5.3.2/Zend/zend_variables.c:36
#2 0x00007f31684da9bd in _zval_ptr_dtor (zval_ptr=0x7f316e4f9b68) at /build/buildd/php5-5.3.2/Zend/zend_variables.h:35
#3 0x00007f31684f3642 in zend_hash_apply_deleter (ht=0x7f3168bdea48, p=0x7f316e4f9b50) at /build/buildd/php5-5.3.2/Zend/zend_hash.c:813
#4 0x00007f31684f38c8 in zend_hash_graceful_reverse_destroy (ht=0x7f3168bdea48) at /build/buildd/php5-5.3.2/Zend/zend_hash.c:848
#5 0x00007f31684daf6e in shutdown_executor () at /build/buildd/php5-5.3.2/Zend/zend_execute_API.c:256
#6 0x00007f31684e76d2 in zend_deactivate () at /build/buildd/php5-5.3.2/Zend/zend.c:962
#7 0x00007f3168493385 in php_request_shutdown (dummy=0x7f316e2b8a30) at /build/buildd/php5-5.3.2/main/main.c:1649
#8 0x00007f3168573dc7 in php_handler (r=0x7f3168573dc7) at /build/buildd/php5-5.3.2/sapi/apache2handler/sapi_apache2.c:512
#9 0x00007f316c337140 in ap_run_handler (r=0x7f316e536a68) at /build/buildd/apache2-2.2.14/server/config.c:159
#10 0x00007f316c33aaa8 in ap_invoke_handler (r=0x7f316e536a68) at /build/buildd/apache2-2.2.14/server/config.c:373
#11 0x00007f316c34849c in ap_internal_redirect (new_uri=<value optimized out>, r=<value optimized out>) at /build/buildd/apache2-2.2.14/modules/http/http_request.c:501
#12 0x00007f3165b2fc95 in ?? () from /usr/lib/apache2/modules/mod_rewrite.so
#13 0x00007f316c337140 in ap_run_handler (r=0x7f316e53ca38) at /build/buildd/apache2-2.2.14/server/config.c:159
#14 0x00007f316c33aaa8 in ap_invoke_handler (r=0x7f316e53ca38) at /build/buildd/apache2-2.2.14/server/config.c:373
#15 0x00007f316c348678 in ap_process_request (r=0x7f316e53ca38) at /build/buildd/apache2-2.2.14/modules/http/http_request.c:282
#16 0x00007f316c345528 in ap_process_http_connection (c=0x7f316e525728) at /build/buildd/apache2-2.2.14/modules/http/http_core.c:190
#17 0x00007f316c33ecf8 in ap_run_process_connection (c=0x7f316e525728) at /build/buildd/apache2-2.2.14/server/connection.c:43
#18 0x00007f316c34d037 in child_main (child_num_arg=<value optimized out>) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:662
#19 0x00007f316c34d34a in make_child (s=0x7f316e126938, slot=31) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:758
#20 0x00007f316c34d67b in startup_children (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:776
#21 ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized...

[Kurt Huwig (k-huwig)]

bug #589953 has a segfault in the same line of _zend_mm_free_canary_int but with a different stacktrace.

[Kurt Huwig (k-huwig)]

bug #607646 shows a different backtrace and a patch related to pgsql, but the pgsql package is not installed on my machine.

[Clint Byrum (clint-fewbar)]

Kurt, thanks for taking the time to provide all of these details.

Without any other users able to reproduce or having reported this, its difficult to look at this bug. I do believe with the stack trace an extremely skilled engineer can probably have a chance at creating a reproducible test case. Because of that I've marked the bug as Triaged.

However, this having affected only your one box, and with you being unable to reproduce, I've marked the importance as Low.

Perhaps if more users run into this issue, or you find a way to reproduce it, it would be prudent then to raise it to Medium importance.

[Kurt Huwig (k-huwig)]

The bug happens in this line:

        if (--(*op_array->refcount)>0) {
                return;
        }

of destroy_op_array, when the code tries to read the refcount, according to the disassembler:

Dump of assembler code for function destroy_op_array:
   0x00007f6c321e2080 <+0>: push %r13
   0x00007f6c321e2082 <+2>: push %r12
   0x00007f6c321e2084 <+4>: push %rbp
   0x00007f6c321e2085 <+5>: push %rbx
   0x00007f6c321e2086 <+6>: mov %rdi,%rbx
   0x00007f6c321e2089 <+9>: sub $0x8,%rsp
   0x00007f6c321e208d <+13>: mov 0x48(%rdi),%rbp
   0x00007f6c321e2091 <+17>: mov 0x50(%rdi),%r13d
   0x00007f6c321e2095 <+21>: mov 0x90(%rdi),%rdi
   0x00007f6c321e209c <+28>: test %rdi,%rdi
   0x00007f6c321e209f <+31>: je 0x7f6c321e20b2 <destroy_op_array+50>
   0x00007f6c321e20a1 <+33>: callq 0x7f6c321f6940 <zend_hash_destroy>
   0x00007f6c321e20a6 <+38>: mov 0x90(%rbx),%rdi
   0x00007f6c321e20ad <+45>: callq 0x7f6c321ccc80 <_efree>
   0x00007f6c321e20b2 <+50>: mov 0x40(%rbx),%rdx
=> 0x00007f6c321e20b6 <+54>: mov (%rdx),%eax
   0x00007f6c321e20b8 <+56>: sub $0x1,%eax
   0x00007f6c321e20bb <+59>: test %eax,%eax
   0x00007f6c321e20bd <+61>: mov %eax,(%rdx)
   0x00007f6c321e20bf <+63>: je 0x7f6c321e20d0 <destroy_op_array+80>

The pointer look bad, as rdx points to 0x7f6c28390138:

(gdb) info registers
rax 0x7f6c321e2270 140102674031216
rbx 0x7f6c36d2a7b0 140102752970672
rcx 0x201f0 131568
rdx 0x7f6c28390138 140102508020024
rsi 0x0 0
rdi 0x0 0
rbp 0x7f6c28390140 0x7f6c28390140
rsp 0x7fff2f1ae250 0x7fff2f1ae250
r8 0x7f6c36583290 140102744945296
r9 0xc08e69804da8f602 -4571600571954825726
r10 0x7f6c3691aad0 140102748711632
r11 0x7f6c365f2c20 140102745402400
r12 0x7f6c36d6bc00 140102753238016
r13 0xd42 3394
r14 0x7fff2f1aebf8 140733983681528
r15 0x113e0 70624
rip 0x7f6c321e20b6 0x7f6c321e20b6 <destroy_op_array+54>
eflags 0x246 [ PF ZF IF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0

and this location is not memory mapped:

(gdb) x 0x7f6c28390138
0x7f6c28390138: Cannot access memory at address 0x7f6c28390138

The bug does not happen with the php-cli, but with apache-php.