Segfault in dlerror.c 160 on call to dlopen

[Michael Owens]

DistroRelease: Ubuntu 11.10
Package: libc6 2.13-20ubuntu5
Uname: 3.0.0-13-generic #22-Ubuntu SMP Wed Nov 2 13:27:26 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
Architecture: amd64, i386 (this issue exists on both architectures)

I've encountered a segfault in dlerrror.c on XUbuntu 11.10 64-bit desktop in an Apache module, specifically when one of the module's shared libraries calls dlopen(). I have searched launchpad for likely candidates and this problem may be related to the following bugs:

  https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/893605.
  https://bugs.launchpad.net/ubuntu/+source/nspluginwrapper/+bug/762387

The Apache module (which I wrote) embeds a Ruby VM via libruby, which is dynamically linked. The segfault occurs on Ruby startup, where it attempts to load /usr/lib/ruby/1.9.1/x86_64-linux/etc.so via dlopen(). This produces following:

#0 __GI___libc_free (mem=0x20000) at malloc.c:3709
#1 0x00007ffff68b2565 in _dlerror_run (operate=0x7ffff68b1ec0 <dlopen_doit>, args=0x7fffffffa530) at dlerror.c:160
#2 0x00007ffff68b1fc1 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:88
#3 0x00007ffff3f63c65 in dln_load (file=0x7ffff83e3100 "/usr/lib/ruby/1.9.1/x86_64-linux/etc.so") at dln.c:1276
#4 0x00007ffff3f9d29a in load_ext (path=140737356478000) at load.c:554

Ruby calls into dlopen.c as follows:

 /* Load file */
 if ((handle = (void*)dlopen(file, RTLD_LAZY|RTLD_GLOBAL)) == NULL) {
     error = dln_strerror();
     goto failed;
 }

This is the first call that Ruby makes to dlopen(). dlopen() then proceeds into dlerror.c where on line 160 it attempts to free an error string in a result structure that has a bad memory address. The code is as follows:

  if (result->errstring != NULL)
    {
      /* Free the error string from the last failed command. This can
  happen if `dlerror' was not run after an error was found. */
      if (result->malloced)
 free ((char *) result->errstring); <-- dies here.
      result->errstring = NULL;
    }

The value of the result structure is the following:

        $8 = {errcode = -131693408, returned = 32767, malloced = 192, objname = 0x7ffff2149010 "\004", errstring = 0x20000 <Address 0x20000 out of bounds>}

Since this appears to be the first DSO Ruby attempts to load, it lead me to suspect that that it may be related to bug 893605 above.

To test the general call to dlopen() on the etc.so file in particular, I added a line in the Apache module to do so just before it enters Ruby initialization. This invocation of dlopen() worked without a problem. Yet when the Ruby library attempted it next, it caused the above sefault in dlerror.c.

Guessing that that maybe it had something to do with visibility of symbols within Apache, I statically linked libruby to the module in the hopes that dlopen() would work then. But it again produced the same segfault.

Wondering if it could be changes in either the Ruby or Apache code, I compiled and built the same versions of Ruby and Apache from Maverick (which work fine) on 11.10 to rule this out. They produced the exact same segfault in 11.10.

With regard to the Apache module itself as the culprit, I have maintained it for over three years without incident from Hardy up to Maverick on both 32 bit and 64 bit systems. It also compiles and runs on FreeBSD without problems. Further, I have not made any substantial changes to it for the last eight months.

To reproduce the bug, I put the compiled binaries on http://mikeowens.ws/bug/ along with a short README file explaining how to reproduce the problem. Also, the full source code for the project is at https://github.com/linterra/r4a. It has a debian folder so it can be built via pbuilder and dpkg-buildpackage.

The full backtrace is attached.

[Michael Owens (michael-owens)]

Backtrace trace per https://wiki.ubuntu.com/Backtrace.

[Michael Owens (michael-owens)]

Confirmed that this issue exists on i386 as well -- Ubuntu 11.10 server.

[Michael Owens (michael-owens)]

Confirmed that this issue exists on 12.04 alpha as well.