dlopen crashed with Segmentation Fault

[Sankar Tanguturi]

In one of our applications, we use dlopen to open a bunch of library (.so) files in the following order:

1. zlib 1.2.8 version (not the one bundled in Ubuntu 17.10)
2. gtk-x11-2.0.so.0 bundled in the system
3. libSM.so.6 bundled in the system

The dlopen api crashes with 'Segmentation fault' when opening libSM.so.6. You can reproduce the issue using the following simple program.

Sample C program:
=================
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>

int
main(int argc, char **argv)
{
  int i;
  for (i = 1; i < argc; i++) {
   void *handle;
   printf("Opening %s\n", argv[i]);
   handle = dlopen(argv[i], RTLD_LAZY | RTLD_LOCAL);
    if (!handle) {
        printf("Error: %s\n", dlerror());
    }
  }
  exit(EXIT_SUCCESS);
}

Compile and run as follows:
$ ./a.out /path/to/libz/versio/1.2.8 /usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0 libSM.so.6

Note:
- We encountered this issue only in Ubuntu 17.10. In ubuntu 17.04, we didn't have any issue. May be this is an issue with the latest glibc. Not sure.

Investigation done:
- dlopen initially passes while opening zlib version.
- dlopen fails to open libgtk-x11-2.0.so.0 with the following error:

Error: /path/to/zlib/1.2.8/libz.so.1: version `ZLIB_1.2.9' not found (required by /usr/lib/x86_64-linux-gnu/libpng16.so.16)

And then dlopen for libSM.so.6 core dumps.

I understand that the zlib 1.2.8 doesn't provide the necessary required version and hence dlopen for gtk failed. But why does dlopen crash for libSM.so.6?

- I executed valgrind command and attaching the output.

Please let me know if any other information is required.

Thanks
Sankar.

[Sankar Tanguturi (sankaraditya)]

I set LD_DEBUG=all and executed the application. Attaching the output just for a reference.

[Sankar Tanguturi (sankaraditya)]

gdb stacktrace is as follows:

(gdb) where
#0 0x0000000000001510 in ?? ()
#1 0x00007ffff7de5a6a in call_init (l=<optimized out>, argc=argc@entry=4, argv=argv@entry=0x7fffffffe4d8, env=env@entry=0x7fffffffe500) at dl-init.c:72
#2 0x00007ffff7de5b7b in call_init (env=0x7fffffffe500, argv=0x7fffffffe4d8, argc=4, l=<optimized out>) at dl-init.c:30
#3 _dl_init (main_map=main_map@entry=0x602350, argc=4, argv=0x7fffffffe4d8, env=0x7fffffffe500) at dl-init.c:120
#4 0x00007ffff7deab86 in dl_open_worker (a=a@entry=0x7fffffffe180) at dl-open.c:575
#5 0x00007ffff7741d64 in __GI__dl_catch_error (objname=0x7fffffffe170, errstring=0x7fffffffe178, mallocedp=0x7fffffffe16f, operate=0x7ffff7dea7a0 <dl_open_worker>,
    args=0x7fffffffe180) at dl-error-skeleton.c:198
#6 0x00007ffff7dea0d9 in _dl_open (file=0x7fffffffe775 "libSM.so.6", mode=-2147483647, caller_dlopen=0x40065e <main+90>, nsid=-2, argc=<optimized out>,
    argv=<optimized out>, env=0x7fffffffe500) at dl-open.c:660
#7 0x00007ffff7bd1ff6 in dlopen_doit (a=a@entry=0x7fffffffe390) at dlopen.c:66
#8 0x00007ffff7741d64 in __GI__dl_catch_error (objname=0x601680, errstring=0x601688, mallocedp=0x601678, operate=0x7ffff7bd1fa0 <dlopen_doit>, args=0x7fffffffe390)
    at dl-error-skeleton.c:198
#9 0x00007ffff7bd2759 in _dlerror_run (operate=operate@entry=0x7ffff7bd1fa0 <dlopen_doit>, args=args@entry=0x7fffffffe390) at dlerror.c:163
#10 0x00007ffff7bd2092 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#11 0x000000000040065e in main ()