cerr crash in dlopen'ed c++ shared object, if linked to some libs

[Václav Šmilauer]

Binary package hint: gcc-4.4

I have main in c, which dlopens a c++ shared object. Upon calling a function (obtained with dlsym) in this c++ lib which uses cerr, there is segmentation fault. It happens only if the shared object links to some other libraries (like gomp or GL, even if unused). It doesn't happen if the main program is compiled with g++ however.

Here are sources (I will attach those as well) and below is backtrace from the debug version of libstdc++.

I am running fully updated karmic; the same problem occurs at an updated hardy and Debian lenny (I don't have access to other systems).

This is minimal testcase, I have this problem when importing c++ (boost::python) module into python.

=== main.c ===
#include<dlfcn.h>
int main(void){
 void* handle=dlopen("./libfoo.so",RTLD_NOW);
 void(*foo)()=(void(*)())dlsym(handle,"foo");
 foo();
 return 0;
}

=== foo.cc ===
#include<iostream>
extern "C" {
 void foo() { std::cerr<<"foo"<<std::endl; }
}

=== regular run ===
$ g++ foo.cc -o libfoo.so -fPIC -rdynamic -shared
$ gcc main.c -o main -ldl
$ ./main
foo

=== link libfoo.so to another lib ===
$ g++ foo.cc -o libfoo.so -fPIC -rdynamic -shared -lgomp # might be GL, perhaps others as well
$ gcc main.c -o main -ldl
$ ./main
foozsh: segmentation fault (core dumped) ./main

=== compile main with g++, still link with gomp ===
$ g++ foo.cc -o libfoo.so -fPIC -rdynamic -shared -lgomp
$ g++ main.c -o main -ldl
$ ./main
foo

=== gdb bt from the crasher ===
$ g++ foo.cc -o libfoo.so -fPIC -rdynamic -shared -lgomp -g
$ gcc main.c -o main -ldl -g
$ LD_LIBRARY_PATH=/usr/lib/debug ./main
foozsh: segmentation fault (core dumped) LD_LIBRARY_PATH=/usr/lib/debug ./main
$ gdb main core

[ ... ]

(gdb) bt
#0 0x00007f81582c9e49 in std::uncaught_exception () at ../../../../src/libstdc++-v3/libsupc++/eh_catch.cc:136
#1 0x00007f8158294e04 in ~sentry (this=0x7fff3c47bbc0, __in_chrg=<value optimized out>) at /build/buildd/gcc-4.4-4.4.1/build/x86_64-linux-gnu/libstdc++-v3/include/ostream:408
#2 0x00007f8158295445 in std::__ostream_insert<char, std::char_traits<char> > (__out=..., __s=0x7f81590cf3be "foo", __n=3)
    at /build/buildd/gcc-4.4-4.4.1/build/x86_64-linux-gnu/libstdc++-v3/include/bits/ostream_insert.h:110
#3 0x00007f815829502e in std::operator<< <std::char_traits<char> > (__out=..., __s=0x7f81590cf3be "foo")
    at /build/buildd/gcc-4.4-4.4.1/build/x86_64-linux-gnu/libstdc++-v3/include/ostream:510
#4 0x00007f81590cf2e6 in foo () at foo.cc:3
#5 0x00000000004003a2 in main () at main.c:5

ProblemType: Bug
Architecture: amd64
Date: Tue Dec 1 09:56:47 2009
DistroRelease: Ubuntu 9.10
NonfreeKernelModules: fglrx
Package: libstdc++6 4.4.1-4ubuntu8
ProcEnviron:
 PATH=(custom, user)
 LANG=cs_CZ.UTF-8
 SHELL=/bin/zsh
ProcVersionSignature: Ubuntu 2.6.31-16.51~pre2-generic
SourcePackage: gcc-4.4
Uname: Linux 2.6.31-16-generic x86_64
XsessionErrors:
 (gnome-settings-daemon:1984): GLib-CRITICAL **: g_propagate_error: assertion `src != NULL' failed
 (polkit-gnome-authentication-agent-1:2557): GLib-CRITICAL **: g_once_init_leave: assertion `initialization_value != 0' failed
 (nautilus:2412): Eel-CRITICAL **: eel_preferences_get_boolean: assertion `preferences_is_initialized ()' failed
 (gnome-panel:2261): Gdk-WARNING **: /build/buildd/gtk+2.0-2.18.3/gdk/x11/gdkdrawable-x11.c:952 drawable is not a pixmap or window

[RedBrain (herron-philip)]

this is a reasonable error due to the fact you haven't provided in the C++ shared object the necessary symbol Demangling. As well as your maybe calling the dlopen'd function incorrectly since dlsym returns a function pointer which you call as instead of a real function. http://www.dwheeler.com/program-library/Program-Library-HOWTO/dl-libraries.html

http://publib.boulder.ibm.com/infocenter/comphelp/v8v101/index.jsp?topic=/com.ibm.xlcpp8a.doc/proguide/ref/rkcxxflt.htm

http://developers.sun.com/solaris/articles/mixing.html

[Václav Šmilauer (eudoxos)]

Sorry, but I think your comment is irrelevant: the function is linked with extern "C", hence name demangling is not necessary. Besides, the function is called properly, which is manifested by the fact that it does what it should if libfoo.so is not linked with a c++ lib.

I additionaly discovered that if I do

  dlopen("/usr/lib/libstdc++.so.6",RTLD_LAZY);

prior to dlopening the other library, no error occurs (and it does without that, as originally reported):

 == main.c ==
 #include<dlfcn.h>
 int main(void){
  dlopen("/usr/lib/libstdc++.so.6",RTLD_LAZY);
  void* handle=dlopen("./libfoo.so",RTLD_NOW);
  void(*foo)()=(void(*)())dlsym(handle,"foo");
  foo();
  return 0;
 }

I think there is an issue with c++ initialization code not being run properly, unless the stdc++ is opened directly (rather than by ld.so at runtime), but I don't know details.

[Václav Šmilauer (eudoxos)]

I forgot to mention that running

 LD_PRELOAD=/usr/lib/libstdc++.so.6 ./main

makes that work as well.

[prokher (prokher)]

I thing

LD_PRELOAD="/usr/lib/libgomp.so.1"

may help. Me and my colleagues feel that it is somehow connected to thread pool management. It looks like If we preload libgomp then thread pool management works on the level of main module, otherwise it works on the dll level. Thus dlclose works before gomp finalize its pool of threads. If we put sleep before dlclose - it works fine. Actually it is only ideas without any basement...

[Václav Šmilauer (eudoxos)]

After trying this with gcc 4.4, 4.5, 4.6, I am no longer able to reproduce this bug, hence marking as Fix Released.