dynamic library inconsistencies with OpenGL/C++

[swenger]

While using the nvidia-319 driver, the following C++ program won't run:

$ cat > test.cpp << EOF
#include <string>
#include <GL/gl.h>
int main(int argc, char *argv[]) {
 std::string s;
 glEnable(GL_LIGHT0);
}
EOF

Instead, it fails with the following error message:
$ g++ test.cpp -lGL
$ ./a.out
Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!

Using the xserver-xorg-video-nouveau driver, the problem does not occur. Also, when std::string is not used, the program runs without problems. Using OpenGL from pure C programs is also not a problem. The problem can be reproduced with both g++-4.6 and g++-4.8.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: nvidia-319 319.32-0ubuntu7
ProcVersionSignature: Ubuntu 3.11.0-12.19-generic 3.11.3
Uname: Linux 3.11.0-12-generic x86_64
NonfreeKernelModules: openafs nvidia
ApportVersion: 2.12.5-0ubuntu2.1
Architecture: amd64
Date: Wed Nov 6 18:25:48 2013
InstallationDate: Installed on 2013-11-01 (5 days ago)
InstallationMedia: Ubuntu-GNOME 13.10 "Saucy Salamander" - Release amd64 (20131017)
MarkForUpload: True
SourcePackage: nvidia-graphics-drivers-319
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.modprobe.d.nvidia.319.hybrid.conf: [deleted]

[swenger (s-wenger)]

The problem can also be reproduced using the nvidia-304 and nvidia-319-updates drivers (I haven't tried nvidia-304-updates).

[Kate Alhola (kate-alhola)]

The issue is that for some reason it gets libpthread.so.0 referenced but not listed as in a.out NEDED section

1. Conpile std::string s; commented out, it works, do objdump -pR a.out
Dynamic Section:
  NEEDED libGL.so.1
  NEEDED libc.so.6
Version References:
  required from libc.so.6:
    0x09691a75 0x00 02 GLIBC_2.2.5

2. Do same thing the std::string line active , notice that libpthreas.so.0 is not NEEDED but it is in version referennces and there is __pthread_key_create symbol and you got the bug
Dynamic Section:
  NEEDED libGL.so.1
  NEEDED libstdc++.so.6
  NEEDED libgcc_s.so.1
  NEEDED libc.so.6
Version References:
  required from libgcc_s.so.1:
    0x0b792650 0x00 06 GCC_3.0
  required from libpthread.so.0:
    0x09691a75 0x00 04 GLIBC_2.2.5
  required from libc.so.6:
    0x09691a75 0x00 03 GLIBC_2.2.5
  required from libstdc++.so.6:
    0x056bafd3 0x00 05 CXXABI_1.3
    0x08922974 0x00 02 GLIBCXX_3.4

DYNAMIC RELOCATION RECORDS
OFFSET TYPE VALUE
0000000000600ff8 R_X86_64_GLOB_DAT __gmon_start__
0000000000601018 R_X86_64_JUMP_SLOT _ZNSsC1Ev
0000000000601020 R_X86_64_JUMP_SLOT glEnable
0000000000601028 R_X86_64_JUMP_SLOT __gmon_start__
0000000000601030 R_X86_64_JUMP_SLOT __libc_start_main
0000000000601038 R_X86_64_JUMP_SLOT _ZNSsD1Ev
0000000000601040 R_X86_64_JUMP_SLOT __pthread_key_create

3. Add call to pthread , it woirks again
 #include <pthread.h>
#include <string>
#include <GL/gl.h>
int main(int argc, char *argv[]) {
  std::string s;
 glEnable(GL_LIGHT0);
 int i=pthread_getconcurrency();
}

And objdump looks now
Dynamic Section:
  NEEDED libGL.so.1
  NEEDED libstdc++.so.6
  NEEDED libgcc_s.so.1
  NEEDED libpthread.so.0
  NEEDED libc.so.6

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Kyle Nitzsche (knitzsche)]

I have the same problem on trusty (pre-release) and with nvidia-331-updates:
$ cat > test.cpp << EOF
> #include <string>
> #include <GL/gl.h>
> int main(int argc, char *argv[]) {
> std::string s;
> glEnable(GL_LIGHT0);
> }
> EOF
$ g++ test.cpp -lGL
$ ./a.out
Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!
$ apt-cache policy nvidia-331-updates
nvidia-331-updates:
  Installed: 331.20-0ubuntu9
  Candidate: 331.20-0ubuntu9
  Version table:
 *** 331.20-0ubuntu9 0
        500 http://us.archive.ubuntu.com/ubuntu/ trusty/restricted amd64 Packages
        100 /var/lib/dpkg/status
$

[Alberto Milone (albertomilone)]

what does the following command return?
ldd a.out

[Alberto Milone (albertomilone)]

I think you're using NVIDIA's libGL libraries instead of Mesa's.

You can work around the problem by preloading Mesa's libraries, e.g. doing something like:

(on a 64bit x86 system)
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/mesa/libGL.so.1

or (on a 32bit x86 system)
export LD_PRELOAD=/usr/lib/i386-linux-gnu/mesa/libGL.so.1

then recompile, and the program will run with no issues.

Make sure to do that every time you need to run that program.

[Kyle Nitzsche (knitzsche)]

@AlbertoM:
$ ldd ./a.out
Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!

[Alberto Milone (albertomilone)]

@Kyle: my suggestion in comment #7 works here, please give it a try.

[Kyle Nitzsche (knitzsche)]

@Alberto: yes, the ./a.out executes correctly:

$ g++ test.cpp -lGL
$ ./a.out
$

:-)

Noting though that I am not sure how to apply this to my actual use case: running an HTML5 app in the Ubuntu SDK (executable is the ubuntu-html5-app-launcher provided by the ubuntu-html5-container package in the ppa:ubuntu-sdk/ppa)

[Alberto Milone (albertomilone)]

I'm not familiar with the Ubuntu SDK but you can probably execute that and preload the library:

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/mesa/libGL.so.1 ubuntu-html5-app-launcher

Or even better, create a script that makes sure that the correct libGL.so.1 library is used.

[Alberto Milone (albertomilone)]

@Daniel Dadap: is there a reason why this fails with NVIDIA's libGL?

[Kyle Nitzsche (knitzsche)]

@Alberto, it does not seem to work:
$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/mesa/libGL.so.1 ubuntu-html5-app-launcher .
ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/mesa/libGL.so.1' from LD_PRELOAD cannot be preloaded: ignored.
ubuntu-html5-app-launcher: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory

[Alberto Milone (albertomilone)]

I guess that interferes with the Qt libraries used by ubuntu-html5-app-launcher.

Let's wait for Daniel's response.

[Daniel Dadap (ddadap)]

Hi Alberto,

One of our GL driver engineers looked into this a bit, and it actually seems to be a linker regression that is exposed when linking an application against the Mesa libGL, and then running it against the NVIDIA one. He is gathering some more information before filing a bug, but in the mean time, please find attached a test case demonstrating the linker bug, which doesn't involve either the Mesa libGL or the NVIDIA one. See the README in the tarball for details.

When bisecting binutils in an attempt to identify the regression, our engineer found that commit commit b64fb44af4f416fbbbda3de03fcfff61d80c841c ("Also track weak references") actually introduced a link time error when attempting to link the test executable:

ld: /tmp/cc3K1rHK.o: undefined reference to symbol 'pthread_create@@GLIBC_2.2.5'
ld: note: 'pthread_create@@GLIBC_2.2.5' is defined in DSO //lib/x86_64-linux-gnu/libpthread.so.0 so try adding it to the linker command line

That ld error no longer occurs after commit
879707c642925947e156b7ae2169b89f844532cd ("Exclude weak refs when considering whether an --as-needed library is needed"); however, the bug described here (assertion from ld.so) does.

We believe that this Debian bug may be related:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=728529

[Daniel Dadap (ddadap)]

Hi Alberto,

One of our GL driver engineers looked into this a bit, and it actually seems to be a linker regression that is exposed when linking an application against the Mesa libGL, and then running it against the NVIDIA one. He is gathering some more information before filing a bug, but in the mean time, please find attached a test case demonstrating the linker bug, which doesn't involve either the Mesa libGL or the NVIDIA one. See the README in the tarball for details.

When bisecting binutils in an attempt to identify the regression, our engineer found that commit commit b64fb44af4f416fbbbda3de03fcfff61d80c841c ("Also track weak references") actually introduced a link time error when attempting to link the test executable:

ld: /tmp/cc3K1rHK.o: undefined reference to symbol 'pthread_create@@GLIBC_2.2.5'
ld: note: 'pthread_create@@GLIBC_2.2.5' is defined in DSO //lib/x86_64-linux-gnu/libpthread.so.0 so try adding it to the linker command line

That ld error no longer occurs after commit
879707c642925947e156b7ae2169b89f844532cd ("Exclude weak refs when considering whether an --as-needed library is needed"); however, the bug described here (assertion from ld.so) does.

We believe that this Debian bug may be related:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=728529

[Daniel Dadap (ddadap)]

Apologies for the duplicated post: I neglected to include the attachment the first time around, and when I attempted to add it later, it resulted in the text of my previous update being posted again.

[Brian Nguyen (brnguyen)]

Hi Alberto,

I have been investigating this issue, and have filed binutils bug #16452 (http://sourceware.org/bugzilla/show_bug.cgi?id=16452) to track this. This can be worked around by explicitly linking the executable against pthreads, or explicitly specifying the path to NVIDIA's libGL using gcc's -L flag, e.g.:

$ g++ ./test.cpp -pthread -lGL

or

$ g++ ./test.cpp -L/usr/lib/nvidia-319-updates -lGL

[Alberto Milone (albertomilone)]

Thank you Brian and Daniel

[Peter Würtz (pwuertz)]

@Daniel Dadap, @Brian Nguyen
This kind of problem seems to be similar to a very old and persistent bug where PyQt cannot create shaders when using the NVIDIA driver on Ubuntu (bug #941826).
This small script (https://bugs.launchpad.net/ubuntu/+source/python-qt4/+bug/941826/+attachment/3950419/+files/nvidia-python-qt-bug.py) demonstrates the problem and a workaround by "preloading" another opengl module. This really looks related, I don't see how though since the problem is way older and python is definitely linked to pthreads.

[dinamic (dinamic6661)]

i get the same error "Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!" but when running Box2D on ubuntu 14.04 with nvidia 331.38 driver

[etiam (etiam-public)]

I also get 'Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!' when running a OpenGL tutorial build, with nvidia 331.20 driver. The system is Mint 16.

Adding the suggested gcc flags wasn't an effective workaround for me, and the preload option still resulted in another runtime error, so I would like to add that the 'solution' in my case was to exchange the linker.

If I change /usr/bin/ld to refer to /usr/bin/ld.gold instead of as usual /usr/bin/ld.bfd the program builds and runs fine. Does anyone know a way to specify the linker as a command line option by the way? Making changes to /bin might not be a practical option for some people, and I'm not entirely pleased with it myself.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Sam Watkins (w-sam-l)]

celestia-glut is broken in Ubuntu stable with nvidia gl, giving this error. fortunately celestia-gnome does work

[Michael Murphey (z-michael-u)]

In Ubuntu 14.04.1 LTS Trusty Tahr, the nvidia-304.123 driver worked fine for me. I upgraded to Ubuntu 14.10 Utopic Unicorn and now I am experiencing this problem. I have tried nvidia 304, 331 and 340 and got the error with all of them.

[Axel Siebenwirth (asiebenwirth)]

Also effects me on Ubuntu 14.10 with nvidia-331 and when running glmark2:

Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!

[computeristgeek (mailbot-computeristgeek)]

Affected me on codeblocks, used #3 work around and it worked.

[Graham Inggs (ginggs)]

Is this still present in 15.04?

[Graham Inggs (ginggs)]

I am unable to reproduce the problem with Daniel's example from #16 in 15.04.
Reproducible in 14.04 though.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Steve Langasek (vorlon)]

Hello swenger, or anyone else affected,

Accepted binutils into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/binutils/2.24-5ubuntu13 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Paul Kilgo (paulkilgo)]

I have tested the new binutils package on the test program at the top of this ticket. I no longer see the "Inconsistency detected..." message, but I do receive a segfault.

Here is what I did to upgrade:

#> cat /etc/apt/sources.list.d/trusty-proposed.list
deb http://archive.ubuntu.com/ubuntu/ trusty-proposed restricted main multiverse universe
#> apt-get update
#> apt-get install binutils=2.24-5ubuntu13
#> apt-cache policy binutils
binutils:
  Installed: 2.24-5ubuntu13
  Candidate: 2.24-5ubuntu13
  Version table:
 *** 2.24-5ubuntu13 0
        100 /var/lib/dpkg/status

Here is what I did to compile the test program:

$> g++ test.cpp -lGL
$> ./a.out
Segmentation fault (core dumped)

Forcing linking against NVIDIA's libGL.so seems to get rid of the segfault:

$> g++ test.cpp -L /usr/lib/nvidia-331 -lGL
$> ./a.out

Here is a backtrace from gdb.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00007ffff406c291 in init () at dlerror.c:177
#2 0x00007ffff406c6d7 in _dlerror_run (operate=operate@entry=0x7ffff406c130 <dlsym_doit>,
    args=args@entry=0x7fffffffe550) at dlerror.c:129
#3 0x00007ffff406c198 in __dlsym (handle=<optimized out>, name=<optimized out>) at dlsym.c:70
#4 0x00007ffff7b4eb3e in ?? () from /usr/lib/nvidia-331/libGL.so.1
#5 0x00007ffff7b32db4 in ?? () from /usr/lib/nvidia-331/libGL.so.1
#6 0x00007ffff7dea0fd in call_init (l=0x7ffff7ff94c0, argc=argc@entry=1,
    argv=argv@entry=0x7fffffffe698, env=env@entry=0x7fffffffe6a8) at dl-init.c:64
#7 0x00007ffff7dea223 in call_init (env=<optimized out>, argv=<optimized out>,
    argc=<optimized out>, l=<optimized out>) at dl-init.c:36
#8 _dl_init (main_map=0x7ffff7ffe1c8, argc=1, argv=0x7fffffffe698, env=0x7fffffffe6a8)
    at dl-init.c:126
#9 0x00007ffff7ddb30a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#10 0x0000000000000001 in ?? ()
#11 0x00007fffffffe968 in ?? ()
#12 0x0000000000000000 in ?? ()

I am using nvidia-331=331.113-0ubuntu0.0.4, but nvidia-331-updates=331.113-0ubuntu0.0.4 seems to have the same behavior too. If I can provide more information let me know. I'll update the tag too (if I can find where to do it).

[Graham Inggs (ginggs)]

ginggs wrote previously:
> I am unable to reproduce the problem with Daniel's example from #16 in 15.04.
> Reproducible in 14.04 though.

Daniel's example from #16 is now fixed with binutils 2.24-5ubuntu13.

@paulkilgo: the test program from #1 still segfaults in 15.04.

[SuperThin (nhomkcbt)]

I have the same #26, Ubuntu 14.10 Utopic Unicorn, nvidia-331 and nvidia-331-updates

[Matthias Klose (doko)]

I don't agree with the comment in #33, that verification failed for the binutils upload. the binutils induced error is solved, and loading fails later. And apparently, this, or a similar issue is known in other places. I can't see how the new binutils would make this worse.

[Matthias Klose (doko)]

subscribing glibc, looks like an issue in the dynamic loader

[Launchpad Janitor (janitor)]

This bug was fixed in the package binutils - 2.24-5ubuntu13

---------------
binutils (2.24-5ubuntu13) trusty-proposed; urgency=medium

  * SRU LP: #1311866.
  * Update from the 2.24 branch, 20141113.
    - Fix PR gold/16945, properly handle 64-bit GOT relocations on x86_64.
    - Fix PR gold/16900, an issue where first reserved word of GOT is not
      initialized if there is no PLT.
    - gold: Fix handling of __ehdr_start when it cannot be defined.
    - Apply mainline patches for ppc476 workaround.
    - Add binutils test cases for AArch64.
    - Disassembler fix on AArch64.
    - Fix PR ld/17047, crash in the bfd linker with MALLOC_PERTURB.
    - Fix PR ld/17277, ARM32, bogus DT_TEXTREL marker (and R_ARM_NONE)
      for PC-relative cross-section relocs.
    - [AArch64] Cortex-A53 erratum 835769 linker workaround.
  * Remove the aarch64-fix-instruction-mask, applied on the branch.
  * Fix PR gold/15639, -flto and ld.gold on ARM. LP: #1191909.
  * Use 64k for COMMONPAGESIZE on PPC. LP: #1412553.
  * Fix PR ld/16452, PR ld/16457, don't output symbol version definitions
    for non-DT_NEEDED. LP: #1248642.
  * Add powerpc target for ppc64el builds. Closes: #760395. LP: #1433238.
  * binutils-doc: Include all info files. LP: #1410780.
  * Fix PR ld/16715 (ARM), set st_value to zero for undefined symbols.
    LP: #1441961.
 -- Matthias Klose <email address hidden> Tue, 14 Apr 2015 19:52:45 +0200

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for binutils has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Charles Wilkins (cg-chas)]

Comment #33 worked for me for nvidia-352-updates on Ubuntu 14.04.3 LTS.

Specifically, by linking with -L/usr/lib/nvidia-352-updates the following additional dependency was reported by ldd:

libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xb7302000)

with all other dependencies appearing to be the same as when not linking the additional library path.

what did not work was simply linking with -pthread -lpthread.
gdb backtraces with the nvidia-352-updates packages present are similar to the problems described in this thread.

preloading libpthread also worked as described:
LD_PRELOAD=/lib/i386-linux-gnu/libpthread.so.0 ./a.out

Any number of stringstream, ostringstream, string object instantiations in the openGL app using the nvidia proprietary driver led to the segfault as described above.

It looks like the status of this has toggled at least a couple times. Is there still a binutils fix for this issue?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[KeithG (grider-4)]

running 15.04 with the nvidia 304.131(proprietary, tested) and get a similar result:

Inconsistency detected by ld.so: dl-version.c: 224: _dl_check_map_versions: Assertion `needed != ((void *)0)' failed!

The binutils installed is 2.25-5ubuntu7

I get this on another machine with the nvidia 352.63 driver as well. Any updates?