[libomp.so.5] OMPT with LD_PRELOAD not working

[Matheus Castro]

This bug was observed on multiple versions of Ubuntu and clang.

Versions:
  Ubuntu: Ubuntu 18.04.5 LTS, Ubuntu 20.04.1 LTS
  clang: 8.0.0-3~ubuntu18.04.2, 9.0.0-2~ubuntu18.04.2, 10.0.0-4ubuntu1~18.04.2,
         9.0.1-12, 10.0.0-4ubuntu1

When trying to use OMPT (https://www.openmp.org/spec-html/5.0/openmpch4.html) by defining the corresponding functions in an shared object and loading it with LD_PRELOAD it gives no result. The functions are not called, not even the first function: ompt_start_tool.

I have included a small code to reproduce the bug.

To run it just do:
make
make run

The expected output is (up to execution reordering):

ompt_start_tool was called
Hello from thread 0 of 4!
Hello from thread 3 of 4!
Hello from thread 2 of 4!
Hello from thread 1 of 4!

While Ubuntu outputs (up to execution reordering):

Hello from thread 0 of 4!
Hello from thread 3 of 4!
Hello from thread 2 of 4!
Hello from thread 1 of 4!

It does not call ompt_start_tool.

It works as expected on Fedora 32 with clang version 10.0.1-2. It also works as expected on Debian GNU/Linux bullseye/sid with clang version 9.0.1-14.

The problem lies in libomp.so.5. When replacing it by the one from a working Debian version the program runs as expected.

[Matheus Castro (matheuscastro)]

The problem lies in libomp.so.5. When replacing it by the one from a working Debian version the program runs as expected.

[Jonas Hahnfeld (hahnjo)]

I can confirm this. I was actually involved in implementing OMPT support in the LLVM OpenMP runtime and the startup works by overwriting the weak symbol ompt_start_tool via LD_PRELOAD. The symbol is there, as seen with
 $ objdump -T /usr/lib/x86_64-linux-gnu/libomp.so.5 | grep ompt
00000000000aa4b0 w DF .text 0000000000000036 VERSION ompt_start_tool

but there is no relocation:
 $ readelf --relocs /usr/lib/x86_64-linux-gnu/libomp.so.5 | grep ompt
[no output]

This might actually be the same problem as https://bugs.launchpad.net/ubuntu/+source/llvm-defaults/+bug/1964487 that I just reported.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Joachim (jprotze)]

This issue should be caused by applying `-Wl,-Bsymbolic-functions` as a linker flag when building the Ubuntu package for libomp.so. This linker flag strictly binds all function calls to function implementation within the same compilation unit and effectively disables LD_PRELOAD-ing of the ompt_start_tool function. I wonder how the MPI libraries are built, so that LD_PRELOAD-ing MPI tool libraries works.

In any case, using `OMP_TOOL_LIBRARIES=$LD_PRELOAD` should be the work-around suggested by the OpenMP standard. Using this environmental variable, LD_PRELOAD is only necessary, if your library wants to overload other API (such as MPI) at dynamic link-time. For OMPT, it is sufficient to let the OpenMP runtime load the tool.

[Lukas Märdian (slyon)]

seems related to bug #2016471