libc6-lse does not replace ld.so with LSE enabled version

[Kyle Huey]

When installing libc6-lse on AArch64 (tested on focal and an AWS Graviton 2) the libc, pthreads, etc used by programs on the system are replaced with the LSE enabled versions. This is not, however, true of ld.so, despite an LSE enabled version of ld.so being included in libc6-lse. As far as I can tell this is because the other libraries are all substituted through some dynamic linker magic (that I don't understand), but ld.so's location is fixed by the PT_INTERP header compiled into every binary on the system, and libc6-lse doesn't e.g. symlink that path to the new ld.so.

This is a problem for porting rr (https://rr-project.org/) to AArch64. The old-style exclusive load/store atomics are inherently non-deterministic and cannot be handled by rr. And not replacing ld.so with an LSE version means every program contains code that can execute the old-style atomics (in practice only programs with threads actually trigger that codepath in ld.so, but that's still most interesting programs).

[Florian Weimer (fweimer)]

* Kyle Huey:

> This is a problem for porting rr (https://rr-project.org/) to AArch64.
> The old-style exclusive load/store atomics are inherently non-
> deterministic and cannot be handled by rr. And not replacing ld.so with
> an LSE version means every program contains code that can execute the
> old-style atomics (in practice only programs with threads actually
> trigger that codepath in ld.so, but that's still most interesting
> programs).

Would outline atomics solve this rr issue?

Thanks,
Florian

[Kyle Huey (kylehuey)]

If ld.so itself can be built with -moutline-atomics, then yes, it would fix this for us.

[Kyle Huey (kylehuey)]

Apparently -moutline-atomics is on by default in gcc 10.1 so perhaps this problem will just go away over time. Debian seems to be relying on that rather than shipping a separate libc6-lse.