openmpi link failure with ld --as-needed

[Matthias Klose]

$ cat test.F
      program main
       include 'mpif.h'
       integer ierr
       call mpi_init(ierr)
      end
$ mpif90 test.F -Wl,--as-needed -Wall -Wno-unused-variable -g -I/usr/lib/openmpi/include -I/usr/lib/openmpi/lib -L/usr/lib/openmpi/lib -L/usr/lib/gcc/i686-linux-gnu/4.5.2 -ldl -lmpi -lopen-rte -lopen-pal -lnsl -lutil -lgcc_s -lpthread -ldl /usr/bin/ld: /tmp/cccq3IWF.o(.debug_info+0x48d): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_'
/usr/bin/ld: /tmp/cccq3IWF.o(.debug_info+0x4a7): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_'
/usr/bin/ld: /tmp/cccq3IWF.o(.debug_info+0x511): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_errcodes_ignore_'
/usr/bin/ld: /tmp/cccq3IWF.o(.debug_info+0x52b): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_errcodes_ignore_'

works with --no-as-needed.

[Matthias Klose (doko)]

libopenmpi-dev needed to build

[Evan Broder (broder)]

It seems that this was fixed in 1.4.3-2.1

[Jean-Louis Dupond (dupondje)]

Seems not fixed yet:
root@artemis:/# mpif90 test.F -Wl,--as-needed -Wall -Wno-unused-variable -g -I/usr/lib/openmpi/include -I/usr/lib/openmpi/lib -L/usr/lib/openmpi/lib -L/usr/lib/gcc/i686-linux-gnu/4.5.2 -ldl -lmpi -lopen-rte -lopen-pal -lnsl -lutil -lgcc_s -lpthread -ldl
/usr/bin/ld: /tmp/cc3UWpD5.o(.debug_info+0x491): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_'
/usr/bin/ld: /tmp/cc3UWpD5.o(.debug_info+0x4ab): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_'
/usr/bin/ld: /tmp/cc3UWpD5.o(.debug_info+0x521): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_errcodes_ignore_'
/usr/bin/ld: /tmp/cc3UWpD5.o(.debug_info+0x53b): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_errcodes_ignore_'

[Evan Broder (broder)]

Misread the bug. This isn't actually fixed.

Apologies for the spam.

[Matthias Klose (doko)]

fails on the 2.22 branch and trunk, works with ld.gold on both the branch and the trunk.

$ cat mpitest.f90
program main
include 'mpif.h'
integer ierr
call mpi_init(ierr)
end
$ mpif90 -Wl,--as-needed -g mpitest.f90
/usr/bin/ld: /tmp/cc9rtjMR.o(.debug_info+0x493): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_'
/usr/bin/ld: /tmp/cc9rtjMR.o(.debug_info+0x4ad): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_'
/usr/bin/ld: /tmp/cc9rtjMR.o(.debug_info+0x523): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_errcodes_ignore_'
/usr/bin/ld: /tmp/cc9rtjMR.o(.debug_info+0x53d): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_errcodes_ignore_'

The undefined references to `__libc_csu_(init|fini)' are an artefact in the self-contained testcase.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Matthias Klose (doko)]

self-contained test case at
http://people.debian.org/~doko/tmp/mpi.tar.xz

[Hjl-tools (hjl-tools)]

Please provide the source of mpitest.o as well as command line
used to create mpitest.o.

[Hjl-tools (hjl-tools)]

A small testcase:

hjl@gnu-6 pr13250]$ cat foo.c
char mpi_fortran_argv_null_[8];
[hjl@gnu-6 pr13250]$ cat bar.c
extern char mpi_fortran_argv_null_[8];

int
bar ()
{
 return mpi_fortran_argv_null_[0];
}
[hjl@gnu-6 pr13250]$ cat main.c
char mpi_fortran_argv_null_[1];

extern int bar ();

int
main ()
{
  bar ();
  return 0;
}
[hjl@gnu-6 pr13250]$ make
gcc -g -c -o main.o main.c
gcc -shared -fPIC -g -o libfoo.so foo.c
gcc -shared -fPIC -g -o libbar.so bar.c libfoo.so
gcc -Wl,--as-needed -o x main.o libbar.so libfoo.so -Wl,-rpath,.
/usr/local/bin/ld: main.o(.debug_info+0x7e): unresolvable R_X86_64_64 relocation against symbol `mpi_fortran_argv_null_'
./x
[hjl@gnu-6 pr13250]$

[Hjl-tools (hjl-tools)]

Created attachment 5963
A patch

This patch preserves the maximum alignment and size for common
symbols.

[Matthias Klose (doko)]

the patch fixes the original issue as well.

[Hjl-tools (hjl-tools)]

A patch is posted at

http://sourceware.org/ml/binutils/2011-10/msg00035.html

[Cvs-commit (cvs-commit)]

CVSROOT: /cvs/src
Module name: src
Changes by: <email address hidden> 2011-10-08 16:51:11

Modified files:
 bfd : ChangeLog elflink.c
 ld/testsuite : ChangeLog
 ld/testsuite/ld-elf: shared.exp
Added files:
 ld/testsuite/ld-elf: pr13250-1.c pr13250-2.c pr13250-3.c

Log message:
 Preserve the maximum alignment/size for common symbols.

 bfd/

 2011-10-08 H.J. Lu <email address hidden>

 PR ld/13250
 * elflink.c (elf_link_add_object_symbols): Preserve the maximum
 alignment and size for common symbols.

 ld/testsuite/

 2011-10-08 H.J. Lu <email address hidden>

 PR ld/13250
 * ld-elf/shared.exp (build_tests): Add tests for PR ld/13250.
 (run_tests): Likewise.

 * ld-elf/pr13250-1.c: New.
 * ld-elf/pr13250-2.c: Likewise.
 * ld-elf/pr13250-3.c: Likewise.

Patches:
http://sourceware.org/cgi-bin/cvsweb.cgi/src/bfd/ChangeLog.diff?cvsroot=src&r1=1.5483&r2=1.5484
http://sourceware.org/cgi-bin/cvsweb.cgi/src/bfd/elflink.c.diff?cvsroot=src&r1=1.424&r2=1.425
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ChangeLog.diff?cvsroot=src&r1=1.1465&r2=1.1466
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/pr13250-1.c.diff?cvsroot=src&r1=NONE&r2=1.1
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/pr13250-2.c.diff?cvsroot=src&r1=NONE&r2=1.1
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/pr13250-3.c.diff?cvsroot=src&r1=NONE&r2=1.1
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/shared.exp.diff?cvsroot=src&r1=1.17&r2=1.18

[Martin Pitt (pitti)]

@Matthias: Did something go wrong here? The uploaded proposed package only changes the manpage, but nothing else:

  http://launchpadlibrarian.net/82305059/binutils_2.21.53.20110810-0ubuntu3_2.21.53.20110810-0ubuntu4.diff.gz

The upstream patch actually changes code:

  http://sourceware.org/bugzilla/attachment.cgi?id=5963

As this touches central code (not fortran specific), could this cause other packages to now FTBFS or malfunction?

[Kate Stewart (kate.stewart)]

Rather worried about this change going into Oneiric, esp. at this late in the cycle since it impacts central code.
The risk is too high, definitely NOT for the release, and update needs much more discussion and exploration.

Would rather it go out in Precise, and a discussion happen at UDS whether its a candidate for Oneiric updates or not.

[Matthias Klose (doko)]

On 10/10/2011 01:07 AM, Kate Stewart wrote:
> Rather worried about this change going into Oneiric, esp. at this late in the cycle since it impacts central code.
> The risk is too high, definitely NOT for the release,

this was uploaded to to -proposed. You are drawing wrong conclusions here.

[Hjl-tools (hjl-tools)]

Fixed.

[Matthias Klose (doko)]

> As this touches central code (not fortran specific),
> could this cause other packages to now FTBFS or malfunction?

I don't see why it would cause malfunction or cause build failures. This is a regression in oneiric, compared to natty (where the ld --as-needed default was reverted before the release).

[Clint Byrum (clint-fewbar)]

Hello Matthias, or anyone else affected,

Accepted binutils into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package binutils - 2.21.90.20111004-2ubuntu1

---------------
binutils (2.21.90.20111004-2ubuntu1) oneiric; urgency=low

  * Merge with Debian.

binutils (2.21.90.20111004-2) unstable; urgency=low

  * Fix PR ld/13250, preserve the maximum alignment and size for common
    symbols (H.J. Lu). Closes: #608901. LP: #697229.

binutils (2.21.90.20111004-1) unstable; urgency=medium

  * Snapshot, taken from the binutils-2_22-branch 20111004.
  * Fix PR ld/13195, taken from the trunk. Closes: #643858, #644183.

binutils (2.21.53.20110922-1) unstable; urgency=low

  * Snapshot, taken from the binutils-2_22-branch 20110924.

binutils (2.21.53.20110910-1) unstable; urgency=low

  * Snapshot, taken from the trunk 20110910.
    - Fix PR gas/13024. Closes: #635592.
  * Build with proposed patch for PR ld/12965.
  * Don't ship .la files in the cross binutils packages.
  * For native builds, add the multiarch directories to the default
    search path.

binutils (2.21.53.20110823-3) unstable; urgency=low

  * Revert the last change:
    - Default to --hash-style=both in ld.bfd and ld.gold.

binutils (2.21.53.20110823-2) unstable; urgency=low

  * Default to --hash-style=both in ld.bfd and ld.gold.
  * Fix s390x build (Aurelian Jarno). Closes: #635167.

binutils (2.21.53.20110823-1) unstable; urgency=low

  * Snapshot, taken from the trunk 20110823.
    - ld --no-copy-dt-needed-entries/--no-add-needed is now the default.

binutils (2.21.53.20110810-0ubuntu4) oneiric-proposed; urgency=low

  * Fix PR ld/13250, preserve the maximum alignment and size for common
    symbols (H.J. Lu). Closes: #608901. LP: #697229.
 -- Matthias Klose <email address hidden> Tue, 11 Oct 2011 12:36:33 +0200

[Cvs-commit (cvs-commit)]

CVSROOT: /cvs/src
Module name: src
Branch: binutils-2_22-branch
Changes by: <email address hidden> 2011-10-25 02:57:24

Modified files:
 bfd : ChangeLog elflink.c
 ld/testsuite : ChangeLog
 ld/testsuite/ld-elf: shared.exp
Added files:
 ld/testsuite/ld-elf: pr13250-1.c pr13250-2.c pr13250-3.c

Log message:
 PR ld/13250
 bfd/
 2011-10-08 H.J. Lu <email address hidden>
 * elflink.c (elf_link_add_object_symbols): Preserve the maximum
 alignment and size for common symbols.
 ld/testsuite/
 2011-10-08 H.J. Lu <email address hidden>
 * ld-elf/shared.exp (build_tests): Add tests for PR ld/13250.
 (run_tests): Likewise.
 * ld-elf/pr13250-1.c: New.
 * ld-elf/pr13250-2.c: Likewise.
 * ld-elf/pr13250-3.c: Likewise.

Patches:
http://sourceware.org/cgi-bin/cvsweb.cgi/src/bfd/ChangeLog.diff?cvsroot=src&only_with_tag=binutils-2_22-branch&r1=1.5473.2.10&r2=1.5473.2.11
http://sourceware.org/cgi-bin/cvsweb.cgi/src/bfd/elflink.c.diff?cvsroot=src&only_with_tag=binutils-2_22-branch&r1=1.420.2.5&r2=1.420.2.6
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ChangeLog.diff?cvsroot=src&only_with_tag=binutils-2_22-branch&r1=1.1460.2.6&r2=1.1460.2.7
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/pr13250-1.c.diff?cvsroot=src&only_with_tag=binutils-2_22-branch&r1=NONE&r2=1.1.2.1
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/pr13250-2.c.diff?cvsroot=src&only_with_tag=binutils-2_22-branch&r1=NONE&r2=1.1.2.1
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/pr13250-3.c.diff?cvsroot=src&only_with_tag=binutils-2_22-branch&r1=NONE&r2=1.1.2.1
http://sourceware.org/cgi-bin/cvsweb.cgi/src/ld/testsuite/ld-elf/shared.exp.diff?cvsroot=src&only_with_tag=binutils-2_22-branch&r1=1.17&r2=1.17.2.1

[Matthias Klose (doko)]

the example builds using the new binutils upload, and fails with the old one.

[Launchpad Janitor (janitor)]

This bug was fixed in the package binutils - 2.21.53.20110810-0ubuntu4

---------------
binutils (2.21.53.20110810-0ubuntu4) oneiric-proposed; urgency=low

  * Fix PR ld/13250, preserve the maximum alignment and size for common
    symbols (H.J. Lu). Closes: #608901. LP: #697229.
 -- Matthias Klose <email address hidden> Thu, 06 Oct 2011 09:22:34 +0200

[Cvs-commit (cvs-commit)]

The master branch has been updated by Alan Modra <email address hidden>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=7ba115508aa02ffbb01a09613b5dffdd0c6563e3

commit 7ba115508aa02ffbb01a09613b5dffdd0c6563e3
Author: Alan Modra <email address hidden>
Date: Tue Sep 8 13:02:31 2020 +0930

    PR26580, Size and alignment of commons vs as-needed shared lib

    Two pieces to this puzzle:
    1) Revert HJ's fix for PR13250 so that size and alignment isn't
       sticky, instead attack the real underlying problem that
       _bfd_generic_link_add_one_symbol does the wrong thing in making a
       common section in a shared library bfd.
    2) Save and restore common u.c.p fields, which hold the section and
       alignment.

    A better fix for (2) would be to throw away all of that horrible code
    saving and restoring the hash table when loading as-needed library
    symbols, and instead do a scan over as-needed library symbols before
    adding anything.

    bfd/
            PR 13250
            PR 26580
            * elflink.c (_bfd_elf_merge_symbol): Make "override" a bfd**.
            Return oldbfd in override when old common should override new
            common.
            (_bfd_elf_add_default_symbol): Adjust to suit.
            (elf_link_add_object_symbols): Likewise. Pass "override" to
            _bfd_generic_link_add_one_symbol. Save and restore common u.c.p
            field for --as-needed shared libraries. Revert pr13250 changes.
    ld/
            * testsuite/ld-elf/pr26580-a.s,
            * testsuite/ld-elf/pr26580-b.s,
            * testsuite/ld-elf/pr26580-1.sd,
            * testsuite/ld-elf/pr26580-2.sd: New tests
            * testsuite/ld-elf/comm-data.exp: Run new tests.
            * testsuite/ld-elf/pr26580-a.c,
            * testsuite/ld-elf/pr26580-b.c,
            * testsuite/ld-elf/pr26580-3.out,
            * testsuite/ld-elf/pr26580-4.out: New tests.
            * testsuite/ld-elf/shared.exp: Run new tests.