dpkg-shlibdeps fails on armhf ELF binaries that do not define architecture specific information

[Robie Basak]

[Impact]

This causes a build failure on armhf for any package that uses a toolchain that does not provide the optional ELF architecture specific tags like the gcc toolchain does. Example: golang. This stops us building armhf golang packages on Precise, such as the juju-core precise backport.

[Stable Fix]

Before reading any architecture specific tags, consider the ELF header flags field first for a match against the new "hard-float ABI" flag, and use this information over architecture specific tags if available.

[Development Fix]

Exactly the same patch as the stable fix.

[Test Case]

Attempt to build the unmodified golang source package from Saucy on Precise armhf.

Expected results: successful build.

Failure results:

Errors of the form: dpkg-shlibdeps: error: couldn't find library libpthread.so.0 needed by debian/golang-go/usr/bin/go (ELF format: 'elf32-littlearm-sfabi'; RPATH: '').

[Regression Potential]

The change is limited to dpkg-shlibdeps on ARM ELF binaries only, so should only affect rebuilds or backports.

[Original Description]

This causes an FTBFS in golang. See: https://launchpad.net/ubuntu/+source/golang/2:1.1-1/+build/4578681

Affected: dpkg 1.16.10ubuntu1
Not affected: dpkg 1.16.10

I believe the difference is caused by Steve McIntyre's armhf detection patch
applied in the Ubuntu dpkg delta.

For convenience, I attach a golang armhf binary which dpkg-shlibdeps doesn't work with. The error is:

dpkg-shlibdeps: error: couldn't find library libpthread.so.0 needed by /tmp/go (ELF format: 'elf32-littlearm-sfabi'; RPATH: '')
dpkg-shlibdeps: error: couldn't find library libc.so.6 needed by /tmp/go (ELF format: 'elf32-littlearm-sfabi'; RPATH: '')
dpkg-shlibdeps: error: couldn't find library ld-linux-armhf.so.3 needed by /tmp/go (ELF format: 'elf32-littlearm-sfabi'; RPATH: '')
dpkg-shlibdeps: warning: binaries to analyze should already be installed in their package's directory
dpkg-shlibdeps: error: cannot continue due to the errors listed above
Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file.
To help dpkg-shlibdeps find private libraries, you might need to set LD_LIBRARY_PATH.

The cause is that "readelf -A -- /tmp/go" doesn't produce any output. The
arm-specific handling in Dpkg::Shlibs::Objdump then errnoneously decides that
this means that the binary is "elf32-littlearm-sfabi", which mismatches its
dependencies. This causes it to fail, when in fact there is no problem with
the binary's linkage or execution.

Is it a requirement for armhf binaries to have Tag_ABI_VFP_args? I'm not sure
whether the ABI spec actually requires this in order for a binary to be
expected to dynamically link against a libc that does declare the tag. Even if
so, does it make sense for dpkg to handle this condition more gracefully?

Is there a spec which defines a requirement to have Tag_ABI_VFP_args set, so
that we may refer golang upstream to it for a fix? Or is golang producing valid
binaries and dpkg-shlibdeps is buggy?

[Robie Basak (racb)]

@Steve

I'd appreciate your feedback on this. Am I right in assuming that we need your patch for armel/armhf multiarch? Is my diagnosis correct? Do you think that dpkg should behave correctly in this case, or cannot be expected to do so?

[Robie Basak (racb)]

I've filed upstream bug http://code.google.com/p/go/issues/detail?id=5640 to request that the "readelf -A" output be fixed in golang. Depending on what Steve says, that may be the same issue as this bug, or a separate wishlist issue bug, so I will not link it as the upstream version of this bug right now.

[Dave Cheney (dave-cheney)]

I'll hold this bug. I believe a fix is going to land upstream soon. We'll figure out how to backport it then.

[Robie Basak (racb)]

Subscribing Adam.

Dave, Adam was already looking at this in some form. Something to do with the ELF header and not the eabi section tags I think, but he should be able to fill you in. Either way, I thought I'd connect you both to avoid duplicate work.

[Adam Conrad (adconrad)]

This was fixed in both dpkg and golang, I didn't realize there was a bug. Tasks closed now.

[Robie Basak (racb)]

Here's Adam's patch. Dave, do you think this is appropriate to commit upstream, and if so could you commit it, please?

[Dave Cheney (dave-cheney)]

yup, i'll take care of that upstream. https://codereview.appspot.com/11485045

[Dave Cheney (dave-cheney)]

umm, re that patch, objdump says

private flags = 5000402: [Version5 EABI] [has entry point]<Unrecognised flag bits set>

[Robie Basak (racb)]

What version objdump are you using? I get:

private flags = 5000402: [Version5 EABI] [hard-float ABI] [has entry point]

GNU objdump (GNU Binutils for Ubuntu) 2.23.52.20130620
binutils 2.23.52.20130620-1ubuntu1

[Dave Cheney (dave-cheney)]

Mine must be out of date,

alarm(~) % objdump -v
GNU objdump (GNU Binutils) 2.23.1

On Fri, Jul 19, 2013 at 5:16 PM, Robie Basak <email address hidden> wrote:
> What version objdump are you using? I get:
>
> private flags = 5000402: [Version5 EABI] [hard-float ABI] [has entry
> point]
>
> GNU objdump (GNU Binutils for Ubuntu) 2.23.52.20130620
> binutils 2.23.52.20130620-1ubuntu1
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1187722
>
> Title:
> dpkg-shlibdeps fails on armhf ELF binaries that do not define
> architecture specific information
>
> Status in “dpkg” package in Ubuntu:
> Fix Released
> Status in “golang” package in Ubuntu:
> Fix Released
>
> Bug description:
> This causes an FTBFS in golang. See:
> https://launchpad.net/ubuntu/+source/golang/2:1.1-1/+build/4578681
>
> Affected: dpkg 1.16.10ubuntu1
> Not affected: dpkg 1.16.10
>
> I believe the difference is caused by Steve McIntyre's armhf detection patch
> applied in the Ubuntu dpkg delta.
>
> For convenience, I attach a golang armhf binary which dpkg-shlibdeps
> doesn't work with. The error is:
>
> dpkg-shlibdeps: error: couldn't find library libpthread.so.0 needed by /tmp/go (ELF format: 'elf32-littlearm-sfabi'; RPATH: '')
> dpkg-shlibdeps: error: couldn't find library libc.so.6 needed by /tmp/go (ELF format: 'elf32-littlearm-sfabi'; RPATH: '')
> dpkg-shlibdeps: error: couldn't find library ld-linux-armhf.so.3 needed by /tmp/go (ELF format: 'elf32-littlearm-sfabi'; RPATH: '')
> dpkg-shlibdeps: warning: binaries to analyze should already be installed in their package's directory
> dpkg-shlibdeps: error: cannot continue due to the errors listed above
> Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file.
> To help dpkg-shlibdeps find private libraries, you might need to set LD_LIBRARY_PATH.
>
> The cause is that "readelf -A -- /tmp/go" doesn't produce any output. The
> arm-specific handling in Dpkg::Shlibs::Objdump then errnoneously decides that
> this means that the binary is "elf32-littlearm-sfabi", which mismatches its
> dependencies. This causes it to fail, when in fact there is no problem with
> the binary's linkage or execution.
>
> Is it a requirement for armhf binaries to have Tag_ABI_VFP_args? I'm not sure
> whether the ABI spec actually requires this in order for a binary to be
> expected to dynamically link against a libc that does declare the tag. Even if
> so, does it make sense for dpkg to handle this condition more gracefully?
>
> Is there a spec which defines a requirement to have Tag_ABI_VFP_args set, so
> that we may refer golang upstream to it for a fix? Or is golang producing valid
> binaries and dpkg-shlibdeps is buggy?
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/dpkg/+bug/1187722/+subscriptions

[Adam Conrad (adconrad)]

For precise, this will be a byte-identical backport of the upload done for saucy: http://launchpadlibrarian.net/144462697/dpkg_1.16.10ubuntu2_1.16.10ubuntu3.diff.gz

[Colin Watson (cjwatson)]

Hello Robie, or anyone else affected,

Accepted dpkg into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/dpkg/1.16.1.2ubuntu7.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Robie Basak (racb)]

Rebuilt golang 2:1.1.2-2ubuntu1 in a Precise schroot.

On amd64, it works with dpkg 1.16.1.2ubuntu7.2 and the produced go binary works.
On armhf, the build fails with dpkg 1.16.1.2ubuntu7.1 as expected.
On armhf, the build works with dpkg 1.16.1.2ubuntu7.2 and the produced go binary works.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dpkg - 1.16.1.2ubuntu7.2

---------------
dpkg (1.16.1.2ubuntu7.2) precise; urgency=low

  * scripts/Dpkg/Shlibs/Objdump.pm: Adjust the ARM special-casing in
    dpkg-shlibdeps to try ELF ABI flags before arch tag (LP: #1187722)
  * Include powerpc64le support in cputable for ppc64el (LP: #1239595)
 -- Adam Conrad <email address hidden> Mon, 14 Oct 2013 10:42:53 +0100

[Adam Conrad (adconrad)]

Is this actually needed for any sane reason for quantal and raring, or is this just a wishlist from people who like to pointlessly build everything on every release in PPAs?