apport-retrace should install and use gdb for target release

[Brian Murray]

apport-retrace will use the version of gdb installed on the system performing the retrace. This can cause issues retracing crash reports from releases that have a newer toolchain revision than the system performing the retrace. Subsequently, it would be better if apport-retrace were to install gdb into the sandbox being used for retracing and used that version of gdb for analyzing the core dump.

[Brian Murray (brian-murray)]

This is a xenial crash retraced on precise with the precise version of gdb:

https://errors.ubuntu.com/problem/46cdc799549ce925789dda64598d44de62337f13

The same crash retraced in xenial produced the following results:

http://pastebin.ubuntu.com/13300108/

Notice how the backtrace is much more informative.

[Martin Pitt (pitti)]

Some preparation to avoid installing gdb-dbg: http://bazaar.launchpad.net/~apport-hackers/apport/trunk/revision/3075
Fix: http://bazaar.launchpad.net/~apport-hackers/apport/trunk/revision/3076

[Martin Pitt (pitti)]

I reverted http://bazaar.launchpad.net/~apport-hackers/apport/trunk/revision/3076 in http://bazaar.launchpad.net/~apport-hackers/apport/trunk/revision/3080.

This cannot work like this as we need a gdb command which can actually run on the host. This is not the case when processing crashes from foreign architectures. This is already breaking with an i386 sandbox on amd64:

$ file /tmp/si386/usr/bin/gdb
/tmp/si386/usr/bin/gdb: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=4797df5acda65947a8d421d67e9bc772986ca5ba, stripped

$ ldd /tmp/si386/usr/bin/gdb
 not a dynamic executable
$ /tmp/si386/usr/bin/gdb
bash: /tmp/si386/usr/bin/gdb: not found

This could be made to work with some multi-arch magic on the host, but it will break completely when trying to retrace armhf or ppc reports.

So we'd need to download the apt indexes twice: once for the target arch and downloading the debs/ddebs, and another time for downloading gdb and gdb-multiarch for the host architecture. This is prohibitively expensive when doing it every time, so we need to add a lot of local caching for that. This is also a lot of work, I'm afraid.

[Martin Pitt (pitti)]

Idea from sprint discussion:

In apport:
 - Don't try to run gdb from the retracing target sandbox

 - Add --gdb-root <path> argument to apport-retrace that will set PATH, LD_LIBRARY_PATH, and possibly some env var to specify the gdb plugin dir to appropriate subdirs of <path>. Calling "gdb" should then prefer running gdb from that dir.

In the retracer deployment:
 - add --gdb-root on the sandbox with the same release but the host's native architecture
 - Ensure that gdb is installed in that

[Martin Pitt (pitti)]

> some env var to specify the gdb plugin dir

That seems to be it:
(gdb) show data-directory
GDB's data directory is "/usr/share/gdb".

so we can add that to the gdb invocation (gdb_cmd in add_gdb_info()).

[Martin Pitt (pitti)]

... and change the original patch to only install gdb into the sandbox if it matches the host architecture, as otherwise it'd be a waste.

[Brian Murray (brian-murray)]

LD_LIBRARY_PATH=/path/to/native/sandbox/lib/$(native_multiarch):/path/to/native/sandbox/lib/:/path/to/native/sandbox/usr/lib/$(native_multiarch):/path/to/native/sandbox/usr/path/to/native/sandbox/lib64/ld-linux-x86-64.so.2 /path/to/native/sandbox/usr/bin/gdb-multiarch [options]

[Launchpad Janitor (janitor)]

This bug was fixed in the package apport - 2.20.5-0ubuntu3

---------------
apport (2.20.5-0ubuntu3) artful; urgency=medium

  * Revert change to test/test_crash_digger.py to use the source's
    crash-digger.

 -- Brian Murray <email address hidden> Mon, 15 May 2017 16:00:06 -0700