add testbed package list to artifacts.tar.gz

[Steve Langasek]

Sometimes, a package testsuite will regress for reasons unrelated to any direct dependency of the package, and as a result there can be a rather large search space to find the package responsible. Although autopkgtest logs will include the output of all of the various apt runs, this output is very difficult to diff in order to find out what packages have changed, making it even harder to find the responsible package. There may also be regressions caused by packages already installed on the system, that therefore don't show up in any apt output.

Including a sorted list of all packages installed on the system (with versions) would make this a lot easier to sift through.

(As a further example, it appears that GTK has somehow regressed for all tests in xenial-proposed, trying to connect to a mir backend instead of the xvfb provided by the autopkgtest runner - but this change is not attributable to any of the packages that are being installed, so it must be something different in the base environment?)

[Martin Pitt (pitti)]

The "artifacts" tarball already contains a per-test list of all installed packages. This is easier to process automatically than having to find it in the log.

[Steve Langasek (vorlon)]

On Sun, Mar 20, 2016 at 08:13:50PM -0000, Martin Pitt wrote:
> The "artifacts" tarball already contains a per-test list of all
> installed packages. This is easier to process automatically than having
> to find it in the log.

Oh. Thanks!

[Steve Langasek (vorlon)]

I just noticed that the package list in the artifacts tarball is not a complete list of all packages on the system, only a list of the packages that were installed for the test. This is sometimes not all that's needed in order to debug a change in test behavior.

[Martin Pitt (pitti)]

Right, it's the per-test delta to the "testbed package list" which is in results.tar and also copied to the "debci log" e. g. to http://autopkgtest.ubuntu.com/data/packages/xenial/armhf/e/exim4/20160325_051059@.log . If that's inconvenient we can reshuffle this.

[Steve Langasek (vorlon)]

Ok, forgot to look in the debci log for it. That does cover the need, though it would be nice to have the information all in one place (e.g. in artifacts.tar.gz).

[Martin Pitt (pitti)]

Fixed in https://git.launchpad.net/~ubuntu-release/+git/autopkgtest-cloud/commit/?id=0f74bdccbfd62 and rolled out.