Log files do not contain some failure strings that they used to

[Brian Murray]

Recently a version of systemd was uploaded which broke resolved and caused the autopackage tests for systemd to fail. The following messages were logged:

autopkgtest [10:17:56]: rebooting testbed after setup commands that affected boot
autopkgtest [10:18:11]: testbed running kernel: Linux 5.15.0-27-generic #28-Ubuntu SMP Thu Apr 14 04:55:28 UTC 2022
autopkgtest [10:18:12]: testbed dpkg architecture: amd64
autopkgtest [10:18:14]: @@@@@@@@@@@@@@@@@@@@ apt-source systemd
qemu-system-x86_64: terminating on signal 15 from pid 983677 (/usr/bin/python3)
autopkgtest [10:21:21]: ERROR: testbed failure: rules extract failed with exit code 100 (apt failure)

However, running the test w/ autopkgtest and --shell-fail we see the following:

Err:1 https://ppa.launchpadcontent.net/enr0n/systemd-broken/ubuntu kinetic/main systemd 251.4-1ubuntu1~ppa6 (tar)
  Temporary failure resolving 'ppa.launchpadcontent.net'
Err:2 https://ppa.launchpadcontent.net/enr0n/systemd-broken/ubuntu kinetic/main systemd 251.4-1ubuntu1~ppa6 (diff)
  Temporary failure resolving 'ppa.launchpadcontent.net'
Err:3 https://ppa.launchpadcontent.net/enr0n/systemd-broken/ubuntu kinetic/main systemd 251.4-1ubuntu1~ppa6 (dsc)
  Temporary failure resolving 'ppa.launchpadcontent.net'
E: Failed to fetch https://ppa.launchpadcontent.net/enr0n/systemd-broken/ubuntu/pool/main/s/systemd/systemd_251.4.orig.tar.gz Temporary failure resolving 'ppa.launchpadcontent.net'
E: Failed to fetch https://ppa.launchpadcontent.net/enr0n/systemd-broken/ubuntu/pool/main/s/systemd/systemd_251.4-1ubuntu1%7eppa6.debian.tar.xz Temporary failure resolving 'ppa.launchpadcontent.net'
E: Failed to fetch https://ppa.launchpadcontent.net/enr0n/systemd-broken/ubuntu/pool/main/s/systemd/systemd_251.4-1ubuntu1%7eppa6.dsc Temporary failure resolving 'ppa.launchpadcontent.net'
E: Failed to fetch some archives.

Looking at FAIL_PKG_STRINGS in autopkgtest-cloud/worker/worker we can see that it contains 'Temporary failure resolving'. So it seems like either autopkgtest used to log more information or the worker code did. We should look into what changed as it is likely that other FAIL_PKG_STRINGS won't match and some failures will not be considered regressions any more.

[Brian Murray (brian-murray)]

The result.tar file from running the bad version of systemd in the staging autopkgtest environment.

[Brian Murray (brian-murray)]

The log file for the test run in the staging environment of the bad version of systemd.