ADT tests fail since 4.11.0-11.16

[Iain Lane]

The ADT tests started failing with 4.11.0-11.16 on amd64, i386 and ppc64el: all the arches we run using qemu over OpenStack. The virtual machines hang completely.

The last lines in the log are:

12:11:18 DEBUG| [stdout] Test icebp [Ok]
12:11:18 DEBUG| [stdout] Test int 3 trap [Failed]
12:11:18 DEBUG| [stdout] selftests: breakpoint_test [PASS]
12:11:18 DEBUG| [stdout] /tmp/autopkgtest.zKFdZm/build.ham/linux-4.11.0/autotest/client/tmp/ubuntu_kernel_selftests/src/linux/tools/testing/selftests/breakpoints

...

and on the console we see a lot of messages like:

** 35 printk messages dropped ** [ 3577.683828] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20170119/evevent-286)
** 33 printk messages dropped ** [ 3577.686564] ACPI Error: Could not disable RealTimeClock events (20170119/evxfevnt-267)
** 25 printk messages dropped ** [ 3577.688566] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20170119/evevent-286)
** 35 printk messages dropped ** [ 3577.692788] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20170119/evevent-286)
** 29 printk messages dropped ** [ 3577.695184] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20170119/evevent-286)
** 34 printk messages dropped ** [ 3577.697955] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20170119/evevent-286)

I've attached the full log. In the log file there is the invocation I used to generate the failing log. You should be able to try similar using either `qemu' locally. If that doesn't reproduce, try on Canonistack. If *that* doesn't reproduce, apw or I can run test builds for you if there's a PPA to test from.

This tickles an unfortunate behaviour in the way we run autopkgtest. What happens is: the machine hangs, autopkgtest notices this has happened, it tries to clean up the testbed and can't get into it because it is hung. This is counted as a 'temporary failure', and so the job is requeued forever. As a result, I've blacklisted linux from the workers. Please reach out to us for manual runs, and then once it is confirmed to not loop in this way we'll remove the blacklist.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1706283

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Stefan Bader (smb)]

So the problem drilled down to the autotest script which was supposed to disable adding a certain compile option / sub-module to the breakpoints subtest. However, the naming in the Makefile changed in the newer kernel and so this problem happened (again).

From the test script:
# currently disable step_after_suspend_test as this breaks ssh'd
# login connections to the test VMs and real H/W

-cmd = 'sed -i "s/TEST_PROGS += step_after_suspend_test/# TEST_PROGS += step_after_suspend_test/" ' + fn
+cmd = 'sed -i "s/\(.* += step_after_suspend_test\)/#\\1/" ' + fn

Will update the script and re-enable the testing again.

[Stefan Bader (smb)]

autotest-client-test repository updated now (on the positive side this was no real regression in the kernel but a failure to disable a known problematic part of the self tests).

[Iain Lane (laney)]

On Tue, Jul 25, 2017 at 06:49:02PM -0000, Stefan Bader wrote:
> autotest-client-test repository updated now (on the positive side this
> was no real regression in the kernel but a failure to disable a known
> problematic part of the self tests).

Alright - re-running, will unblacklist if it doesn't hang.

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]

[Iain Lane (laney)]

ok, the test completes without hanging. It fails though:

  ubuntu-regression-suite FAIL non-zero exit status 253

but that's not what this bug was about, so closing & unblacklisting. Thanks!

[Iain Lane (laney)]

(ah, it was already closed)