xenial/linux-hwe ADT test times out on arm64

[Kleber Sacilotto de Souza]

Testing failed on xenial/linux-hwe 4.15.0-45.48~16.04.1:
    arm64: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/arm64/l/linux-hwe/20190201_153130_e7a9e@/log.gz

xenial/linux-hwe ADT tests have been timing out on almost every run on arm64.

This is not a regression on ADT, the linux-hwe results had been ignored until recently.

[Kleber Sacilotto de Souza (kleber-souza)]

From a quick look at the logs, it seems that a lot more time than it should.

on arm64:

13:46:55 INFO | START ubuntu_qrt_apparmor.test-apparmor.py ubuntu_qrt_apparmor.test-apparmor.py timestamp=1549028815 timeout=10800 localtime=Feb 01 13:46:55
[...]
15:24:17 INFO | END GOOD ubuntu_qrt_apparmor.test-apparmor.py ubuntu_qrt_apparmor.test-apparmor.py timestamp=1549034657 localtime=Feb 01 15:24:17

So that one took almost 1h40 min to complete, being most of the time spent on this part:

13:55:52 ERROR| [stderr] Run kernel regression tests from 14.04's apparmor_2.8.95~2430-0ubuntu5.3 ... (skipped: This test is only for 14.04 systems with the apparmor 2.10.95 SRU or newer installed) ok
13:55:52 ERROR| [stderr] test_parser_testsuite (__main__.ApparmorTestsuites)
13:55:56 DEBUG| [stdout]
15:05:40 ERROR| [stderr] Run parser regression tests ... (Applying patch r3446-fix-sysctl-syscall-detection.patch) (Applying patch tests-readdir-lp1674245.patch) (Applying patch tests-max_path-lp1692543.p
atch) (Applying patch r3701-tests-pivot_root.patch) (Applying patch r3702-tests-query_label.patch) (Applying patch 0001-mount-regression-test-convert-mount-test-to-use-MS_N.patch) ok
15:05:40 ERROR| [stderr] test_regression_testsuite (__main__.ApparmorTestsuites)
15:05:44 DEBUG| [stdout] preparing apparmor_2.10.95-0ubuntu2.10.dsc... done
15:05:44 DEBUG| [stdout]
15:21:42 ERROR| [stderr] Run kernel regression tests ... (Applying patch r3446-fix-sysctl-syscall-detection.patch) (Applying patch tests-readdir-lp1674245.patch) (Applying patch tests-max_path-lp1692543.p
atch) (Applying patch r3701-tests-pivot_root.patch) (Applying patch r3702-tests-query_label.patch) (Applying patch 0001-mount-regression-test-convert-mount-test-to-use-MS_N.patch) ok
15:21:42 ERROR| [stderr] test_stress_parser_testsuite (__main__.ApparmorTestsuites)
15:21:42 DEBUG| [stdout] preparing apparmor_2.10.95-0ubuntu2.10.dsc... done

Whereas, on amd64[1] it took only ~17 min:

12:38:52 INFO | START ubuntu_qrt_apparmor.test-apparmor.py ubuntu_qrt_apparmor.test-apparmor.py timestamp=1549024732 timeout=10800 localtime=Feb 01 12:38:52
[...]
12:55:05 INFO | END GOOD ubuntu_qrt_apparmor.test-apparmor.py ubuntu_qrt_apparmor.test-apparmor.py timestamp=1549025705 localtime=Feb 01 12:55:05

[1] https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/l/linux-hwe/20190201_132455_e7a9e@/log.gz

This testcase could be not the only culprit, but it would be a start for investigation.

[Kleber Sacilotto de Souza (kleber-souza)]

A similar issue is also happening with bionic/linux-hwe:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/arm64/l/linux-hwe/20190327_130612_309fc@/log.gz