test_072_strict_devmem in ubuntu_qrt_kernel_security should be skipped on Trusty ARM64

[Po-Hsu Lin]

This was reported in bug 1630000
I think it's better it tear it down into pieces.

From the error message, it looks like this test should be skipped on Trusty 3.13 kernel:

dev/mem unreadable for kernel memory ... (using 0x4fab5a72c0) (skipped: only 3.16 and later for ARM64) (exit code 7) FAIL

 FAIL: test_072_strict_devmem (__main__.KernelSecurityTest)
 /dev/mem unreadable for kernel memory
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "./test-kernel-security.py", line 766, in test_072_strict_devmem
     self.assertTrue(rc in expected, 'exit code: %d (wanted %s). Output:\n%s' % (rc, ", ".join(["%d" % (x) for x in expected]), output))
 AssertionError: exit code: 7 (wanted 4). Output:
   4000080000-4000901d13 : Kernel code
 0x4000080000 ... readable
 0x8000100000 ... missing (EFAULT), ran off end of physical memory?
 0x4fef98f540 ... 0xfeedface
 FAIL: scanned memory without EPERMs, and can read actual values

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-3.13.0-141-generic 3.13.0-141.190
ProcVersionSignature: User Name 3.13.0-141.190-generic 3.13.11-ckt39
Uname: Linux 3.13.0-141-generic aarch64
NonfreeKernelModules: signpost
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access /dev/snd/: No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.14.1-0ubuntu3.27
Architecture: arm64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
CRDA: Error: [Errno 2] No such file or directory: 'iw'
CurrentDmesg:

Date: Wed Feb 7 07:06:33 2018
Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: console=ttyS0,9600n8r ro
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-141-generic N/A
 linux-backports-modules-3.13.0-141-generic N/A
 linux-firmware 1.127.24
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This change was made by a bot.

[Po-Hsu Lin (cypressyew)]

On Trusty ARM64, it will be skipped now.

  test_072_strict_devmem (__main__.KernelSecurityTest)
  /dev/mem unreadable for kernel memory ... (skipped: CONFIG_DEVMEM not enabled, skipping checks) ok

Closing this issue.

[Steve Beattie (sbeattie)]

Actually, this is an accidental pass, the reason is that the CONFIG_DEVMEM option did not come into being until the 3.19 kernel. I've adjusted the tests for that in https://git.launchpad.net/qa-regression-testing/commit/?id=f405927a76483d29f3a6bde092ebc29d7de6d945 .

Given that CONFIG_STRICT_DEVMEM is not set for arm64 on trusty, looking at the specific test exception, it was not accepting the correct failure value from the test (we run the test even when expecting failure, because we want to know if/when it gets fixed). I've added the correct failure value in the expected set of values for arm64 on trusty and older in https://git.launchpad.net/qa-regression-testing/commit/?id=ab5ba157aa0ba979e8e97612fe56678fbaabcdc5 .

I'm closing this bug based on the above commits; please re-open if I made a thinko somewhere in my logic and the test is still failing. Thanks!