QRT failed on test_072_config_debug_rodata and test_072_strict_devmem

[C de-Avillez]

ubuntu@panda-quantal:~/qrt-test-kernel$ sudo ./test-kernel-security.py -v
Running test: './test-kernel-security.py' distro: 'Ubuntu 12.10' kernel: '3.4.0-203.8 (Ubuntu 3.4.0-203.8-omap4 3.4.0)' arch: 'armhf' uid: 0/0 SUDO_USER: 'ubuntu')
test_000_make (__main__.KernelSecurityTest)
Prepare to build helper tools ... (4.7.1 (Ubuntu/Linaro 4.7.1-5ubuntu1)) ok
test_010_proc_maps (__main__.KernelSecurityTest)
/proc/$pid/maps is correctly protected ... ok
test_020_aslr_00_proc (__main__.KernelSecurityTest)
ASLR enabled ... ok
test_020_aslr_dapper_stack (__main__.KernelSecurityTest)
ASLR of stack ... ok
test_021_aslr_dapper_libs (__main__.KernelSecurityTest)
ASLR of libs ... ok
test_021_aslr_dapper_mmap (__main__.KernelSecurityTest)
ASLR of mmap ... ok
test_022_aslr_hardy_text (__main__.KernelSecurityTest)
ASLR of text ... ok
test_022_aslr_hardy_vdso (__main__.KernelSecurityTest)
ASLR of vdso ... (skipped: only x86) ok
test_022_aslr_intrepid_brk (__main__.KernelSecurityTest)
ASLR of brk ... ok
test_030_mmap_min (__main__.KernelSecurityTest)
Low memory allocation respects mmap_min_addr ... (32768) ok
test_031_apparmor (__main__.KernelSecurityTest)
AppArmor loaded ... ok
test_031_seccomp (__main__.KernelSecurityTest)
PR_SET_SECCOMP works ... ok
test_032_dev_kmem (__main__.KernelSecurityTest)
/dev/kmem not available ... ok
test_033_syn_cookies (__main__.KernelSecurityTest)
SYN cookies is enabled ... ok
test_040_pcaps (__main__.KernelSecurityTest)
init's CAPABILITY list is clean ... ok
test_050_personality (__main__.KernelSecurityTest)
init missing READ_IMPLIES_EXEC ... (/proc/1/personality) ok
test_060_nx (__main__.KernelSecurityTest)
NX bit is working ... ok
test_061_guard_page (__main__.KernelSecurityTest)
Userspace stack guard page exists (CVE-2010-2240) ... ok
test_070_config_brk (__main__.KernelSecurityTest)
CONFIG_COMPAT_BRK disabled ... ok
test_070_config_devkmem (__main__.KernelSecurityTest)
CONFIG_DEVKMEM disabled ... ok
test_070_config_seccomp (__main__.KernelSecurityTest)
CONFIG_SECCOMP enabled ... ok
test_070_config_security (__main__.KernelSecurityTest)
CONFIG_SECURITY enabled ... ok
test_070_config_security_selinux (__main__.KernelSecurityTest)
CONFIG_SECURITY_SELINUX enabled ... ok
test_070_config_syn_cookies (__main__.KernelSecurityTest)
CONFIG_SYN_COOKIES enabled ... ok
test_071_config_seccomp (__main__.KernelSecurityTest)
CONFIG_SECCOMP enabled ... ok
test_072_config_compat_vdso (__main__.KernelSecurityTest)
CONFIG_COMPAT_VDSO disabled ... ok
test_072_config_debug_rodata (__main__.KernelSecurityTest)
CONFIG_DEBUG_RODATA enabled ... (skipped: only x86) FAIL
test_072_config_debug_set_module_ronx (__main__.KernelSecurityTest)
CONFIG_DEBUG_SET_MODULE_RONX enabled ... (skipped: only x86) ok
test_072_config_security_apparmor (__main__.KernelSecurityTest)
CONFIG_SECURITY_APPARMOR enabled ... ok
test_072_config_strict_devmem (__main__.KernelSecurityTest)
CONFIG_STRICT_DEVMEM enabled ... ok
test_072_strict_devmem (__main__.KernelSecurityTest)
/dev/mem unreadable for kernel memory ... FAIL
test_073_config_security_file_capabilities (__main__.KernelSecurityTest)
CONFIG_SECURITY_FILE_CAPABILITIES enabled ... (skipped: only Intrepid through Lucid) ok
test_073_config_security_smack (__main__.KernelSecurityTest)
CONFIG_SECURITY_SMACK enabled ... ok
test_074_config_security_default_mmap_min_addr (__main__.KernelSecurityTest)
CONFIG_DEFAULT_MMAP_MIN_ADDR ... (32768) ok
test_075_config_stack_protector (__main__.KernelSecurityTest)
CONFIG_CC_STACKPROTECTOR set ... ok
test_082_stack_guard_kernel (__main__.KernelSecurityTest)
Kernel stack guard ... ok
test_090_module_blocking (__main__.KernelSecurityTest)
Sysctl to disable module loading exists ... ok
test_091_symlink_following_in_sticky_directories (__main__.KernelSecurityTest)
Symlinks not followable across differing uids in sticky directories ... ok
test_092_hardlink_restriction (__main__.KernelSecurityTest)
Hardlink disallowed for unreadable/unwritable sources ... ok
test_093_ptrace_restriction (__main__.KernelSecurityTest)
ptrace allowed only on children or declared processes ... ok
test_093_ptrace_restriction_extras (__main__.KernelSecurityTest)
ptrace from thread on tracee that used prctl(PR_SET_PTRACER) ... ok
test_093_ptrace_restriction_parent_via_thread (__main__.KernelSecurityTest)
ptrace of child works from parent threads (LP: #737676) ... ok
test_093_ptrace_restriction_prctl_via_thread (__main__.KernelSecurityTest)
prctl(PR_SET_PTRACER) works from threads (LP: #729839) ... ok
test_094_rare_net_autoload (__main__.KernelSecurityTest)
rare network modules do not autoload ... ok
test_095_kernel_symbols_acl (__main__.KernelSecurityTest)
/proc/sys/kernel/kptr_restrict is enabled ... ok
test_095_kernel_symbols_missing (__main__.KernelSecurityTest)
kernel addresses in kallsyms and modules are zeroed out ... ok
test_096_boot_symbols_unreadable (__main__.KernelSecurityTest)
kernel addresses in /boot are not world readable ... ok
test_096_proc_entries_unreadable (__main__.KernelSecurityTest)
sensitive files in /proc are not world readable ... ok
test_100_keep_acpi_method_disabled (__main__.KernelSecurityTest)
/sys/kernel/debug/acpi/custom_method stays disabled ... ok
test_101_proc_fd_leaks (__main__.KernelSecurityTest)
/proc/$pid/ DAC bypass on setuid (CVE-2011-1020) ... ok
test_110_seccomp_filter (__main__.KernelSecurityTest)
seccomp_filter works ... (skipped: only x86 on 3.0 kernel) ok

======================================================================
FAIL: test_072_config_debug_rodata (__main__.KernelSecurityTest)
CONFIG_DEBUG_RODATA enabled
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test-kernel-security.py", line 664, in test_072_config_debug_rodata
    self.assertEqual(self._test_config('DEBUG_RODATA'), expected)
AssertionError: True != False

======================================================================
FAIL: test_072_strict_devmem (__main__.KernelSecurityTest)
/dev/mem unreadable for kernel memory
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test-kernel-security.py", line 730, in test_072_strict_devmem
    (self.kernel_version)])
  File "/home/ubuntu/qrt-test-kernel/testlib.py", line 943, in assertShellExitEquals
    self.assertEquals(expected, rc, msg + result + report)
AssertionError: Got exit code 100, expected 0
Command: 'apt-get', '-y', 'install', 'linux-headers-3.4.0-203-omap4'
Output:
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package linux-headers-3.4.0-203-omap4
E: Couldn't find any package by regex 'linux-headers-3.4.0-203-omap4'

----------------------------------------------------------------------
Ran 51 tests in 120.761s

FAILED (failures=2)

ubuntu@panda-quantal:~/qrt-test-kernel$

ProblemType: Bug
DistroRelease: Ubuntu 12.10
Package: linux-image-3.4.0-203-omap4 3.4.0-203.8
ProcVersionSignature: Ubuntu 3.4.0-203.8-omap4 3.4.0
Uname: Linux 3.4.0-203-omap4 armv7l
ApportVersion: 2.4-0ubuntu4
Architecture: armhf
Date: Sat Jul 21 18:34:45 2012
SourcePackage: linux-ti-omap4
UpgradeStatus: No upgrade log present (probably fresh install)

[C de-Avillez (hggdh2)]

additional info not collected due to
(1) apport hook?
(2) the linux-image is considered a non-official package.

[Paolo Pisati (p-pisati)]

test_072_config_debug_rodata (__main__.KernelSecurityTest)
CONFIG_DEBUG_RODATA enabled ... (skipped: only x86) FAIL

this is clearly not arm related

[Paolo Pisati (p-pisati)]

how do i get and run these tests? how do i reproduce it?

[C de-Avillez (hggdh2)]

2012-07-24 09:00:13 ppisati hggdh: when you are awake, give me an update on bug1027524
2012-07-24 09:02:01 hggdh ppisati: I am awake
2012-07-24 09:03:57 hggdh ppisati: all tests come from git://kernel.ubuntu.com/ubuntu/kernel-testing
2012-07-24 09:06:18 hggdh ppisati: the QRT home is lp:qa-regression-testing (bzr). I understand the kernel-testing grabs from there
2012-07-24 09:08:04 hggdh ppisati: you can branch the QRT, and then run (from ./scripts) 'make-test-tarball' to select a specific test. In our case, it is 'kernel-test-security'
2012-07-24 09:08:48 hggdh then copy the tarball to the system, untar it, and run as 'sudo ./test-kernel-security.py -v'

[Paolo Pisati (p-pisati)]

[flag@newluxor scripts]$ pwd
/home/flag/canonical/qa-regression-testing/scripts
[flag@newluxor scripts]$ ./make-test-tarball kernel-security/
Copying: kernel-security
cp: omitting directory `kernel-security'

what does it mean? what do i have to do now?

[Paolo Pisati (p-pisati)]

ok, after installing the tests and running it manually i saw no failures here (except for the RODATA fail above that is bogus):

======================================================================
FAIL: test_072_config_debug_rodata (__main__.KernelSecurityTest)
CONFIG_DEBUG_RODATA enabled
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test-kernel-security.py", line 664, in test_072_config_debug_rodata
    self.assertEqual(self._test_config('DEBUG_RODATA'), expected)
AssertionError: True != False

----------------------------------------------------------------------
Ran 51 tests in 49.813s

FAILED (failures=1)

flag@panda:~/qrt-test-kernel-security$ uname -a
Linux panda 3.4.0-204-omap4 #9 SMP PREEMPT Fri Jul 6 15:45:33 UTC 2012 armv7l armv7l armv7l GNU/Linux
flag@panda:~/qrt-test-kernel-security$

and in particular:

test_072_config_compat_vdso (__main__.KernelSecurityTest)
CONFIG_COMPAT_VDSO disabled ... ok
test_072_config_debug_rodata (__main__.KernelSecurityTest)
CONFIG_DEBUG_RODATA enabled ... (skipped: only x86) FAIL
test_072_config_debug_set_module_ronx (__main__.KernelSecurityTest)
CONFIG_DEBUG_SET_MODULE_RONX enabled ... (skipped: only x86) ok
test_072_config_security_apparmor (__main__.KernelSecurityTest)
CONFIG_SECURITY_APPARMOR enabled ... ok
test_072_config_strict_devmem (__main__.KernelSecurityTest)
CONFIG_STRICT_DEVMEM enabled ... ok
test_072_strict_devmem (__main__.KernelSecurityTest)
/dev/mem unreadable for kernel memory ... (using 0xa74e28c0L) [ 476.587219] Program readmem tried to access /dev/mem between 80008000->80008004.
 (exit code 0) ok

and here is a snippet of the test:

def test_072_strict_devmem(self):
        '''/dev/mem unreadable for kernel memory'''

        os.chdir('mem')
        target = None
        if self.kernel_is_ubuntu:
            # There isn't a sane way to install the headers via the QRT fields,
            # since there is no meta package for just "linux-headers".
            self.assertShellExitEquals(0, ["apt-get","-y","install",
                                           "linux-headers-%s" %
                                            (self.kernel_version)])

i wonder, do you have the correct headers installed?

You need these:

ii linux-headers-3.4.0-204 3.4.0-204.9 Header files related to Linux kernel version 3.4.0
ii linux-headers-3.4.0-204-omap4 3.4.0-204.9 Linux kernel headers for version 3.4.0 on TI OMAP4-based systems

try it manually.

[C de-Avillez (hggdh2)]

Paolo, no worries on the headers, I was force-running the tests.

Added qa-regression-testing as also-affects, still happening on the tests of 3.4.0-204.9-omap4 this week.

[Paolo Pisati (p-pisati)]

i retried again from scratch with latest kernel (and headers) on my pandaes:

flag@flag-desktop:~$ bzr branch lp:qa-regression-testing
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data. See "bzr help launchpad-login".
Branched 1769 revisions.
flag@flag-desktop:~$ cd qa-regression-testing/
flag@flag-desktop:~/qa-regression-testing/scripts$ ./make-test-tarball test-kernel-security.py
Copying: test-kernel-security.py
Copying: testlib.py
Copying: install-packages
Copying: kernel-security
Skipping 'private/qrt/kernel_security.py' (couldn't find 'private')

Test files: /tmp/qrt-test-kernel-security.tar.gz

To run, copy the tarball somewhere, then do:

$ tar -zxf qrt-test-kernel-security.tar.gz

$ cd ./qrt-test-kernel-security

$ sudo ./install-packages test-kernel-security.py

$ ./test-kernel-security.py -v
flag@flag-desktop:~/qa-regression-testing/scripts$ cd
flag@flag-desktop:~$ tar -zxf /tmp/qrt-test-kernel-security.tar.gz
flag@flag-desktop:~$ cd qrt-test-kernel-security/
flag@flag-desktop:~/qrt-test-kernel-security$ sudo ./install-packages test-kernel-security.py
Requiring 'python-apt' ...
Requiring 'python-unit' ...
Requiring 'lsb-release' ...
Requiring 'build-essential' ...
Requiring 'python-unit' ...
Requiring 'sudo' ...
Requiring 'gdb' ...
Requiring 'gawk' ...
Requiring 'libcap-dev' ...
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  libcap-dev
0 upgraded, 1 newly installed, 0 to remove and 417 not upgraded.
Need to get 22.5 kB of archives.
After this operation, 85.0 kB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports/ quantal/main libcap-dev armhf 1:2.22-1ubuntu3 [22.5 kB]
Fetched 22.5 kB in 0s (84.8 kB/s)
Selecting previously unselected package libcap-dev:armhf.
(Reading database ... 217262 files and directories currently installed.)
Unpacking libcap-dev:armhf (from .../libcap-dev_1%3a2.22-1ubuntu3_armhf.deb) ...
Processing triggers for man-db ...
Setting up libcap-dev:armhf (1:2.22-1ubuntu3) ...

Trying 'libcap2-bin' ...
Trying 'execstack' ...
Trying 'e2fslibs-dev' ...
flag@flag-desktop:~/qrt-test-kernel-security$

flag@flag-desktop:~/qrt-test-kernel-security$ sudo ./test-kernel-security.py -v
Running test: './test-kernel-security.py' distro: 'Ubuntu 12.10' kernel: '3.5.0-211.18 (Ubuntu 3.5.0-211.18-omap4 3.5.4)' arch: 'armhf' uid: 0/0 SUDO_USER: 'flag')
test_000_make (__main__.KernelSecurityTest)
Prepare to build helper tools ... (4.7.2 (Ubuntu/Linaro 4.7.2-1ubuntu1)) ok
test_010_proc_maps (__main__.KernelSecurityTest)
/proc/$pid/maps is correctly protected ... ok
test_020_aslr_00_proc (__main__.KernelSecurityTest)
ASLR enabled ... ok
test_020_aslr_dapper_stack (__main__.KernelSecurityTest)
ASLR of stack ... ok
test_021_aslr_dapper_libs (__main__.KernelSecurityTest)
ASLR of libs ... ok
test_021_aslr_dapper_mmap (__main__.KernelSecurityTest)
ASLR of mmap ... ok
test_022_aslr_hardy_text (__main__.KernelSecurityTest)
ASLR of text ... ok
test_022_aslr_hardy_vdso (__main__.KernelSecurityTest)
ASLR of vdso ... (skipped: only x86) ok
test_022_aslr_intrepi...

[Paolo Pisati (p-pisati)]

this is revision of my tests:

flag@flag-desktop:~/qa-regression-testing$ bzr log|head
------------------------------------------------------------
revno: 1769
committer: Jamie Strandboge <email address hidden>
branch nick: master
timestamp: Thu 2012-09-27 08:28:42 -0500
message:
  update comments in scripts/test-python2.*.py
------------------------------------------------------------
revno: 1768
committer: Marc Deslauriers <email address hidden>

[Paolo Pisati (p-pisati)]

after a face to face chat, it seems this one was a no-issue actually, closing the kernel side.

[Para Siva (psivaa)]

This is still occurring in the latest Kernel SRU regression testing of 3.5.0-221.31-omap4 3.5.7.6. It appears that the test should be skipped for non x86 installations but for some reason it reports FAIL on armhf.

The failure can be seen in,

https://jenkins.qa.ubuntu.com/view/SRU%20Kernel/job/sru_kernel-quantal-generic-armhf_omap4_panda_ES-serial/94/testReport/autotest/ubuntu_qrt_kernel_security/test_kernel_security_py/?

[Steve Beattie (sbeattie)]

The tests in question have been modified to take the different architectures where CONFIG_DEBUG_RODATA and CONFIG_STRICT_DEVMEM are enabled into account. Closing this bug.