risc-v 5.8 kernel test-kernel-security.py test failures

[Colin Ian King]

Running RISC-V in QEMU emulator we hit the following adt test failures:

15:30:56 ERROR| [stderr] ======================================================================
15:30:56 ERROR| [stderr] FAIL: test_130_config_debug_rodata (__main__.KernelSecurityConfigTest)
15:30:56 ERROR| [stderr] CONFIG_DEBUG_RODATA/CONFIG_STRICT_KERNEL_RWX enabled
15:30:56 ERROR| [stderr] ----------------------------------------------------------------------
15:30:56 ERROR| [stderr] Traceback (most recent call last):
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 2360, in test_130_config_debug_rodata
15:30:56 ERROR| [stderr] self.assertKernelConfig(option, expected)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 216, in assertKernelConfig
15:30:56 ERROR| [stderr] self.assertKernelConfigUnset(name)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 207, in assertKernelConfigUnset
15:30:56 ERROR| [stderr] '%s option was expected to be unset in the kernel config' % name)
15:30:56 ERROR| [stderr] AssertionError: STRICT_KERNEL_RWX option was expected to be unset in the kernel config
15:30:56 ERROR| [stderr]
15:30:56 ERROR| [stderr] ======================================================================
15:30:56 ERROR| [stderr] FAIL: test_180_config_stack_protector (__main__.KernelSecurityConfigTest)
15:30:56 ERROR| [stderr] CONFIG_CC_STACKPROTECTOR set
15:30:56 ERROR| [stderr] ----------------------------------------------------------------------
15:30:56 ERROR| [stderr] Traceback (most recent call last):
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 2520, in test_180_config_stack_protector
15:30:56 ERROR| [stderr] self.assertKernelConfig(config_option, expected)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 214, in assertKernelConfig
15:30:56 ERROR| [stderr] self.assertKernelConfigSet(name)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 201, in assertKernelConfigSet
15:30:56 ERROR| [stderr] '%s option was expected to be set in the kernel config' % name)
15:30:56 ERROR| [stderr] AssertionError: STACKPROTECTOR option was expected to be set in the kernel config
15:30:56 ERROR| [stderr] ======================================================================
15:30:56 ERROR| [stderr] FAIL: test_180_config_stack_protector (__main__.KernelSecurityConfigTest)
15:30:56 ERROR| [stderr] CONFIG_CC_STACKPROTECTOR set
15:30:56 ERROR| [stderr] ----------------------------------------------------------------------
15:30:56 ERROR| [stderr] Traceback (most recent call last):
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 2520, in test_180_config_stack_protector
15:30:56 ERROR| [stderr] self.assertKernelConfig(config_option, expected)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 214, in assertKernelConfig
15:30:56 ERROR| [stderr] self.assertKernelConfigSet(name)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 201, in assertKernelConfigSet
15:30:56 ERROR| [stderr] '%s option was expected to be set in the kernel config' % name)
15:30:56 ERROR| [stderr] AssertionError: STACKPROTECTOR option was expected to be set in the kernel config
15:30:56 ERROR| [stderr]
15:30:56 ERROR| [stderr] ======================================================================
15:30:56 ERROR| [stderr] FAIL: test_185_config_stack_protector_strong (__main__.KernelSecurityConfigTest)
15:30:56 ERROR| [stderr] CONFIG_CC_STACKPROTECTOR_STRONG set
15:30:56 ERROR| [stderr] ----------------------------------------------------------------------
15:30:56 ERROR| [stderr] Traceback (most recent call last):
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 2541, in test_185_config_stack_protector_strong
15:30:56 ERROR| [stderr] self.assertKernelConfig(config_option, expected)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 214, in assertKernelConfig
15:30:56 ERROR| [stderr] self.assertKernelConfigSet(name)
15:30:56 ERROR| [stderr] File "./test-kernel-security.py", line 201, in assertKernelConfigSet
15:30:56 ERROR| [stderr] '%s option was expected to be set in the kernel config' % name)
15:30:57 ERROR| [stderr] AssertionError: STACKPROTECTOR_STRONG option was expected to be set in the kernel config
15:30:57 ERROR| [stderr]
15:30:57 ERROR| [stderr] ======================================================================
15:30:57 ERROR| [stderr] FAIL: test_190_config_have_stack_protector (__main__.KernelSecurityConfigTest)
15:30:57 ERROR| [stderr] CONFIG_HAVE_CC_STACKPROTECTOR set
15:30:57 ERROR| [stderr] ----------------------------------------------------------------------
15:30:57 ERROR| [stderr] Traceback (most recent call last):
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 2555, in test_190_config_have_stack_protector
15:30:57 ERROR| [stderr] self.assertKernelConfig(config_option, expected)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 214, in assertKernelConfig
15:30:57 ERROR| [stderr] self.assertKernelConfigSet(name)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 201, in assertKernelConfigSet
15:30:57 ERROR| [stderr] '%s option was expected to be set in the kernel config' % name)
15:30:57 ERROR| [stderr] AssertionError: HAVE_STACKPROTECTOR option was expected to be set in the kernel config
15:30:57 ERROR| [stderr]
15:30:57 ERROR| [stderr] ======================================================================
15:30:57 ERROR| [stderr] FAIL: test_290_config_hardened_usercopy (__main__.KernelSecurityConfigTest)
15:30:57 ERROR| [stderr] Ensure CONFIG_HARDENED_USERCOPY is set
15:30:57 ERROR| [stderr] ----------------------------------------------------------------------
15:30:57 ERROR| [stderr] Traceback (most recent call last):
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 2684, in test_290_config_hardened_usercopy
15:30:57 ERROR| [stderr] self.assertKernelConfigUnset(config_name)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 207, in assertKernelConfigUnset
15:30:57 ERROR| [stderr] '%s option was expected to be unset in the kernel config' % name)
15:30:57 ERROR| [stderr] AssertionError: HARDENED_USERCOPY option was expected to be unset in the kernel config
15:30:57 ERROR| [stderr] ======================================================================
15:30:57 ERROR| [stderr] FAIL: test_330_config_debug_wx (__main__.KernelSecurityConfigTest)
15:30:57 ERROR| [stderr] Ensure DEBUG_WX is set
15:30:57 ERROR| [stderr] ----------------------------------------------------------------------
15:30:57 ERROR| [stderr] Traceback (most recent call last):
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 2757, in test_330_config_debug_wx
15:30:57 ERROR| [stderr] self.assertKernelConfig('DEBUG_WX', expected)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 216, in assertKernelConfig
15:30:57 ERROR| [stderr] self.assertKernelConfigUnset(name)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 207, in assertKernelConfigUnset
15:30:57 ERROR| [stderr] '%s option was expected to be unset in the kernel config' % name)
15:30:57 ERROR| [stderr] AssertionError: DEBUG_WX option was expected to be unset in the kernel config
15:30:57 ERROR| [stderr]
15:30:57 ERROR| [stderr] ======================================================================
15:30:57 ERROR| [stderr] FAIL: test_350_config_thread_info_in_stack (__main__.KernelSecurityConfigTest)
15:30:57 ERROR| [stderr] Ensure THREAD_INFO_IN_TASK is set
15:30:57 ERROR| [stderr] ----------------------------------------------------------------------
15:30:57 ERROR| [stderr] Traceback (most recent call last):
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 2782, in test_350_config_thread_info_in_stack
15:30:57 ERROR| [stderr] self.assertKernelConfig('THREAD_INFO_IN_TASK', expected)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 216, in assertKernelConfig
15:30:57 ERROR| [stderr] self.assertKernelConfigUnset(name)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 207, in assertKernelConfigUnset
15:30:57 ERROR| [stderr] '%s option was expected to be unset in the kernel config' % name)
15:30:57 ERROR| [stderr] AssertionError: THREAD_INFO_IN_TASK option was expected to be unset in the kernel config
15:30:57 ERROR| [stderr]
15:30:57 ERROR| [stderr] ======================================================================
15:30:57 ERROR| [stderr] FAIL: test_020_aslr_dapper_stack (__main__.KernelSecurityTest)
15:30:57 ERROR| [stderr] ASLR of stack
15:30:57 ERROR| [stderr] ----------------------------------------------------------------------
15:30:57 ERROR| [stderr] Traceback (most recent call last):
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 1718, in test_020_aslr_dapper_stack
15:30:57 ERROR| [stderr] self._test_aslr('stack', 0)
15:30:57 ERROR| [stderr] File "./test-kernel-security.py", line 1704, in _test_aslr
15:30:57 ERROR| [stderr] self.assertShellExitEquals(0, build)
15:30:57 ERROR| [stderr] File "/home/cking/autotest/client/tmp/ubuntu_qrt_kernel_security/src/qa-regression-testing/scripts/testlib.py", line 1158, in assertShellExitEquals
15:30:57 ERROR| [stderr] self.assertEqual(expected, rc, msg + result + report)
15:30:57 ERROR| [stderr] AssertionError: Got exit code 2, expected 0
15:30:57 ERROR| [stderr] Command: 'make'
15:30:57 ERROR| [stderr] Output:
15:30:57 ERROR| [stderr] cc -Wall -Wformat-security -D_FORTIFY_SOURCE=2 -O2 -fPIC -pie -o aslr aslr.c -ldl
15:30:57 ERROR| [stderr] aslr.c: In function ‘area_pointer’:
15:30:57 ERROR| [stderr] aslr.c:144:41: warning: format ‘%p’ expects argument of type ‘void *’, but argument 3 has type ‘uintptr_t’ {aka ‘long unsigned int’} [-Wformat=]
15:30:57 ERROR| [stderr] 144 | fprintf(stderr, "[pie] AREA_MMAP: %p\tAREA_TEXT: %p\n", area_pointer(AREA_MMAP),
15:30:57 ERROR| [stderr] | ~^ ~~~~~~~~~~~~~~~~~~~~~~~
15:30:57 ERROR| [stderr] | | |
15:30:58 ERROR| [stderr] | void * uintptr_t {aka long unsigned int}
15:30:58 ERROR| [stderr] | %ld
15:30:58 ERROR| [stderr] aslr.c:144:56: warning: format ‘%p’ expects argument of type ‘void *’, but argument 4 has type ‘uintptr_t’ {aka ‘long unsigned int’} [-Wformat=]
15:30:58 ERROR| [stderr] 144 | fprintf(stderr, "[pie] AREA_MMAP: %p\tAREA_TEXT: %p\n", area_pointer(AREA_MMAP),
15:30:58 ERROR| [stderr] | ~^
15:30:58 ERROR| [stderr] | |
15:30:58 ERROR| [stderr] | void *
15:30:58 ERROR| [stderr] | %ld
15:30:58 ERROR| [stderr] 145 | area_pointer(AREA_TEXT));
15:30:58 ERROR| [stderr] | ~~~~~~~~~~~~~~~~~~~~~~~
15:30:58 ERROR| [stderr] | |
15:30:58 ERROR| [stderr] | uintptr_t {aka long unsigned int}
15:30:58 ERROR| [stderr] cc -Wall -Wformat-security -D_FORTIFY_SOURCE=2 -O2 -fPIC -pie -o mmap-growth-direction mmap-growth-direction.c
15:30:58 ERROR| [stderr] cc -Wall -Wformat-security -D_FORTIFY_SOURCE=2 -O2 -m32 -fPIC -pie -o aslr32 aslr.c -ldl
15:30:58 ERROR| [stderr] cc: error: unrecognized command-line option ‘-m32’
15:30:58 ERROR| [stderr] make: *** [Makefile:28: aslr32] Error 1

[Colin Ian King (colin-king)]

10:02:27 ERROR| [stderr] CONFIG_DEBUG_RODATA/CONFIG_STRICT_KERNEL_RWX enabled^M
10:02:27 ERROR| [stderr] ----------------------------------------------------------------------^M
10:02:27 ERROR| [stderr] Traceback (most recent call last):^M
10:02:27 ERROR| [stderr] File "./test-kernel-security.py", line 2360, in test_130_config_debug_rodata^M
10:02:27 ERROR| [stderr] self.assertKernelConfig(option, expected)^M
10:02:27 ERROR| [stderr] File "./test-kernel-security.py", line 216, in assertKernelConfig^M
10:02:27 ERROR| [stderr] self.assertKernelConfigUnset(name)^M
10:02:27 ERROR| [stderr] File "./test-kernel-security.py", line 207, in assertKernelConfigUnset^M
10:02:27 ERROR| [stderr] '%s option was expected to be unset in the kernel config' % name)^M
10:02:27 ERROR| [stderr] AssertionError: STRICT_KERNEL_RWX option was expected to be unset in the kernel config^M

cking@ubuntu:/boot$ uname -a
Linux ubuntu 5.8.0-1-generic #1-Ubuntu SMP Thu Aug 27 19:51:38 UTC 2020 riscv64 riscv64 riscv64 GNU/Linux
cking@ubuntu:/boot$ grep CONFIG_STRICT_KERNEL_RWX /boot/config-$(uname -r)
CONFIG_STRICT_KERNEL_RWX=y

[Colin Ian King (colin-king)]

10:02:27 ERROR| [stderr] FAIL: test_180_config_stack_protector (__main__.KernelSecurityConfigTest)^
10:02:27 ERROR| [stderr] CONFIG_CC_STACKPROTECTOR set

cking@ubuntu:/boot$ grep STACKPROTECTOR /boot/config-$(uname -r)
CONFIG_CC_HAS_STACKPROTECTOR_NONE=y

[Colin Ian King (colin-king)]

10:02:27 ERROR| [stderr] FAIL: test_185_config_stack_protector_strong (__main__.KernelSecurityConfigTest)
10:02:27 ERROR| [stderr] CONFIG_CC_STACKPROTECTOR_STRONG set

grep CONFIG_CC_STACKPROTECTOR_STRONG /boot/config-$(uname -r)

..nothing

[Steve Beattie (sbeattie)]

To summarize, the following tests are failing on riscv:

  test_130_config_debug_rodata
  test_180_config_stack_protector
  test_185_config_stack_protector_strong
  test_190_config_have_stack_protector
  test_290_config_hardened_usercopy
  test_330_config_debug_wx
  test_350_config_thread_info_in_stack
  test_020_aslr_dapper_stack

Colin provided patches to address the first four issues, which I have applied, though I modified and coalesced the stack protector ones into a single change/commit that should have the same effect. They are qrt commits:

  https://git.launchpad.net/qa-regression-testing/commit/?id=f7e0467e3ff97e0004c5b779472d01265a3fb298
  https://git.launchpad.net/qa-regression-testing/commit/?id=7d1068507dc11e52ab1e0016dfcba56615fd24db

I'm digging in to the rest.

[Steve Beattie (sbeattie)]

test_290_config_hardened_usercopy looks to be failing despite the fact that the HARDENED_USERCOPY option is enabled. The reason for this is that there is an additional assumption that STRICT_DEVMEM will be enabled as well (the Kconfig for HARDENED_USERCOPY has an 'imply' on it) but apparently that can't be enabled on riscv because STRICT_DEVMEM has a config dependency on ARCH_HAS_DEVMEM_IS_ALLOWED which is not apparently support for riscv. Will try to fix this up in QRT.

[Steve Beattie (sbeattie)]

For test_330_config_debug_wx, groovy's riscv kernel has it enabled, but the test does not expect it to be (it's disabled for linux-riscv in focal -- ideally we would fix that). At some point between 5.4 and 5.8 the kernel added a check config CONFIG_ARCH_HAS_DEBUG_WX for some arches, so the test should probably be updated to check that (although oddly, it's not set for the armhf kernel configs, so may be buggy.

[Steve Beattie (sbeattie)]

For test_350_config_thread_info_in_stack, this is enabled for linux-riscv in both focal and groovy and the test should be updated to reflect that.

[Steve Beattie (sbeattie)]

Finally, the test_020_aslr_dapper_stack test is failing to build it's 32bit version of the test utility, with the compiler complaining that -m32 is unknown. I don't actually know if we've configured the toolchain for 32 bit builds to work on riscv64, so this needs a ittle more exploring.

[Po-Hsu Lin (cypressyew)]

Riscv 5.8 EOL, closing this bug.