Handle the skip return code in kernel_selftests on Bionic

Bug #1812352 reported by Po-Hsu Lin on 2019-01-18
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-kernel-tests
Undecided
Po-Hsu Lin
linux (Ubuntu)
Undecided
Unassigned
Trusty
Undecided
Unassigned
Xenial
Undecided
Unassigned
Bionic
Undecided
Po-Hsu Lin

Bug Description

== Justification ==
In the ubuntu_kernel_selftests, the skipped test will uses return code 4
(KSFT_SKIP).

However the code to handle this non-zero return code was not implemented
in the kselftest framework. And this will generate some false-positive
results as those skipped tests were treated as a failure.

For example the raw_skew test in timers on 4.15:
    1..0 # Skipped: The clock was adjusted externally. Shutdown NTPd or other time sync daemons
    not ok 1..7 selftests: raw_skew [FAIL]

And the test_user_copy.sh test in user (failure could be found in KVM kernels):
    user: module test_user_copy is not found [SKIP]
    not ok 1..1 selftests: test_user_copy.sh [FAIL]

Xenial is affected by this issue as well. Considering the fact that this issue in Bionic affects more tests / kernel variants, and it's way easier to fix this in Bionic, I decided to go for Bionic first.

== Fix ==
771cbc3b (selftests: lib.mk set KSFT_TAP_LEVEL to prevent nested TAP headers)
a3355440 (selftests: Fix lib.mk run_tests target shell script)
42b44c34 (selftests: lib.mk: cleanup RUN_TESTS define and make it readable)
3f4435b5 (selftests: lib.mk: add SKIP handling to RUN_TESTS define)

All these can be cherry-picked into Bionic.

The crucial patch is commit 3f4435b5, the other three patches are just
some minor fixes that pave the road for it to be cherry-picked.

== Test ==
Tested on a Bionic amd64 node, the result shows positive.

A test summary before the patch:
END GOOD ubuntu_kernel_selftests.setup
END GOOD ubuntu_kernel_selftests.breakpoints
END GOOD ubuntu_kernel_selftests.cpu-hotplug
END GOOD ubuntu_kernel_selftests.efivarfs
END GOOD ubuntu_kernel_selftests.memfd
END GOOD ubuntu_kernel_selftests.memory-hotplug
END GOOD ubuntu_kernel_selftests.mount
END GOOD ubuntu_kernel_selftests.net
END GOOD ubuntu_kernel_selftests.ptrace
END GOOD ubuntu_kernel_selftests.seccomp
END ERROR ubuntu_kernel_selftests.timers
END GOOD ubuntu_kernel_selftests.powerpc
END GOOD ubuntu_kernel_selftests.user
END GOOD ubuntu_kernel_selftests.ftrace

A test summary after the patch:
END GOOD ubuntu_kernel_selftests.setup
END GOOD ubuntu_kernel_selftests.breakpoints
END GOOD ubuntu_kernel_selftests.cpu-hotplug
END GOOD ubuntu_kernel_selftests.efivarfs
END GOOD ubuntu_kernel_selftests.memfd
END GOOD ubuntu_kernel_selftests.memory-hotplug
END GOOD ubuntu_kernel_selftests.mount
END GOOD ubuntu_kernel_selftests.net
END GOOD ubuntu_kernel_selftests.ptrace
END GOOD ubuntu_kernel_selftests.seccomp
END GOOD ubuntu_kernel_selftests.timers
END GOOD ubuntu_kernel_selftests.powerpc
END GOOD ubuntu_kernel_selftests.user
END GOOD ubuntu_kernel_selftests.ftrac

No tests were skipped unintentionally. And the raw_skew test in timers
is now marked as [SKIPPED] and no longer causing failure.

A complete test report could be found in the bug report attachment.

== Regression Potential ==
Low, this patch set just added support to handle the skip return code
and changes are limited to the kselftest framework itself, no actual
impact to normal users.

== Original bug report ==
In the selftests tool, we are using a KSFT_SKIP=4 return code for those skipped tests in the latest upstream kernel tree.

But the corresponding code to take care of this return value does not exist in Bionic and Xenial kernel, this will cause the test still being marked as failed:

selftests: test_user_copy.sh
========================================
user: module test_user_copy is not found [SKIP]
not ok 1..1 selftests: test_user_copy.sh [FAIL]

Po-Hsu Lin (cypressyew) on 2019-01-18
Changed in linux (Ubuntu):
assignee: nobody → Po-Hsu Lin (cypressyew)

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1812352

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Po-Hsu Lin (cypressyew) on 2019-01-18
Changed in linux (Ubuntu):
status: Incomplete → In Progress

For Bionic, it's:
771cbc3bcbb59084766a501772853f2de7009534
10f531f6748228583b8c4244216e9e3f34d47379
a33554401e4746cc33307910a1baad63ce3fd650
42b44c34136857ccdf90ebb1cbc38f2bf0aec7a1
3f4435b5149372b3bbc5acab5c835d490490d6bc

Po-Hsu Lin (cypressyew) on 2019-05-28
Changed in linux (Ubuntu):
status: In Progress → Confirmed
assignee: Po-Hsu Lin (cypressyew) → nobody
Po-Hsu Lin (cypressyew) on 2019-09-16
tags: added: ubuntu-kernel-selftests
tags: added: bionic trusty
tags: added: xenial
Po-Hsu Lin (cypressyew) wrote :

We don't have kselftest in Trusty, mark this as Invalid for it.

Changed in linux (Ubuntu Trusty):
status: New → Invalid
Sean Feole (sfeole) on 2019-09-26
tags: added: sru-20190902
tags: added: linux-kvm
Po-Hsu Lin (cypressyew) on 2019-10-30
Changed in linux (Ubuntu Bionic):
assignee: nobody → Po-Hsu Lin (cypressyew)
status: New → In Progress
Po-Hsu Lin (cypressyew) on 2019-11-06
Changed in ubuntu-kernel-tests:
assignee: nobody → Po-Hsu Lin (cypressyew)
status: New → In Progress
Po-Hsu Lin (cypressyew) wrote :

Test result for B-4.15 before this patch set.

description: updated
Po-Hsu Lin (cypressyew) wrote :

Test result for B-4.15 after this patch set.

description: updated
Po-Hsu Lin (cypressyew) wrote :

Xenial will require more work, I will split the test report here (see bug 1851486 for X)

summary: - Handle the skip return code in kernel_selftests
+ Handle the skip return code in kernel_selftests on Bionic
Changed in linux (Ubuntu Xenial):
status: New → Invalid
Changed in linux (Ubuntu):
status: Confirmed → Fix Released
description: updated
Changed in linux (Ubuntu Bionic):
status: In Progress → Fix Committed

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
Po-Hsu Lin (cypressyew) wrote :

Verified on Bionic. Works as expected.

Changed in ubuntu-kernel-tests:
status: In Progress → Fix Released
tags: added: verification-done-bionic
removed: verification-needed-bionic
tags: removed: xenial
Launchpad Janitor (janitor) wrote :
Download full text (28.6 KiB)

This bug was fixed in the package linux - 4.15.0-72.81

---------------
linux (4.15.0-72.81) bionic; urgency=medium

  * bionic/linux: 4.15.0-72.81 -proposed tracker (LP: #1854027)

  * [Regression] Bionic kernel 4.15.0-71.80 can not boot on ThunderX
    (LP: #1853326)
    - Revert "arm64: Use firmware to detect CPUs that are not affected by
      Spectre-v2"
    - Revert "arm64: Get rid of __smccc_workaround_1_hvc_*"

  * [Regression] Bionic kernel 4.15.0-71.80 can not boot on ThunderX2 and
    Kunpeng920 (LP: #1852723)
    - SAUCE: arm64: capabilities: Move setup_boot_cpu_capabilities() call to
      correct place

linux (4.15.0-71.80) bionic; urgency=medium

  * bionic/linux: 4.15.0-71.80 -proposed tracker (LP: #1852289)

  * Bionic update: upstream stable patchset 2019-10-29 (LP: #1850541)
    - panic: ensure preemption is disabled during panic()
    - f2fs: use EINVAL for superblock with invalid magic
    - [Config] updateconfigs for USB_RIO500
    - USB: rio500: Remove Rio 500 kernel driver
    - USB: yurex: Don't retry on unexpected errors
    - USB: yurex: fix NULL-derefs on disconnect
    - USB: usb-skeleton: fix runtime PM after driver unbind
    - USB: usb-skeleton: fix NULL-deref on disconnect
    - xhci: Fix false warning message about wrong bounce buffer write length
    - xhci: Prevent device initiated U1/U2 link pm if exit latency is too long
    - xhci: Check all endpoints for LPM timeout
    - usb: xhci: wait for CNR controller not ready bit in xhci resume
    - USB: adutux: fix use-after-free on disconnect
    - USB: adutux: fix NULL-derefs on disconnect
    - USB: adutux: fix use-after-free on release
    - USB: iowarrior: fix use-after-free on disconnect
    - USB: iowarrior: fix use-after-free on release
    - USB: iowarrior: fix use-after-free after driver unbind
    - USB: usblp: fix runtime PM after driver unbind
    - USB: chaoskey: fix use-after-free on release
    - USB: ldusb: fix NULL-derefs on driver unbind
    - serial: uartlite: fix exit path null pointer
    - USB: serial: keyspan: fix NULL-derefs on open() and write()
    - USB: serial: ftdi_sio: add device IDs for Sienna and Echelon PL-20
    - USB: serial: option: add Telit FN980 compositions
    - USB: serial: option: add support for Cinterion CLS8 devices
    - USB: serial: fix runtime PM after driver unbind
    - USB: usblcd: fix I/O after disconnect
    - USB: microtek: fix info-leak at probe
    - USB: dummy-hcd: fix power budget for SuperSpeed mode
    - usb: renesas_usbhs: gadget: Do not discard queues in
      usb_ep_set_{halt,wedge}()
    - usb: renesas_usbhs: gadget: Fix usb_ep_set_{halt,wedge}() behavior
    - USB: legousbtower: fix slab info leak at probe
    - USB: legousbtower: fix deadlock on disconnect
    - USB: legousbtower: fix potential NULL-deref on disconnect
    - USB: legousbtower: fix open after failed reset request
    - USB: legousbtower: fix use-after-free on release
    - staging: vt6655: Fix memory leak in vt6655_probe
    - iio: adc: ad799x: fix probe error handling
    - iio: adc: axp288: Override TS pin bias current for some models
    - iio: light: opt3001: fix mutex unlock race
    - efivar/ssdt: Don't iterate over EFI va...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers