few kernel selftest failed on Ubuntu 17.10

[bugproxy]

== Comment: #0 - Harish Sriram <email address hidden> - 2017-08-01 05:11:09 ==
Problem Description
----------------------------
Few kernel selftest failed on Ubuntu 17.10

Environment
------------------
Kernel Build: 4.11.0-10-generic
System Name : ltc-test-ci2
Model : 8247-22L
Platform : PowerNV ( P8 )

Uname output
-------------------
# uname -a
Linux ltc-test-ci2 4.11.0-10-generic #15-Ubuntu SMP Thu Jun 29 15:02:54 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Steps to reproduce:
----------------------------------------
1. Get kernel source using apt source
2. Start powerpc specific tests

The following FAILS are observed.
selftests: cycles_test [FAIL]
selftests: cycles_with_freeze_test [FAIL]
selftests: pmc56_overflow_test [FAIL]
selftests: ebb_vs_cpu_event_test [FAIL]
selftests: cpu_event_vs_ebb_test [FAIL]
selftests: task_event_vs_ebb_test [FAIL]
selftests: multi_ebb_procs_test [FAIL]
selftests: multi_counter_test [FAIL]
selftests: pmae_handling_test [FAIL]
selftests: close_clears_pmcc_test [FAIL]
selftests: instruction_count_test [FAIL]
selftests: ebb_on_willing_child_test [FAIL]
selftests: back_to_back_ebbs_test [FAIL]
selftests: lost_exception_test [FAIL]
selftests: cycles_with_mmcr2_test [FAIL]
selftests: tm-vmxcopy [FAIL]

Full run log is attached.

== Comment: #5 - Harish Sriram <email address hidden> - 2017-08-08 03:33:36 ==
With the latest ubuntu mainline kernel 4.13-rc4 from http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.13-rc4/, I see following additional failures

selftests: ptrace-gpr [FAIL]
selftests: ptrace-tm-gpr [FAIL]
selftests: ptrace-tm-spd-gpr [FAIL]

# uname -a
Linux ltc-test-ci2 4.13.0-041300rc4-generic #201708062231 SMP Mon Aug 7 03:29:19 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Thanks,
Harish

[bugproxy (bugproxy)]

Default Comment by Bridge

[Joseph Salisbury (jsalisbury)]

Did this issue start happening after an update/upgrade? Was there a prior kernel version where you were not having this particular problem?

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.13 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.13-rc4

[Joseph Salisbury (jsalisbury)]

No need to test the mainline kernel requested in comment #2. I see in the bug description that was already done.

However, it still would be good to know if this is a regression.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-09-06 21:17 EDT-------
Hi,

I happen to stumble on this bug, while I'm about to open another bug. The issue seems to stem from the build in powerpc directory. If you do a 'make all' in testing/selftests/powerpc on either 4.12 and 4.13 upstream kernel, it'll fail.

Vicky

[Seth Forshee (sforshee)]

Yes, we've seen build failures too. I've fixed one, I still need to send this fix upstream:

https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/artful/commit/tools/testing/selftests/powerpc?id=12edfcfa8c02c366d32e889d225756db810d7e4b

The other I've reported upstream:

http://lkml.kernel.org/r/<20170830130525.xwj3oy2cx2qdwvqc@ubuntu-xps13>

[Seth Forshee (sforshee)]

Fixing the second link above:

http://lkml.kernel.org/r/20170830130525.xwj3oy2cx2qdwvqc@ubuntu-xps13

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-10-05 05:58 EDT-------
On a PowerVM machine seeing more failures with the 4.13.0-12-generic kernel

selftests: mmap_bench [FAIL]
selftests: futex_bench [FAIL]
selftests: memcpy_64 [FAIL]
selftests: count_instructions [FAIL]
selftests: reg_access_test [FAIL]
selftests: event_attributes_test [FAIL]
selftests: cycles_test [FAIL]
selftests: cycles_with_freeze_test [FAIL]
selftests: pmc56_overflow_test [FAIL]
selftests: ebb_vs_cpu_event_test [FAIL]
selftests: cpu_event_vs_ebb_test [FAIL]
selftests: cpu_event_pinned_vs_ebb_test [FAIL]
selftests: task_event_vs_ebb_test [FAIL]
selftests: task_event_pinned_vs_ebb_test [FAIL]
selftests: multi_ebb_procs_test [FAIL]
selftests: multi_counter_test [FAIL]
selftests: pmae_handling_test [FAIL]
selftests: close_clears_pmcc_test [FAIL]
selftests: instruction_count_test [FAIL]
selftests: fork_cleanup_test [FAIL]
selftests: ebb_on_child_test [FAIL]
selftests: ebb_on_willing_child_test [FAIL]
selftests: back_to_back_ebbs_test [FAIL]
selftests: lost_exception_test [FAIL]
selftests: no_handler_test [FAIL]
selftests: cycles_with_mmcr2_test [FAIL]
selftests: memcmp [FAIL]
selftests: tm-resched-dscr [FAIL]
selftests: tm-vmx-unavail [FAIL]
selftests: ptrace-tm-spd-gpr [FAIL]

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-01-11 04:13 EDT-------
Lot of kernel selftests tests fail on 18.04 ( Bionic Beaver )

# uname -a
Linux ltc-wspoon12 4.14.0 #1 SMP Tue Jan 9 21:34:11 CST 2018 ppc64le ppc64le ppc64le GNU/Linux

2018-01-09 02:47:56,816 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?copy_unaligned [FAIL]
2018-01-09 02:47:56,816 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?copy_first_unaligned [FAIL]
2018-01-09 02:47:56,816 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?paste_unaligned [FAIL]
2018-01-09 02:47:56,816 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?paste_last_unaligned [FAIL]
2018-01-09 02:47:56,816 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?cp_abort [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?subpage_prot [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?reg_access_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?event_attributes_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?cycles_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?cycles_with_freeze_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?ebb_on_child_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?ebb_vs_cpu_event_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?cpu_event_vs_ebb_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?cpu_event_pinned_vs_ebb_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?task_event_vs_ebb_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?task_event_pinned_vs_ebb_test [FAIL]
2018-01-09 02:47:56,823 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?multi_ebb_procs_test [FAIL]
2018-01-09 02:47:56,824 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?multi_counter_test [FAIL]
2018-01-09 02:47:56,824 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?pmae_handling_test [FAIL]
2018-01-09 02:47:56,824 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?instruction_count_test [FAIL]
2018-01-09 02:47:56,824 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?cycles_with_mmcr2_test [FAIL]
2018-01-09 02:47:56,824 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?ebb_on_willing_child_test [FAIL]
2018-01-09 02:47:56,824 kselftest ? ? ? ?L0101 INFO | Testcase failed. Log from debug: selftests: ?back_to_back_ebbs_test [FAIL]
2018-01-09 02:47:56,824 kselftest ? ? ? ?L0101 INFO | Te...

[Manoj Iyer (manjo)]

Could you please re-test with a kernel from the Ubuntu archives and report back here if you are able to reproduce the failures.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-12 22:09 EDT-------
(In reply to comment #14)
> Could you please re-test with a kernel from the Ubuntu archives and report
> back here if you are able to reproduce the failures.

With the mailine kernel 4.16-rc1 on Ubuntu 18.04, the following issues are observed.
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.16-rc1/

not ok 1..1 selftests: copy_unaligned [FAIL]
not ok 1..3 selftests: paste_unaligned [FAIL]
not ok 1..4 selftests: paste_last_unaligned [FAIL]
not ok 1..1 selftests: cp_abort [FAIL]
not ok 1..2 selftests: subpage_prot [FAIL]
not ok 1..1 selftests: reg_access_test [FAIL]
not ok 1..2 selftests: event_attributes_test [FAIL]
not ok 1..3 selftests: cycles_test [FAIL]
not ok 1..4 selftests: cycles_with_freeze_test [FAIL]
not ok 1..5 selftests: pmc56_overflow_test [FAIL]
not ok 1..6 selftests: ebb_vs_cpu_event_test [FAIL]
not ok 1..7 selftests: cpu_event_vs_ebb_test [FAIL]
not ok 1..8 selftests: cpu_event_pinned_vs_ebb_test [FAIL]
not ok 1..9 selftests: task_event_vs_ebb_test [FAIL]
not ok 1..10 selftests: task_event_pinned_vs_ebb_test [FAIL]
not ok 1..11 selftests: multi_ebb_procs_test [FAIL]
not ok 1..12 selftests: multi_counter_test [FAIL]
not ok 1..13 selftests: pmae_handling_test [FAIL]
not ok 1..14 selftests: close_clears_pmcc_test [FAIL]
not ok 1..15 selftests: instruction_count_test [FAIL]
not ok 1..16 selftests: fork_cleanup_test [FAIL]
not ok 1..17 selftests: ebb_on_child_test [FAIL]
not ok 1..18 selftests: ebb_on_willing_child_test [FAIL]
not ok 1..19 selftests: back_to_back_ebbs_test [FAIL]
not ok 1..20 selftests: lost_exception_test [FAIL]
not ok 1..21 selftests: no_handler_test [FAIL]
not ok 1..22 selftests: cycles_with_mmcr2_test [FAIL]
not ok 1..10 selftests: tm-unavailable [FAIL]
not ok 1..11 selftests: tm-trap [FAIL]

Thanks,
Harish

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-02-22 07:10 EDT-------
Canonical, Any update on this ?

[Andrew Cloke (andrew-cloke)]

Are these self-test failures leading to any "real-world" or workload-related issues? If so, we will reprioritize.

[Dimitri John Ledkov (xnox)]

Hi, I do not believe canonical is committing to fix all the upstream/mainline selftest failures. However, if there are additional selftest regressions with the ubuntu kernel which are not seen with the mainline build, those are worth looking into. If these selftests never pass on POWER maybe it is worth pinging power kernel maintainers to mark these as expected failures? Regards, Dimitri.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-03-01 00:35 EDT-------
When run on a P8 PowerNV, following tests failed.

23:28:59 INFO | Testcase failed. Log from debug: selftests: reg_access_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: event_attributes_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: cycles_with_freeze_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: ebb_on_child_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: ebb_vs_cpu_event_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: cpu_event_vs_ebb_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: cpu_event_pinned_vs_ebb_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: task_event_vs_ebb_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: task_event_pinned_vs_ebb_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: multi_ebb_procs_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: multi_counter_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: pmae_handling_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: cycles_with_mmcr2_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: no_handler_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: ebb_on_willing_child_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: back_to_back_ebbs_test [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: tm-trap [FAIL]
23:28:59 INFO | Testcase failed. Log from debug: selftests: ptrace-tm-spd-gpr [FAIL]

Thanks,
Harish

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-03-05 05:29 EDT-------
*** Bug 165334 has been marked as a duplicate of this bug. ***

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-03-19 09:02 EDT-------
What's up on this defect? Not much updates though...

[Manoj Iyer (manjo)]

Based on comment #13, please work with upstream kernel maintainers for power to either fix failing tests or mark them as expected failures.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-04-23 04:28 EDT-------
The following are the list of failures on Ubuntu mainline 4.17-rc1 kernel, which is the same as of distro kernel.

Link: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.17-rc1/

# uname -a
Linux lep8d 4.17.0-041700rc1-generic #201804152230 SMP Mon Apr 16 03:15:58 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux

02:49:57 selftests: reg_access_test [FAIL]
02:49:57 selftests: event_attributes_test [FAIL]
02:49:57 selftests: cycles_with_freeze_test [FAIL]
02:49:57 selftests: ebb_on_child_test [FAIL]
02:49:57 selftests: ebb_vs_cpu_event_test [FAIL]
02:49:57 selftests: cpu_event_vs_ebb_test [FAIL]
02:49:57 selftests: cpu_event_pinned_vs_ebb_test [FAIL]
02:49:57 selftests: task_event_vs_ebb_test [FAIL]
02:49:57 selftests: task_event_pinned_vs_ebb_test [FAIL]
02:49:57 selftests: multi_ebb_procs_test [FAIL]
02:49:57 selftests: multi_counter_test [FAIL]
02:49:57 selftests: pmae_handling_test [FAIL]
02:49:57 selftests: cycles_with_mmcr2_test [FAIL]
02:49:57 selftests: no_handler_test [FAIL]
02:49:57 selftests: ebb_on_willing_child_test [FAIL]
02:49:57 selftests: back_to_back_ebbs_test [FAIL]
02:49:57 selftests: ptrace-tm-spd-gpr [FAIL]

- Harish

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-05-07 10:52 EDT-------
*** Bug 167297 has been marked as a duplicate of this bug. ***

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-09-18 06:28 EDT-------
Hi
I tried to run kernel self test on ubuntu18.10 still facing as some of the self test is failing

root@test:~# uname -a
Linux test 4.18.0-7-generic #8-Ubuntu SMP Tue Aug 28 18:20:56 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux
root@test:~#

selftests: ebb: reg_access_test [FAIL]
selftests: ebb: event_attributes_test [FAIL]
selftests: ebb: cycles_with_freeze_test [FAIL]
selftests: ebb: ebb_on_child_test [FAIL]
selftests: ebb: ebb_vs_cpu_event_test [FAIL]
selftests: ebb: cpu_event_vs_ebb_test [FAIL]
selftests: ebb: cpu_event_pinned_vs_ebb_test [FAIL]
selftests: ebb: task_event_vs_ebb_test [FAIL]
selftests: ebb: task_event_pinned_vs_ebb_test [FAIL]
selftests: ebb: multi_ebb_procs_test [FAIL]
selftests: ebb: multi_counter_test [FAIL]
selftests: ebb: pmae_handling_test [FAIL]
selftests: ebb: cycles_with_mmcr2_test [FAIL]
selftests: ebb: no_handler_test [FAIL]
selftests: ebb: ebb_on_willing_child_test [FAIL]
selftests: ebb: back_to_back_ebbs_test [FAIL]
selftests: ptrace: ptrace-tm-spd-gpr [FAIL]

Regards
Praveen

[Desnes A. Nunes do Rosario (desnesn)]

The cycle_test failure does not reproduce on everyrun. In order to validate you'll have to run the test a few times. Moreover, seems that ZZ lpars are easier to reproduce this.

Last, this patch fixes this bug: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=3337bf41e0dd70b4064cdf60acdfcdc2d050066c

[Thierry FAUCK (thierry-j)]

This problem seems to be fixed in more recent release - ug can be closed