[Power 9, Power NV][WSP/Boston]Ubuntu 16.04.03 : perf test is hung at Breakpoint overflow signal handler

[bugproxy]

== Comment: # - Shriya R. Kulkarni <> - 2017-06-14 01:11:02 ==
Problem Description :
=============
perf test is hung at Breakpoint overflow signal handler.

Machine details :
==========
System : WSP , P9 baremetal.
OS : Ubuntu 16.04.03
uname -a : Linux ltc-wspoon3 4.10.0-23-generic #25~16.04.1-Ubuntu SMP Fri Jun 9 10:43:34 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Testing:
====
root@ltc-wspoon3:~# perf test
 1: vmlinux symtab matches kallsyms : Skip
 2: Detect openat syscall event : Ok
 3: Detect openat syscall event on all cpus : Ok
 4: Read samples using the mmap interface : Ok
 5: Parse event definition strings : Ok
 6: PERF_RECORD_* events & perf_sample fields : Ok
 7: Parse perf pmu format : Ok
 8: DSO data read : Ok
 9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler :^C

root@ltc-wspoon3:~# perf test -v 17
17: Breakpoint overflow signal handler :
--- start ---
test child forked, pid 3802
failed opening event 0
failed opening event 0

Attach : strace output.

Steps to reproduce :
============
1. Install perf.
2. Run perf test.
3. When system hangs , exit it by : ^c.

== Comment: # - Ravikumar B. Bangoria <> - 2017-07-13 04:44:29 ==
Shriya,

Can you please check after applying:
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=d89ba5353f301971dd7d2f9fdf25c4432728f38e

== Comment: # - Shriya R. Kulkarni <> - 2017-07-19 06:28:13 ==
Verified and Tested on Ubuntu 16.04.03:
=========================
uname -a : 4.10.0-27-generic
System : Boston

Testing without patch using distro perf:
========================
root@ltc-boston27:~# perf test
 1: vmlinux symtab matches kallsyms : Skip
 2: Detect openat syscall event : Ok
 3: Detect openat syscall event on all cpus : Ok
 4: Read samples using the mmap interface : Ok
 5: Parse event definition strings : Ok
 6: PERF_RECORD_* events & perf_sample fields : Ok
 7: Parse perf pmu format : Ok
 8: DSO data read : Ok
 9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler :^C
root@ltc-boston27:~#

Testing with Patch using distro perf :
======================
root@ltc-boston27:~/linux-hwe-4.10.0/tools/perf# ./perf test
 1: vmlinux symtab matches kallsyms : Ok
 2: Detect openat syscall event : Ok
 3: Detect openat syscall event on all cpus : Ok
 4: Read samples using the mmap interface : Ok
 5: Parse event definition strings : Ok
 6: PERF_RECORD_* events & perf_sample fields : Ok
 7: Parse perf pmu format : Ok
 8: DSO data read : Ok
 9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Ok
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler : FAILED!
18: Breakpoint overflow sampling : FAILED!
19: Number of exit events of a simple workload : Ok
20: Software clock events period values : Ok
21: Object code reading : FAILED!
22: Sample parsing : Ok
23: Use a dummy software event to keep tracking: Ok
24: Parse with no sample_id_all bit set : Ok
25: Filter hist entries : Ok
26: Lookup mmap thread : Ok
27: Share thread mg : Ok
28: Sort output of hist entries : Ok
29: Cumulate child hist entries : Ok
30: Track with sched_switch : Ok
31: Filter fds with revents mask in a fdarray : Ok
32: Add fd to a fdarray, making it autogrow : Ok
33: kmod_path__parse : Ok
34: Thread map : Ok
35: LLVM search and compile :
35.1: Basic BPF llvm compile : Skip
35.2: kbuild searching : Skip
35.3: Compile source for BPF prologue generation: Skip
35.4: Compile source for BPF relocation : Skip
36: Session topology : Ok
37: BPF filter :
37.1: Basic BPF filtering : Skip
37.2: BPF relocation checker : Skip
38: Synthesize thread map : Ok
39: Remove thread map : Ok
40: Synthesize cpu map : Ok
41: Synthesize stat config : Ok
42: Synthesize stat : Ok
43: Synthesize stat round : Ok
44: Synthesize attr update : Ok
45: Event times : Ok
46: Read backward ring buffer : Ok
47: Print cpu map : Ok
48: Probe SDT events : Ok
49: is_printable_array : Ok
50: Print bitmap : Ok
51: perf hooks : Ok
52: builtin clang support : Skip (not compiled in)
53: Test dwarf unwind : Ok

[bugproxy (bugproxy)]

Default Comment by Bridge

[Joseph Salisbury (jsalisbury)]

I built a Zesty test kernel with a pick of commit d89ba5353f301. The test kernel can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1711075

Can you test this kernel and see if it resolves this bug?

Thanks in advance!

[Manoj Iyer (manjo)]

IBM, could you pls test the kernel provided by Joe and report back here?

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-09-27 03:26 EDT-------
Verified on the kernel : http://kernel.ubuntu.com/~jsalisbury/lp1711075/

System : Witherspoon + DD1.0
uname -a : Linux ltc-wspoon3 4.10.0-32-generic #36~lp1711075 SMP Wed Aug 16 14:09:56 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

root@ltc-wspoon3:/usr/lib/linux-tools/4.10.0-36-generic# ./perf test
1: vmlinux symtab matches kallsyms : Skip
2: Detect openat syscall event : Ok
3: Detect openat syscall event on all cpus : Ok
4: Read samples using the mmap interface : Ok
5: Parse event definition strings : Ok
6: PERF_RECORD_* events & perf_sample fields : Ok
7: Parse perf pmu format : Ok
8: DSO data read : Ok
9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler : FAILED!
18: Breakpoint overflow sampling : FAILED!
19: Number of exit events of a simple workload : Ok
20: Software clock events period values : Ok
21: Object code reading : FAILED!
22: Sample parsing : Ok
23: Use a dummy software event to keep tracking: Ok
24: Parse with no sample_id_all bit set : Ok
25: Filter hist entries : Ok
26: Lookup mmap thread : Ok
27: Share thread mg : Ok
28: Sort output of hist entries : Ok
29: Cumulate child hist entries : Ok
30: Track with sched_switch : Ok
31: Filter fds with revents mask in a fdarray : Ok
32: Add fd to a fdarray, making it autogrow : Ok
33: kmod_path__parse : Ok
34: Thread map : Ok
35: LLVM search and compile :
35.1: Basic BPF llvm compile : Skip
35.2: kbuild searching : Skip
35.3: Compile source for BPF prologue generation: Skip
35.4: Compile source for BPF relocation : Skip
36: Session topology : Ok
37: BPF filter :
37.1: Basic BPF filtering : Skip
37.2: BPF prologue generation : Skip
37.3: BPF relocation checker : Skip
38: Synthesize thread map : Ok
39: Remove thread map : Ok
40: Synthesize cpu map : Ok
41: Synthesize stat config : Ok
42: Synthesize stat : Ok
43: Synthesize stat round : Ok
44: Synthesize attr update : Ok
45: Event times : Ok
46: Read backward ring buffer : Ok
47: Print cpu map : Ok
...

[Joseph Salisbury (jsalisbury)]

Commit d89ba5353f301 was SRU'd for bug 1706033, so I'll mark this bug as a duplicate of that bug.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-12-13 23:26 EDT-------
Verified with latest kernel on Ubuntu 16.04.03 and it works as expected
=============================================

root@ltc-wspoon12:/usr/lib/linux-hwe-tools-4.10.0-42# ./perf test
1: vmlinux symtab matches kallsyms : Skip
2: Detect openat syscall event : Ok
3: Detect openat syscall event on all cpus : Ok
4: Read samples using the mmap interface : Ok
5: Parse event definition strings : Ok
6: PERF_RECORD_* events & perf_sample fields : Ok
7: Parse perf pmu format : Ok
8: DSO data read : Ok
9: DSO data cache : Ok
10: DSO data reopen : Ok
11: Roundtrip evsel->name : Ok
12: Parse sched tracepoints fields : Ok
13: syscalls:sys_enter_openat event fields : Ok
14: Setup struct perf_event_attr : Skip
15: Match and link multiple hists : Ok
16: 'import perf' in python : Ok
17: Breakpoint overflow signal handler : FAILED!
18: Breakpoint overflow sampling : FAILED!
19: Number of exit events of a simple workload : Ok
20: Software clock events period values : Ok
21: Object code reading : FAILED!
22: Sample parsing : Ok
23: Use a dummy software event to keep tracking: Ok
24: Parse with no sample_id_all bit set : Ok
25: Filter hist entries : Ok
26: Lookup mmap thread : Ok
27: Share thread mg : Ok
28: Sort output of hist entries : Ok
29: Cumulate child hist entries : Ok
30: Track with sched_switch : Ok
31: Filter fds with revents mask in a fdarray : Ok
32: Add fd to a fdarray, making it autogrow : Ok
33: kmod_path__parse : Ok
34: Thread map : Ok
35: LLVM search and compile :
35.1: Basic BPF llvm compile : Skip
35.2: kbuild searching : Skip
35.3: Compile source for BPF prologue generation: Skip
35.4: Compile source for BPF relocation : Skip
36: Session topology : Ok
37: BPF filter :
37.1: Basic BPF filtering : Skip
37.2: BPF prologue generation : Skip
37.3: BPF relocation checker : Skip
38: Synthesize thread map : Ok
39: Remove thread map : Ok
40: Synthesize cpu map : Ok
41: Synthesize stat config : Ok
42: Synthesize stat : Ok
43: Synthesize stat round : Ok
44: Synthesize attr update : Ok
45: Event times : Ok
46: Read backward ring buffer : Ok
47: Print cpu map : Ok
48: Probe SDT events : Skip
49: is_printable_array : Ok
50: Print ...