[18.04/18.10] File libperf-jvmti.so is missing in linux-tools-common deb on Ubuntu

Bug #1761379 reported by bugproxy on 2018-04-05
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
High
Canonical Kernel Team
linux (Ubuntu)
High
Canonical Kernel Team
Artful
High
Canonical Kernel Team
Bionic
High
Canonical Kernel Team
Cosmic
High
Canonical Kernel Team
Disco
High
Canonical Kernel Team

Bug Description

[Impact]
File libperf-jvmti.so is missing in linux-tools-common deb making it impossible to use perf for the JVM JITed methods.

[Test case]
$ sudo perf record -k 1 -e instructions:u ./java -agentpath:/usr/lib/linux-tools-5.0.0-8/libperf-jvmti.so crc32
$ sudo perf inject -i ./perf.data -j -o ./perf.data.jitted
$ sudo perf report -f -i ./perf.data.jitted

[Fix]
Include java build dependencies and install the library into linux-tools package.

[Regression potential]
Small regression potential, an extra file is distributed and is not automatically linked to anything. It could impact the build, which was tested.

---Problem Description---
File libperf-jvmti.so is missing in linux-tools-common deb making it impossible to use perf for the JVM JITed methods

---uname output---
linux-image-4.13.0-36-generic

Machine Type = not relevant

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 File libperf-jvmti.so is missing in linux-tools-common deb provided for Ubuntu 17.10 making it impossible to use perf for the JVM JITed methods. I also checked if the file is available on launchpad (https://launchpad.net/ubuntu/+source/linux) for Bionic Beaver proposed (main) at it's also absent there:

gromero@ltc-wspoon3:~/download$ dpkg -c linux-tools-common_4.15.0-13.14_all.deb | fgrep jvm
gromero@ltc-wspoon3:~/download$ dpkg -c linux-tools-4.15.0-13-generic_4.15.0-13.14_ppc64el.deb | fgrep jvm

I do see the file in tools/perf/jvmti dir in the source .tar.gz, but apparently it's no being packaged in any .deb file?

Thanks.

Userspace tool common name: perf

The userspace tool has the following bit modes: 64-bit

Userspace tool obtained from project website: na

bugproxy (bugproxy) on 2018-04-05
tags: added: architecture-ppc64le bugnameltc-166440 severity-medium targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → kernel-package (Ubuntu)
affects: kernel-package (Ubuntu) → linux (Ubuntu)
summary: File libperf-jvmti.so is missing in linux-tools-common deb on Ubuntu
- 17.10 and above
+ 17.10 and 18.04
Changed in ubuntu-power-systems:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)

This probably affects one of the packages linux-tools-* that I can't mark, hence using linux (kernel) for now.
It looks to me that we don't ship "libperf-jvmti.so" with any package (and never have been).
But letting the kernel-team leave a comment on this ...

Manoj Iyer (manjo) on 2018-04-05
Changed in linux (Ubuntu):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Canonical Kernel Team (canonical-kernel-team)
importance: Undecided → Medium
tags: added: triage-g
Changed in linux (Ubuntu):
status: New → Triaged
Changed in linux (Ubuntu Artful):
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
tags: added: artful bionic kernel-da-key

------- Comment From <email address hidden> 2018-05-07 07:16 EDT-------
Hello,

So have the kernel team confirmed that issue?

Is the absent file (libperf-jvmti.so) marked to be included in any packaged already?

Thank you.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-05-11 11:07 EDT-------
Hi Canonical,

Now it's Triaged, what's your thought on that issue?

Thank you.

Regards,
Gustavo

Support for artful will end soon (July 2018) therefore marking this as a won't fix.

Changed in linux (Ubuntu Artful):
status: Triaged → Won't Fix
summary: - File libperf-jvmti.so is missing in linux-tools-common deb on Ubuntu
- 17.10 and 18.04
+ [18.04/18.10] File libperf-jvmti.so is missing in linux-tools-common deb
+ on Ubuntu

------- Comment From <email address hidden> 2018-06-25 11:18 EDT-------
(In reply to comment #9)
> Support for artful will end soon (July 2018) therefore marking this as a
> won't fix.

OK. But how about 18.04? Could you please at least provide a clarification/confirmation about why that lib is not included on any package related to the kernel package (on any release)? Should I open a different bug to track its inclusion on Bionic? Please advise.

Thank you.

Joseph Salisbury (jsalisbury) wrote :

I'm investigating this for Bionic. I'll post an update shortly.

Changed in linux (Ubuntu Bionic):
status: Triaged → In Progress
Changed in linux (Ubuntu):
status: Triaged → In Progress
Changed in linux (Ubuntu Bionic):
assignee: Canonical Kernel Team (canonical-kernel-team) → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu):
assignee: Canonical Kernel Team (canonical-kernel-team) → Joseph Salisbury (jsalisbury)
Changed in ubuntu-power-systems:
status: Triaged → In Progress
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-08-27 09:00 EDT-------
Hi Salibury,

Any update on that?

Thanks.

Joseph Salisbury (jsalisbury) wrote :

I built a test kernel with NO_JVMTI disabled in tools/perf/Makefile.config.

The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1761379

Can you test this kernel and see if it resolves this bug?

Note about installing test kernels:
• If the test kernel is prior to 4.15(Bionic) you need to install the linux-image and linux-image-extra .deb packages.
• If the test kernel is 4.15(Bionic) or newer, you need to install the linux-modules, linux-modules-extra and linux-image-unsigned .deb packages.

Thanks in advance!

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-10-10 17:08 EDT-------
Hi Joseph,

It looks like 'perf' binary is missing in linux-tools-common in the pointed out packages:

root@gromero16:~/perf/lp1761379# dpkg -c linux-tools-common_4.15.0-34.38~lp1761379_all.deb
drwxr-xr-x root/root 0 2018-08-27 10:45 ./
drwxr-xr-x root/root 0 2018-08-27 10:45 ./usr/
drwxr-xr-x root/root 0 2018-08-27 10:45 ./usr/share/
drwxr-xr-x root/root 0 2018-08-27 10:45 ./usr/share/doc/
drwxr-xr-x root/root 0 2018-08-27 10:45 ./usr/share/doc/linux-tools-common/
-rw-r--r-- root/root 133835 2018-08-27 10:45 ./usr/share/doc/linux-tools-common/changelog.Debian.gz
-rw-r--r-- root/root 1292 2018-08-27 10:45 ./usr/share/doc/linux-tools-common/copyright
root@gromero16:~/perf/lp1761379#

I also could not find it and any jvmti.so in the remaining packages.

Would you mind to check that please?

Thank you.

Joseph Salisbury (jsalisbury) wrote :

I'll check into why those are missing and should have more packages ready shortly.

Joseph Salisbury (jsalisbury) wrote :

I built a v2 test kernel. All the .debs can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1761379/v2

Can you test this kernel and see if it resolves this bug?

Changed in ubuntu-power-systems:
importance: Medium → High
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-10-11 12:40 EDT-------
(In reply to comment #18)
> I built a v2 test kernel. All the .debs can be downloaded from:
>
> http://kernel.ubuntu.com/~jsalisbury/lp1761379/v2
>
> Can you test this kernel and see if it resolves this bug?

Hi Joseph,

Thanks for providing v2.

Unfortunately it looks like 'perf' is still missing in linux-tools-common package:

root@gromero16:~/perf/~jsalisbury/lp1761379/v2# dpkg -c linux-tools-common_4.15.0-36.40~lp1761379v2_all.deb
drwxr-xr-x root/root 0 2018-09-24 10:08 ./
drwxr-xr-x root/root 0 2018-09-24 10:08 ./usr/
drwxr-xr-x root/root 0 2018-09-24 10:08 ./usr/share/
drwxr-xr-x root/root 0 2018-09-24 10:08 ./usr/share/doc/
drwxr-xr-x root/root 0 2018-09-24 10:08 ./usr/share/doc/linux-tools-common/
-rw-r--r-- root/root 140984 2018-09-24 10:08 ./usr/share/doc/linux-tools-common/changelog.Debian.gz
-rw-r--r-- root/root 1292 2018-09-24 10:08 ./usr/share/doc/linux-tools-common/copyright
root@gromero16:~/perf/~jsalisbury/lp1761379/v2#

I do see it in an AMD64 .deb file but it's not suitable for PPC64 naturally:

root@gromero16:~/perf/~jsalisbury/lp1761379/v2# dpkg -c ./linux-tools-4.15.0-36_4.15.0-36.40~lp1761379v2_amd64.deb | fgrep \/perf
-rwxr-xr-x root/root 5549944 2018-10-11 08:09 ./usr/lib/linux-tools-4.15.0-36/perf

I also still cant' find any '*jvmti*' in v2 packages.

So could you check it again please?

Thanks.

Regards,
Gustavo

Changed in linux (Ubuntu Bionic):
importance: Medium → High
Changed in linux (Ubuntu Cosmic):
importance: Medium → High
Joseph Salisbury (jsalisbury) wrote :

It's possible this is happening because I'm cross compiling. I'll build the packages in a PPA, so it is a native compile.

The PPA will be located at:
https://launchpad.net/~jsalisbury/+archive/ubuntu/1761379

It is currently building and should be done in an hour or two.

Changed in ubuntu-power-systems:
status: In Progress → Incomplete
Andrew Cloke (andrew-cloke) wrote :

Marking as "incomplete" while awaiting IBM's test results from the PPA.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-10-15 17:33 EDT-------
Hi Joseph,

(In reply to comment #20)
> It's possible this is happening because I'm cross compiling. I'll build the
> packages in a PPA, so it is a native compile.
>
> The PPA will be located at:
> https://launchpad.net/~jsalisbury/+archive/ubuntu/1761379
>
> It is currently building and should be done in an hour or two.

I'm now able to find the 'perf' binary and it works correctly.

However I'm still unable to find the ' libperf-jvmti.so`.

Could you please check it again and confirm in which package exactly Canonical will provide such a file?

Thanks and best regards,
Gustavo

Changed in ubuntu-power-systems:
status: Incomplete → In Progress
Joseph Salisbury (jsalisbury) wrote :

I create one more PPA at the same locate, but with v3 in the name:
 https://launchpad.net/~jsalisbury/+archive/ubuntu/1761379

Just to confirm, you have java on your test system?

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-10-18 09:14 EDT-------
(In reply to comment #23)
> I create one more PPA at the same locate, but with v3 in the name:
> https://launchpad.net/~jsalisbury/+archive/ubuntu/1761379

Thanks. I'll gitve it a try today.

> Just to confirm, you have java on your test system?

Yes. OpenJDK more specifically (8 and 11 downstream, upstream 12, for instance).

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-10-18 16:11 EDT-------
Hi Joseph,

(In reply to comment #23)
> I create one more PPA at the same locate, but with v3 in the name:
> https://launchpad.net/~jsalisbury/+archive/ubuntu/1761379

My understanding is that the PPA ppc64el build broke. There are no binary files available for v3:

Number of packages:
1 source package (157.0 MiB)
0 binary packages (0 bytes)

Please, check the presence of the JVM jvmti agent (libperf-jvmti.so file) in the packages first before pointing me out to the new generated packages and also please point me out the individual package that contains that file since I don't follow closely how Canonical usually splits such a files in the various generated files.

Thank you.

Changed in linux (Ubuntu Bionic):
assignee: Joseph Salisbury (jsalisbury) → Canonical Kernel Team (canonical-kernel-team)
Changed in linux (Ubuntu):
assignee: Joseph Salisbury (jsalisbury) → Canonical Kernel Team (canonical-kernel-team)
Changed in linux (Ubuntu Cosmic):
assignee: Joseph Salisbury (jsalisbury) → Canonical Kernel Team (canonical-kernel-team)
Changed in linux (Ubuntu Artful):
importance: Medium → High
tags: added: cosmic disco
Manoj Iyer (manjo) wrote :

This bug is awaiting a decision from the kernel team because it adds a dependency on Java for kernel builds.

Changed in ubuntu-power-systems:
status: In Progress → Incomplete
Manoj Iyer (manjo) on 2018-11-28
Changed in ubuntu-power-systems:
status: Incomplete → In Progress
Frank Heimes (frank-heimes) wrote :

quick update:
Looks like a way was found (by apw) to modify the dependencies to become able to finally ship libperf-jvmti.so, but this needs to be discussed with the Ubuntu archive team first - which is the next step here...

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-12-21 12:13 EDT-------
Hi Frank,

Thanks for the update.

Looking forward for the final resolution :)

Happy Holidays.

Regards,
Gustavo

Manoj Iyer (manjo) on 2019-03-04
tags: added: triage-r
removed: triage-g
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-03-15 00:38 EDT-------
Any update on this bug ?

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-03-18 14:24 EDT-------
Frankly, it's quite upsetting not hearing from Canonical after so much time. I do understand that kernel team might frown upon adding a Java stuff as a dep for building the kernel packages and that any other similar conundrum might exist. But it seems Canonical is not paying much attention at all to that issue which, besides PPC64, also applies to other architectures.

I could even try to help working that upstream on perf side but without even knowing what's the Canonical plan for that bug it's moot because who knows exactly what's the hindrance here.

Andrew Cloke (andrew-cloke) wrote :

I sincerely apologise for the delay. I'll do my utmost to get you a response as soon as possible.

Terry Rudd (terrykrudd) wrote :

The canonical engineering team has been tracking this issue, but apologies again for the delay in responding to the request for technical discussion on this bug.

We have escalated the priority of this specific issue, and intend to get you a technical response by the end of the week.

I am doing a test build in order to try to provide this package, and look at how many dependencies changes that introduces.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-03-20 12:22 EDT-------
Thanks a lot Andrew, Terry, and Cascardo.

If there is anything I can help on the PPC64 side, please let me know.

Best regards,
Gustavo

Hi, Gustavo.

I would like to ask if using the same location as the real perf binary itself would be fine for your needs.

That is, a symlink at /usr/lib/linux-tools/`uname -r`/libperf_jvmti.so pointing at /usr/lib/linux-tools-SRCVERSION-ABINUMBER/libperf_jvmti.so.

Thanks.
Cascardo.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-03-21 16:33 EDT-------
Hi Cascardo,

(In reply to comment #36)
> I would like to ask if using the same location as the real perf binary
> itself would be fine for your needs.
>
> That is, a symlink at /usr/lib/linux-tools/`uname -r`/libperf_jvmti.so
> pointing at /usr/lib/linux-tools-SRCVERSION-ABINUMBER/libperf_jvmti.so.

That location and symlink are perfectly fine for me.

Just a nit: you probably meant libperf-jvmti.so (with dash) instead of libperf_jvmti.so, right? If not, although that won't impact much in practice, I would like to keep the upstream default file name, with dash.

Thanks a lot & best regards,
Gustavo

Hi, Gustavo.

Yes, we are keeping the filename as upstream, with a dash.

Can you try the kernel at ppa:cascardo/linux, and see if that works for you?

Best regards.
Cascardo.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-03-26 07:18 EDT-------
(In reply to comment #39)
> Gustavo, Could you test please

Yes. Please, give me one more today.

Thanks,
Gustavo

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-03-28 23:03 EDT-------
(In reply to comment #38)
> Can you try the kernel at ppa:cascardo/linux, and see if that works for you?

Hi Cascardo. It looks good. I'm able to get a jit profile using the agent found in linux-tools-5.0.0-8-generic with:

$ sudo perf record -k 1 -e instructions:u ./java -agentpath:/usr/lib/linux-tools-5.0.0-8/libperf-jvmti.so crc32
$ sudo perf inject -i ./perf.data -j -o ./perf.data.jitted
$ sudo perf report -f -i ./perf.data.jitted

Thank you.

Cheers,
Gustavo

Hi, Gustavo.

We have just missed the kernel freeze. So, we will need to SRU this change. I am still waiting for your feedback, before I submit this for review.

Thank you.
Cascardo.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-04-04 08:50 EDT-------
Hi Cascardo!

(In reply to comment #42)
> Hi, Gustavo.
>
> We have just missed the kernel freeze. So, we will need to SRU this change.
> I am still waiting for your feedback, before I submit this for review.

I think it's fine to miss the freeze and postpone it.

That said, are you waiting any additional feedback besides the one I posted on 03/28?

--
Hi Cascardo. It looks good. I'm able to get a jit profile using the agent found in linux-tools-5.0.0-8-generic with:

$ sudo perf record -k 1 -e instructions:u ./java -agentpath:/usr/lib/linux-tools-5.0.0-8/libperf-jvmti.so crc32
$ sudo perf inject -i ./perf.data -j -o ./perf.data.jitted
$ sudo perf report -f -i ./perf.data.jitted

Thank you.

Cheers,
Gustavo
--

Please do let me know if you are expecting any additional testing, but so far from my side status is: ship it! :)

Thanks a lot.
Gustavo

description: updated
description: updated

Hi, Gustavo.

The patch has been sent to the mailing list, but this will have to wait for release, as this has impact/considerations for the derivative kernels. Meanwhile, I am asking for reviews so we can discuss any other issues with the fixes as early as we can.

Cascardo.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers