jtreg fails to run under openjdk-8 due to api incompatibility with openjdk-9 or later

[Tiago Stürmer Daitx]

[Impact]
openjdk-9 introduced a few api changes that result in runtime errors when trying to run the java classes under openjdk 8 (or earlier) even when -source/-target are properly set.

The fix is to build such code using the new openjdk-9 flag "--release" instead of the "-source/-target" pair.

When running Cosmic's jtreg 4.2-b13-1 with agentvm under openjdk-8 it will fail and output a few lines such as:
stderr: Exception: java.lang.NoSuchMethodError thrown from the UncaughtExceptionHandler in thread "main"

The java.lang.NoSuchMethodError is actually caused by "java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer" which can be easily solved by building jtreg with the "--release 7" argument.

This affects the openjdk-8 build in Cosmic as the jtreg testsuites are run as part of the build. We use the test results for checking regressions from an update.

[Test Case]
For this test case we need to create an empty TEST.ROOT file and a test class to satisfy jtreg requirements of a minimum test. jtreg must also be forced to use openjdk-8.

$ sudo apt install openjdk-8-jdk jtreg
$ touch TEST.ROOT
$ cat > Hello.java << EOF
/*
 * @test
 * @bug 2718282
 * @summary Hello test
 */

public class Hello {
    public static void main(String [] args) throws Exception {
        if (true)
            System.out.println("Hello World!");
        else
            throw new Exception("??");
    }
}
EOF
$ JT_JAVA=/usr/lib/jvm/java-8-openjdk-amd64/ jtreg -va -agentvm Hello.java
[2018-11-15 23:42:34,720] Agent[0]: stderr:
[2018-11-15 23:42:34,721] Agent[0]: stderr: Exception: java.lang.NoSuchMethodError thrown from the UncaughtExceptionHandler in thread "main"
--------------------------------------------------
TEST: Hello.java
TEST JDK: /usr/lib/jvm/java-8-openjdk-amd64
[snipped]
Test results: error: 1
Report written to /tmp/JTreport/html/report.html
Results written to /tmp/JTwork
Error: Some tests failed or other problems occurred.

The expected response from jtreg does not contain the "stderr:" lines as in:
$ JT_JAVA=/usr/lib/jvm/java-8-openjdk-amd64/ jtreg -va -agentvm Hello.java
--------------------------------------------------
TEST: Hello.java
TEST JDK: /usr/lib/jvm/java-8-openjdk-amd64
[snipped]
Test results: passed: 1
Report written to /tmp/JTreport/html/report.html
Results written to /tmp/JTwork

Alternatively one can also check that openjdk-lts is not affected by the change, it should output the same before and after the fix:
$ sudo apt install openjdk-11-jdk
$ JT_JAVA=/usr/lib/jvm/java-11-openjdk-amd64/ jtreg -va -agentvm Hello.java
--------------------------------------------------
TEST: Hello.java
TEST JDK: /usr/lib/jvm/java-11-openjdk-amd64
[snipped]
Test results: passed: 1
Report written to /tmp/JTreport/html/report.html
Results written to /tmp/JTwork

[Regression Potential]
1. The --release argument is jdk9+ only, if jtreg is ever build with an older jdk it will FTBFS, so this change should only be backported on releases where the default-jdk is 9+.
2. The --release N argument is equivalent to setting -source N/-target N/-bootclasspath <N bootclasspath>, which can cause issues if the compiled classes use APIs that have been removed or are considered internal. The current jtreg version does not rely on internal classes from jdk 8 (otherwise it would FTBFS) but if a later version does that it will probably FTBFS - note that this is very unlikely as it would mean that upstream decided to break building jtreg with newer jdks.

[Other Info]
See https://github.com/apache/felix/pull/114 for a good overview on the methods that changed on OpenJDK 9 and a better description on the cause. Note that they propose patching the methods to cast ByteBuffer to Buffer when calling the affected methods, but this can be easily prevented by using the --release argument instead.

[Tiago Stürmer Daitx (tdaitx)]

Debdiff with the fix for Cosmic.

[Launchpad Janitor (janitor)]

This bug was fixed in the package jtreg - 4.2-b13-1ubuntu1

---------------
jtreg (4.2-b13-1ubuntu1) disco; urgency=medium

  * Use release instead of source/target. (Closes: #913853, LP: #1803628)

 -- Tiago Stürmer Daitx <email address hidden> Thu, 01 Nov 2018 05:36:28 +0000

[Timo Aaltonen (tjaalton)]

Hello Tiago, or anyone else affected,

Accepted jtreg into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/jtreg/4.2-b13-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Tiago Stürmer Daitx (tdaitx)]

Hi Timo, thanks for helping move this SRU ahead.

I have tested it as per the testcase in a Cosmic chroot and I can confirm that:
1. jtreg 4.2-b13-1ubuntu0.1 fixes this issue when running under openjdk-8.
2. jtreg 4.2-b13-1ubuntu0.1 does not introduces regressions when running under openjdk-11, as expected.

# apt-cache policy jtreg
jtreg:
  Installed: 4.2-b13-1ubuntu0.1
  Candidate: 4.2-b13-1ubuntu0.1
  Version table:
 *** 4.2-b13-1ubuntu0.1 400
        400 http://localhost:9999/ubuntu cosmic-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     4.2-b13-1 500
        500 http://localhost:9999/ubuntu cosmic/universe amd64 Packages

[Launchpad Janitor (janitor)]

This bug was fixed in the package jtreg - 4.2-b13-1ubuntu0.2

---------------
jtreg (4.2-b13-1ubuntu0.2) cosmic-security; urgency=medium

  * No-change re-build upload to pull fixes into the security pocket.

jtreg (4.2-b13-1ubuntu0.1) cosmic; urgency=medium

  * Use release instead of source/target. (Closes: #913853, LP: #1803628)

 -- Steve Beattie <email address hidden> Sun, 18 Nov 2018 15:21:27 -0800

[Łukasz Zemczak (sil2100)]

Hello Tiago, or anyone else affected,

Accepted jtreg into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/jtreg/4.2-b13-1ubuntu0.1~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Tiago Stürmer Daitx (tdaitx)]

I confirm that jtreg 4.2-b13-1ubuntu0.1~18.04.1 works as expected.

Please notice that jtreg 4.2-b10-1 is not affect as it is old enough to have been build with openjdk-8. Had it ever been rebuild with openjdk-lts it would have been affected.

[More Information]
$ apt policy jtreg
jtreg:
  Installed: 4.2-b13-1ubuntu0.1~18.04.1
  Candidate: 4.2-b13-1ubuntu0.1~18.04.1
  Version table:
 *** 4.2-b13-1ubuntu0.1~18.04.1 500
        500 http://us.archive.ubuntu.com/ubuntu bionic-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     4.2-b10-1 500
        500 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages

[Launchpad Janitor (janitor)]

This bug was fixed in the package jtreg - 4.2-b13-1ubuntu0.1~18.04.1

---------------
jtreg (4.2-b13-1ubuntu0.1~18.04.1) bionic; urgency=medium

  * Backport from Cosmic as openjdk-lts requires jtreg's agentvm support.
    (LP: #1814133)

jtreg (4.2-b13-1ubuntu0.1) cosmic; urgency=medium

  * Use release instead of source/target. (Closes: #913853, LP: #1803628)

jtreg (4.2-b13-1) unstable; urgency=medium

  * Team upload.
  * New upstream release
    - Refreshed the patches
  * Standards-Version updated to 4.2.0
  * Switch to debhelper level 11
  * Use salsa.debian.org Vcs-* URLs

 -- Tiago Stürmer Daitx <email address hidden> Sun, 03 Feb 2019 15:33:35 +0000