linux-image-4.8.0-53-generic on 64bit Ubuntu installations causes problems with 32bit Java (e.g., ThreadLocal problems)

[Uwe Schindler]

I am running the famous Jenkins server that tests Oracle's Java releases with Apache Lucene. When migrating to new hardware and installing Ubuntu 16.04.2, the tests failed with strange errors:

The system is Ubuntu 16.04.2 (64bits) with linux-image-4.8.0-53-generic. Java 7, Java 8 and Java 9 preview releases run without any problems if it is 64 bit versions (downloaded from Oracle's homepage). We don't run OpenJDK from Ubuntu package reporsitories, as we have many different combinations of JVMs running tests (also inside VirtualBox).

If tests are using 32 bit JVM versions, the 32 bit compilation fails in most cases. For 32 bit support we use Multiarch i386 packages and installed libc6. This was same setup as Ubuntu 14.04 with Kernel 4.4. Sometimes the compilation with Apache Ant works, but then it fails horribly during running tests. The most often seen stack trace is:

jenkins@serv1:~/tools/java/32bit/jdk1.8.0_131/bin$ ./java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) Server VM (build 25.131-b11, mixed mode)

> java.lang.NullPointerException
> at java.lang.ThreadLocal$ThreadLocalMap.getEntryAfterMiss(ThreadLocal.java:440)
> at java.lang.ThreadLocal$ThreadLocalMap.getEntry(ThreadLocal.java:419)
> at java.lang.ThreadLocal$ThreadLocalMap.access$000(ThreadLocal.java:298)
> at java.lang.ThreadLocal.get(ThreadLocal.java:163)
> at org.apache.tools.ant.property.LocalProperties.current(LocalProperties.java:70)
> at org.apache.tools.ant.property.LocalProperties.enterScope(LocalProperties.java:89)

jenkins@serv1:~/tools/java/32bit/jdk-9-ea+168/bin$ ./java -version
java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+168)
Java HotSpot(TM) Server VM (build 9-ea+168, mixed mode)

> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([716594A5FB7EBE0D:B81AAD979BE8FE98]:0)
> at java.base/java.lang.ThreadLocal$ThreadLocalMap.getEntryAfterMiss(ThreadLocal.java:442)
> at java.base/java.lang.ThreadLocal$ThreadLocalMap.getEntry(ThreadLocal.java:418)
> at java.base/java.lang.ThreadLocal$ThreadLocalMap.access$000(ThreadLocal.java:298)
> at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:163)
> at org.apache.lucene.util.CloseableThreadLocal.get(CloseableThreadLocal.java:78)
> at org.apache.lucene.index.SegmentReader.getFieldsReader(SegmentReader.java:234)

I was not sure what exactly this is caused by and I found out: When updating to the recent linux-hwe-edge kernel (4.10.0-21-generic #23~16.04.1-Ubuntu) the problems are gone, it works as fine as in Ubuntu 14.04 with 4.4 kernel.

So it is not an libc issue or any other hardware-related problem. I think it is caused by some wrongly backported patch. My first guess is some 32bit fix for pthreads, I just don't find the bug report anymore (launchpad search down).

To reproduce this, use the following approach:
- install Ubuntu 16.04.2 with HWE stack on 64 bit platform
- install the i386 multiarch and a basic libc run run 32 bit Java: apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
- download a 32 bit package of Java 8u131 and untar it
- try to compile some larger Java program, e.g. Apache Lucene!

The workaround is currently to go back to 4.4.0 kernel or install the edge HWE (kernel 4.10.0).

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1693854

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.