Java crashes on s390x VM with SIGILL/ILL_PRVOPC at '__kernel_getcpu+0x8'

[John Neffenger]

Host environment

- Operating system: Ubuntu 20.04.3 LTS Desktop
- OS/kernel version: Linux tower 5.11.0-37-generic #41~20.04.2-Ubuntu
    SMP Fri Sep 24 09:06:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
- Architecture: amd64
- QEMU flavor: qemu-system-s390x
- QEMU version: QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17)
- QEMU command line: See attached file 'command-line.txt'

Emulated/Virtualized environment

- Operating system: Ubuntu 20.04.3 LTS Server
- OS/kernel version: Linux s390x-focal 5.4.0-88-generic #99-Ubuntu
    SMP Thu Sep 23 17:27:44 UTC 2021 s390x s390x s390x GNU/Linux
- Architecture: s390x

Description of problem

Java crashes as shown below:

$ java --version
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGILL (0x4) at pc=0x000003ff9f5fe6f4, pid=6789, tid=6818
#
# JRE version: (17.0+35) (build )
# Java VM: OpenJDK 64-Bit Server VM (17+35-snap, mixed mode, sharing,
# tiered, compressed oops, compressed class ptrs, g1 gc, linux-s390x)
# Problematic frame:
# C [linux-vdso64.so.1+0x6f8] __kernel_getcpu+0x8
#
# Core dump will be written. Default location: core.6789 (may not
# exist)
#
# An error report file with more information is saved as:
# /home/ubuntu/src/hs_err_pid6789.log
#
#
Aborted (core dumped)

Steps to reproduce

Run any Java program to reproduce the problem.

Because the 'openjdk' packages in Ubuntu run the 'java' command during installation, they hit the same error and fail to install. As an alternative, you can install the OpenJDK Snap package for the 's390x' architecture as follows:

  $ sudo snap install openjdk

The OpenJDK Snap package has been tested to work on a real IBM/S390 8561 system, namely the IBM LinuxONE III LT1 at Marist College:

  Marist College Installs World’s First IBM LinuxONE III™
  https://www.marist.edu/-/marist-first-linuxone-iii

Additional information

See the following attached files:

command-line.txt - the command-line used to start the virtual machine
hs_err_pid6789.log - the log file resulting from 'java --version'

[John Neffenger (jgneff)]

I just tried the same s390x virtual machine under QEMU version 6.0.0 in the Ubuntu 21.10 Beta release, and the error still occurs. My system information is shown below:

$ qemu-system-s390x --version
QEMU emulator version 6.0.0 (Debian 1:6.0+dfsg-2expubuntu1)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=21.10
DISTRIB_CODENAME=impish
DISTRIB_DESCRIPTION="Ubuntu Impish Indri (development branch)"

$ uname -a
Linux ubuntu 5.13.0-16-generic #16-Ubuntu SMP Fri Sep 3 14:53:27 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

[Thomas Huth (th-huth)]

There were some fixes in QEMU v6.1. Please try that one to see whether it solves your problem, too.

[John Neffenger (jgneff)]

The error still occurs on the same s390x virtual machine with QEMU version 6.1.

After building QEMU 6.1, I started the virtual machine as follows:

  $ ../qemu-6.1.0/build/s390x-softmmu/qemu-system-s390x \
    -m 2048 -drive if=virtio,file=s390x-focal.qcow2,cache=none

The QEMU 6.1.0 monitor shows:

  (qemu) info version
  6.1.0

On the guest, I installed the new 'openjdk-17-jre-headless' OpenJDK 17 package in Ubuntu 20.04. Although the installation fails because of this bug, the 'java' command is available after the attempt to install the package.

  $ sudo apt-get install openjdk-17-jre-headless

The error in this bug report still occurs:

  $ /usr/lib/jvm/java-17-openjdk-s390x/bin/java --version
  #
  # A fatal error has been detected by the Java Runtime Environment:
  #
  # SIGILL (0x4) at pc=0x000003ff9e4fe6f4, pid=2883, tid=2884
  #
  # JRE version: (17.0+35) (build )
  # Java VM: OpenJDK 64-Bit Server VM (17+35-Ubuntu-120.04, mixed
  # mode, sharing, tiered, compressed oops, compressed class ptrs,
  # serial gc, linux-s390x)
  # Problematic frame:
  # C [linux-vdso64.so.1+0x6f8] __kernel_getcpu+0x8
  #
  # Core dump will be written. Default location: Core dumps may
  # be processed with "/usr/share/apport/apport %p %s %c %d %P %E"
  # (or dumping to /home/ubuntu/core.2883)
  #
  # An error report file with more information is saved as:
  # /home/ubuntu/hs_err_pid2883.log
  #
  #
  Aborted (core dumped)

I'll also attach the corresponding log file.

For reference, the guest shows the following CPU information under QEMU 6.1.

  $ lscpu
  Architecture: s390x
  CPU op-mode(s): 32-bit, 64-bit
  Byte Order: Big Endian
  CPU(s): 1
  On-line CPU(s) list: 0
  Thread(s) per core: 1
  Core(s) per socket: 1
  Socket(s) per book: 1
  Book(s) per drawer: 1
  Drawer(s): 1
  NUMA node(s): 1
  Vendor ID: IBM/S390
  Machine type: 3906
  BogoMIPS: 13370.00
  Hypervisor: KVM/Linux
  Hypervisor vendor: KVM
  Virtualization type: full
  Dispatching mode: horizontal
  NUMA node0 CPU(s): 0
  ...

[John Neffenger (jgneff)]

Now that I can reproduce the problem on the latest QEMU 6.1, I created the following issue with the upstream project:

Java crashes on s390x VM with SIGILL/ILL_PRVOPC at '__kernel_getcpu+0x8'
https://gitlab.com/qemu-project/qemu/-/issues/655

[Christian Ehrhardt  (paelzer)]

Thank you John,
emulation on s390x is imperfect at times - the JVM might just use a non emulated instruction or something like that. I've linked your upstream report (thanks) here to that the bug automatically gets updates from there.

Do we already happen to know at which (maybe always the same) instruction sequence this crashes?

[John Neffenger (jgneff)]

For more history on this issue, please see the following Debian bug report created in June 2021:

openjdk-11-jre-headless: running java in qemu s390 gives a SIGILL at C [linux-vdso64.so.1+0x6f8] __kernel_getcpu+0x8
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990417

That report links to a simple C program that can reproduce the error.