[Ubuntu] qemu - backport diag308 stable exception fix

Bug #1812384 reported by bugproxy
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
Medium
Christian Ehrhardt 
qemu (Ubuntu)
Fix Released
Undecided
Skipper Bug Screeners
Bionic
Fix Released
Undecided
Unassigned
Cosmic
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

 * diag308 doesn't behave correctly (hw error instead of program
   interrupt)

 * Solution; Backport upstream accepted fix

[Test Case]

 * Download the test image in comment #3

 * Run the test kernel in qemu (on s390x) like:
   qemu-system-s390x -nodefaults -nographic -machine s390-ccw-virtio,accel=kvm -chardev stdio,id=con0 -device sclpconsole,chardev=con0 -kernel diag308.elf

  * Should report a proper abort/exception like
    ABORT: diag308: Unexpected program interrupt: 6 at 0x134be, ilen 4
    Without the fix it is a hard crash

[Regression Potential]

 * In both cases the guest breaks, with the fix in place it breaks with a
   better message and the guest kernel could actually handle the program
   interrupt if it wanted.
   Since the old case was a full crash of the qemu userspace and this si
   the only code path modified I can't see potential regressions triggered
   by this change.

[Other Info]

 * I doubt we call a special test kernel a "real case" for the SRU, but
   future guests (e.g. Ubuntu 20.04) might trigger subcodes not yet
   known to older qmeu (e.g. in 18.04) so adding this is a good safety
   net for any such upcoming changes.

---

The presence of subcodes is indicated by checking some query subcode.
For example 6 will indicate that 3-6 are available. So future systems
might call new subcodes to check for new features. This should not
trigger a hw error, instead we return the architectured specification
exception.

Commit has been accepted by maintainer:

https://github.com/cohuck/qemu/commit/f0646afaccf6bc1b196fc068eef75938db303814

Commit ID will be provided once it's upstream:

Related branches

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-174838 severity-medium targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → qemu (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Christian Ehrhardt  (paelzer)
tags: added: qemu-19.04
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Changed in qemu (Ubuntu):
status: New → In Progress
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Janosh,
the patch is fine to be added to the new qemu in the next Ubuntu Release.
To be able to add it to Bionic/Cosmic as requested I'd need some sort of a test case that can verify the fix working in the backported version as well.
308 was re-IPL, would you have a test case to share that we could use to test unknown diag 308 subcodes that trigger the new code?

Revision history for this message
bugproxy (bugproxy) wrote : Diag 308 test case

------- Comment (attachment only) From <email address hidden> 2019-01-22 04:27 EDT-------

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2019-01-22 04:31 EDT-------
The attached test case is a minimal kernel generated via the kvm unit test repo.
It has to be executed via qemu kernel boot, the command and expected output in both cases are below.
The path to qemu at the start of the command might need adjustment.

Command:
/usr/local/bin/qemu-system-s390x -nodefaults -nographic -machine s390-ccw-virtio,accel=kvm -chardev stdio,id=con0 -device sclpconsole,chardev=con0 -kernel s390x/diag308.elf

Output on unfixed qemu:
qemu: hardware error: Unhandled diag308 subcode 2
CPU #0:
PSW=mask 0000100180000000 addr 00000000000134be cc 00
R00=0000000000000000 R01=0000000000000000 R02=0000000000000002 R03=0000000000000000
R04=00000000000134ae R05=000000000003fdd8 R06=0000000000000000 R07=0000000000000000
R08=0000000000000000 R09=0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=00000000000134ae R15=000000000003fec0
V00=00000000000000000000000000000000 V01=00000000000000000000000000000000
V02=00000000000000000000000000000000 V03=00000000000000000000000000000000
V04=00000000000000000000000000000000 V05=00000000000000000000000000000000
V06=00000000000000000000000000000000 V07=00000000000000000000000000000000
V08=00000000000000000000000000000000 V09=00000000000000000000000000000000
V10=00000000000000000000000000000000 V11=00000000000000000000000000000000
V12=00000000000000000000000000000000 V13=00000000000000000000000000000000
V14=00000000000000000000000000000000 V15=00000000000000000000000000000000
V16=00000000000000000000000000000000 V17=00000000000000000000000000000000
V18=00000000000000000000000000000000 V19=00000000000000000000000000000000
V20=00000000000000000000000000000000 V21=00000000000000000000000000000000
V22=00000000000000000000000000000000 V23=00000000000000000000000000000000
V24=00000000000000000000000000000000 V25=00000000000000000000000000000000
V26=00000000000000000000000000000000 V27=00000000000000000000000000000000
V28=00000000000000000000000000000000 V29=00000000000000000000000000000000
V30=00000000000000000000000000000000 V31=00000000000000000000000000000000
C00=0000000000040000 C01=0000000000000000 C02=0000000000000000 C03=0000000000000000
C04=0000000000000000 C05=0000000000000000 C06=0000000000000000 C07=0000000000000000
C08=0000000000000000 C09=0000000000000000 C10=0000000000000000 C11=0000000000000000
C12=0000000000000000 C13=0000000000000000 C14=00000000c2000000 C15=0000000000000000

QEMU Aborted

Command:
/usr/local/bin/qemu-system-s390x -nodefaults -nographic -machine s390-ccw-virtio,accel=kvm -chardev stdio,id=con0 -device sclpconsole,chardev=con0 -kernel s390x/diag308.elf

Output on fixed qemu:
ABORT: diag308: Unexpected program interrupt: 6 at 0x134be, ilen 4

SUMMARY: 0 tests

EXIT: STATUS=127

description: updated
description: updated
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-01-30 03:42 EDT-------
@CAN: any update available here ?

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Heinz,
I added and tested it when I was prepping the SUR template on 22nd January.
Currently this is already part of the qemu in Disco-proposed.
=> https://launchpad.net/ubuntu/+source/qemu/1:3.1+dfsg-2ubuntu1

I wait for it to fully migrate into Disco which then will unblock preparing the related SRUs for the former releases. That migration (into Disco) is currently blocked by bug 1813730 which is being worked on by rbalint. If that bug seems to stall I will ask for a test override (to ignore the bad result of livecd-rootfs temporarily).

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (11.4 KiB)

This bug was fixed in the package qemu - 1:3.1+dfsg-2ubuntu1

---------------
qemu (1:3.1+dfsg-2ubuntu1) disco; urgency=medium

  * Merge with Debian testing, Among many other things this fixes LP Bugs:
    LP: #1806104 - fix misleading page size error on ppc64el
    LP: #1782205 - SnowRidge enabled new ISAs
    LP: #1786956 - upgrade to qemu >= 3.0
    LP: #1809083 - Backward migration to Xenial on ppc64el
    LP: #1803315 - s390x Huge page enablement
    LP: #1657409 - enable virglrenderer
    Remaining Changes:
    - qemu-kvm to systemd unit
      - d/qemu-kvm-init: script for QEMU KVM preparation modules, ksm,
        hugepages and architecture specifics
      - d/qemu-kvm.service: systemd unit to call qemu-kvm-init
      - d/qemu-system-common.install: install systemd unit and helper script
      - d/qemu-system-common.maintscript: clean old sysv and upstart scripts
      - d/qemu-system-common.qemu-kvm.default: defaults for
        /etc/default/qemu-kvm
      - d/rules: install /etc/default/qemu-kvm
    - Enable nesting by default
      - d/qemu-system-x86.modprobe: set nested=1 module option on intel.
        (is default on amd)
      - d/qemu-system-x86.postinst: re-load kvm_intel.ko if it was loaded
        without nested=1
      - d/p/ubuntu/expose-vmx_qemu64cpu.patch: expose nested kvm by default
        in qemu64 cpu type.
      - d/p/ubuntu/enable-svm-by-default.patch: Enable nested svm by default
        in qemu64 on amd
      - d/qemu-system-x86.README.Debian: document intention of nested being
        default is comfort, not full support
    - Distribution specific machine type (LP: 1304107 1621042 1776189 1761372)
      - d/p/ubuntu/define-ubuntu-machine-types.patch: define distro machine
        types
      - d/qemu-system-x86.NEWS Info on fixed machine type defintions
        for host-phys-bits=true (LP: 1776189)
      - add an info about -hpb machine type in debian/qemu-system-x86.NEWS
      - d/p/ubuntu/lp-1761372-*: provide pseries-bionic-2.11-sxxm type as
        convenience with all meltdown/spectre workarounds enabled by default.
        (LP: 1761372).
    - improved dependencies
      - Make qemu-system-common depend on qemu-block-extra
      - Make qemu-utils depend on qemu-block-extra
      - let qemu-utils recommend sharutils
    - s390x support
      - Create qemu-system-s390x package
      - Enable numa support for s390x
    - arch aware kvm wrappers
    - d/control: update VCS links (updated to match latest Ubuntu)
    - qemu-guest-agent: freeze-hook fixes (LP: 1484990)
      - d/qemu-guest-agent.install: provide /etc/qemu/fsfreeze-hook
      - d/qemu-guest-agent.dirs: provide /etc/qemu/fsfreeze-hook.d
    - d/control-in: enable RDMA support in qemu (LP: 1692476)
        - enable RDMA config option
        - add libibumad-dev build-dep
    - tolerate ipxe size change on migrations to >=18.04 (LP: 1713490)
      - d/p/ubuntu/pre-bionic-256k-ipxe-efi-roms.patch: old machine types
        reference 256k path
      - d/control-in: depend on ipxe-qemu-256k-compat-efi-roms to be able to
        handle incoming migrations from former releases.
    - d/control-in: Disable capstone disassembler library support (universe...

Changed in qemu (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI the backport for qemu 2.11 is prior to [1]
Since the requested patch doesn't need the return address for its own purpose I'll just use:
   program_interrupt(env, PGM_SPECIFICATION, ILEN_AUTO);
which I think should work fine.

[1]: https://git.qemu.org/?p=qemu.git;a=commit;h=968db419de26d3011670ca7eeab57424f93cd888

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI - the fix for this bug as an SRUs is ready and testable from a PPA for
Cosmic [1] and Bionic [2].

I'm gonna check the test instructions myself by testing that from the PPA before uploading to the SRU queue.

In addition I'll push these PPAs through the automated regression tests for qemu/libvirt.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3620
[2]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3621

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

As-is:
qemu: hardware error: Unhandled diag308 subcode 2
CPU #0:
PSW=mask 0000100180000000 addr 00000000000134be cc 00
R00=0000000000000000 R01=0000000000000000 R02=0000000000000002 R03=0000000000000000
R04=00000000000134ae R05=000000000003fdd8 R06=0000000000000000 R07=0000000000000000
R08=0000000000000000 R09=0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=00000000000134ae R15=000000000003fec0
F00=0000000000000000 F01=0000000000000000 F02=0000000000000000 F03=0000000000000000
F04=0000000000000000 F05=0000000000000000 F06=0000000000000000 F07=0000000000000000
F08=0000000000000000 F09=0000000000000000 F10=0000000000000000 F11=0000000000000000
F12=0000000000000000 F13=0000000000000000 F14=0000000000000000 F15=0000000000000000
V00=00000000000000000000000000000000 V01=00000000000000000000000000000000
V02=00000000000000000000000000000000 V03=00000000000000000000000000000000
V04=00000000000000000000000000000000 V05=00000000000000000000000000000000
V06=00000000000000000000000000000000 V07=00000000000000000000000000000000
V08=00000000000000000000000000000000 V09=00000000000000000000000000000000
V10=00000000000000000000000000000000 V11=00000000000000000000000000000000
V12=00000000000000000000000000000000 V13=00000000000000000000000000000000
V14=00000000000000000000000000000000 V15=00000000000000000000000000000000
V16=00000000000000000000000000000000 V17=00000000000000000000000000000000
V18=00000000000000000000000000000000 V19=00000000000000000000000000000000
V20=00000000000000000000000000000000 V21=00000000000000000000000000000000
V22=00000000000000000000000000000000 V23=00000000000000000000000000000000
V24=00000000000000000000000000000000 V25=00000000000000000000000000000000
V26=00000000000000000000000000000000 V27=00000000000000000000000000000000
V28=00000000000000000000000000000000 V29=00000000000000000000000000000000
V30=00000000000000000000000000000000 V31=00000000000000000000000000000000
C00=0000000000040000 C01=0000000000000000 C02=0000000000000000 C03=0000000000000000
C04=0000000000000000 C05=0000000000000000 C06=0000000000000000 C07=0000000000000000
C08=0000000000000000 C09=0000000000000000 C10=0000000000000000 C11=0000000000000000
C12=0000000000000000 C13=0000000000000000 C14=00000000c2000000 C15=0000000000000000

Aborted

With fix from PPA:
sudo qemu-system-s390x -nodefaults -nographic -machine s390-ccw-virtio,accel=kvm -chardev stdio,id=con0 -device sclpconsole,chardev=con0 -kernel diag308.elf
ABORT: diag308: Unexpected program interrupt: 6 at 0x134be, ilen 4

SUMMARY: 0 tests

EXIT: STATUS=127

That said this bug looks good to go to SRU queue, but there are more to pre-verify

description: updated
Changed in qemu (Ubuntu Bionic):
status: New → Triaged
Changed in qemu (Ubuntu Cosmic):
status: New → Triaged
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI: All Pre-Tests and Review complete - Uploaded to the SRU queue of Bionic and Cosmic

Frank Heimes (fheimes)
Changed in qemu (Ubuntu Cosmic):
status: Triaged → In Progress
Changed in qemu (Ubuntu Bionic):
status: Triaged → In Progress
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted qemu into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.12+dfsg-3ubuntu8.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in qemu (Ubuntu Cosmic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-cosmic
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2019-02-06 06:33 EDT-------
Fix verified for cosmic, via the proposed repository.

root@s38lp20:/mnt/dev/kvm-unit-tests# apt show versions qemu-system-s390x
Package: qemu-system-s390x
Version: 1:2.12+dfsg-3ubuntu8.3
Priority: optional
Section: otherosfs
Source: qemu
Origin: Ubuntu
[...]
Download-Size: 1850 kB
APT-Manual-Installed: yes
APT-Sources: http://ports.ubuntu.com/ubuntu-ports cosmic-proposed/main s390x Packages

root@s38lp20:/mnt/dev/kvm-unit-tests# ./s390x-run s390x/diag308.elf
/usr/bin/qemu-system-s390x -nodefaults -nographic -machine s390-ccw-virtio,accel=kvm -chardev stdio,id=con0 -device sclpconsole,chardev=con0 -kernel s390x/diag308.elf # -initrd /tmp/tmp.24fzdPb8Ds
PASS: diag308: Program interrupt: expected(6) == received(6)
SUMMARY: 1 tests

EXIT: STATUS=1

Revision history for this message
Frank Heimes (fheimes) wrote :

Adjusting tags accordingly ...

tags: added: verification-done verification-done-cosmic
removed: verification-needed verification-needed-cosmic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package qemu - 1:2.12+dfsg-3ubuntu8.3

---------------
qemu (1:2.12+dfsg-3ubuntu8.3) cosmic; urgency=medium

  * d/p/ubuntu/lp-1806104-spapr_pci-Remove-unhelpful-pagesize-warning.patch:
    remove misleading page size warning on qemu-system-ppc64 (LP: #1806104)
  * d/p/ubuntu/lp-1812384-s390x-Return-specification-exception.patch: ensure
    a proper exception on unknown diag 308 subcodes
    (LP: #1812384)
  * d/p/ubuntu/lp-1809083-*: fix backward migration on ppc64el (LP: #1809083)

 -- Christian Ehrhardt <email address hidden> Thu, 31 Jan 2019 14:28:19 +0100

Changed in qemu (Ubuntu Cosmic):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted qemu into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.11+dfsg-1ubuntu7.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in qemu (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
removed: verification-done
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Frank Heimes (fheimes) wrote :

Successfully verified on bionic - adjusting tags accordingly

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Test without fix on Bionic
...
Aborted (core dumped)

Installing new version
...
Setting up qemu-block-extra:s390x (1:2.11+dfsg-1ubuntu7.10) ...
Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.10) ...
Setting up python3-problem-report (2.20.9-0ubuntu7.6) ...
Setting up busybox-initramfs (1:1.27.2-2ubuntu3.1) ...
Setting up snapd (2.37.1.1+18.04) ...
snapd.failure.service is a disabled or a static unit, not starting it.
snapd.snap-repair.service is a disabled or a static unit, not starting it.
Setting up python-boto (2.44.0-1ubuntu2.18.04.0) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Processing triggers for systemd (237-3ubuntu10.13) ...
Setting up unattended-upgrades (1.1ubuntu1.18.04.9) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Setting up busybox-static (1:1.27.2-2ubuntu3.1) ...
Setting up libpci3:s390x (1:3.5.2-1ubuntu1.1) ...
Setting up libx11-data (2:1.6.4-3ubuntu0.2) ...
Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.10) ...
Setting up libx11-6:s390x (2:1.6.4-3ubuntu0.2) ...
Setting up qemu-system-s390x (1:2.11+dfsg-1ubuntu7.10) ...
Setting up python3-apport (2.20.9-0ubuntu7.6) ...
Setting up apport (2.20.9-0ubuntu7.6) ...
apport-autoreport.service is a disabled or a static unit, not starting it.
Setting up pciutils (1:3.5.2-1ubuntu1.1) ...
Setting up qemu-kvm (1:2.11+dfsg-1ubuntu7.10) ...
...

Rerunning testcase
ABORT: diag308: Unexpected program interrupt: 6 at 0x134be, ilen 4
SUMMARY: 0 tests
EXIT: STATUS=12

Setting verified

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@SRU Team - I beg your pardon and wanted to make clear that I try to care for my SRUs.
This looks all good except autopkgtest failing for other reasons.
Those are handled in bug 1817721.
IMHO: Feel free to release this as-is and/or to help getting the MP associated to 1817721 accepted.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package qemu - 1:2.11+dfsg-1ubuntu7.10

---------------
qemu (1:2.11+dfsg-1ubuntu7.10) bionic; urgency=medium

  * d/p/ubuntu/lp-1806104-spapr_pci-Remove-unhelpful-pagesize-warning.patch:
    remove misleading page size warning on qemu-system-ppc64 (LP: #1806104)
  * d/p/ubuntu/lp-1812384-s390x-Return-specification-exception.patch: ensure
    a proper exception on unknown diag 308 subcodes
    (LP: #1812384)
  * d/p/ubuntu/lp-1809083-*: fix backward migration on ppc64el (LP: #1809083)

 -- Christian Ehrhardt <email address hidden> Thu, 31 Jan 2019 14:07:51 +0100

Changed in qemu (Ubuntu Bionic):
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2019-03-05 02:32 EDT-------
IBM bugzilla status->closed, Fix Released with Bionic. all distros done.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.