[UBUNTU 20.04] Fix SIGP processing on KVM/s390

Bug #1962578 reported by bugproxy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Skipper Bug Screeners
linux (Ubuntu)
Fix Released
High
Skipper Bug Screeners
Focal
Fix Released
Medium
Canonical Kernel Team
Impish
Fix Released
Medium
Canonical Kernel Team

Bug Description

SRU Justification:
==================

[Impact]

* The SIGP processing on KVM/s390x is incorrect and not fully conform to
  the 'Principles of Operations', the IBM Z architecture definition
  and documentation.

* KVM: SIGP RESTART is reporting a CPU state 'stopped' after SIGP RESTART
  was issued, while it should report a 'busy' condition until the CPU is
  really started.

* The fix for this consists of three patches:
  812de04661c4 "KVM: s390: Clarify SIGP orders versus STOP/RESTART"
  435d17955519 "KVM: s390: Add a routine for setting userspace CPU state"
  4563bcad46bf "KVM: s390: Simplify SIGP Set Arch handling"
  whereas 812de04661c4 already landed via upstream stable,
  hence only two are left and need to be cherry-picked:

[Fix]

* 67cf68b6a5ccac8bc7dfef0a220b59af4c83fd2c 67cf68b6a5cc "KVM: s390: Add a routine for setting userspace CPU state"

* 8eeba194a32e0f50329354a696baaa2e3d9accc5 8eeba194a32e "KVM: s390: Simplify SIGP Set Arch handling"

[Test Case]

* Install an LPAR on IBM Z (z13+) or LinuxONE (Emperor / Rockhopper)
  with Ubuntu Server 20.04 using kernel standard kernel 5.4.

* Setup this LPAR as QEMU/KVM host and a focal VM as guest.

* The test itself is the following unit test:
  https://<email address hidden>/

* The test and verification will be done by the IBM Z team.

[Where problems could occur]

* A broken patch could harm KVM ioctl on s390x or the (virtual) CPU
  state control in general.

* The SIGP handing (on s390x) can become broken, which could lead to
  incorrect (virtual) CPU states.

* In worst case KVM can become broken on s390x entirely.

* The modified code is all s390x specific QEMU/KVM code,
  no common code is touched.

[Other]

* The two remaining patches are upstream since 5.16,
  the first one already landed in 5.4 via upstream stable bug LP#1959701
  and in 5.13 via upstream stable bug LP#1960861.

* The patches already landed in jammy (master-next) based on LP#1959735,
  hence only SRU to impish and focal is needed.

* This patches got discussed here:
  https://<email address hidden>/

* This not only fixes the SIGP processing, but is also important in terms of
  long term maintainability.

__________

Description:
   KVM: SIGP RESTART is reporting a CPU state 'stopped' after SIGP RESTART was issued, while it should report a 'busy' condition until the CPU is really started.

 The patches to fix this have already been picked for jammy/master-next:

812de04661c4 "KVM: s390: Clarify SIGP orders versus STOP/RESTART"
435d17955519 "KVM: s390: Add a routine for setting userspace CPU state"
4563bcad46bf "KVM: s390: Simplify SIGP Set Arch handling"

 For Ubuntu 20.04 the focal/master-next already contains:
d74b0d2d38d3 KVM: s390: Clarify SIGP orders versus STOP/RESTART

 so we additionally need the following two upstream commits in focal
67cf68b6a5cc KVM: s390: Add a routine for setting userspace CPU state
8eeba194a32e KVM: s390: Simplify SIGP Set Arch handling

CVE References

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-196454 severity-high targetmilestone-inin---
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
importance: Undecided → High
Changed in linux (Ubuntu):
importance: Undecided → High
Changed in linux (Ubuntu Jammy):
status: New → Fix Released
status: Fix Released → In Progress
no longer affects: linux (Ubuntu Jammy)
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

This was adressed in jammy by LP#1959735, hence updating the jammy entry to Fix Released.

Revision history for this message
Frank Heimes (fheimes) wrote : Re: [UBUNTU 20.04] Fix SIGP processing (KVM: s390)

To avoid any regressions after upgrades, it's needed that Impish/21.10 is patched, too.
(SIGP fix is already addressed for jammy in LP#/1959735)

I just check the impish master-next tree and the situation is the same like on focal.
Just the following two need to be added to focal and impish:
67cf68b6a5cc KVM: s390: Add a routine for setting userspace CPU state
8eeba194a32e KVM: s390: Simplify SIGP Set Arch handling

Frank Heimes (fheimes)
summary: - [UBUNTU 20.04] Fix SIGP processing (KVM: s390)
+ [UBUNTU 20.04] Fix SIGP processing on KVM/s390
Revision history for this message
Frank Heimes (fheimes) wrote :

Kernel test builds:
linux - 5.13.0-32.35~lp1962578
linux - 5.4.0-102.115~lp1962578
that incl. the commits mentioned above are available here:
https://launchpad.net/~fheimes/+archive/ubuntu/lp1962578

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2022-03-03 16:21 EDT-------
A draft of the kvm-unit-test patches for these changes are now posted upstream:

https://<email address hidden>/

There are a couple of additional cleanups in the series, but the key points are:
- (Patch 1) The SIGP SENSE used to determine if a CPU is operating/stopped now loops if it returns a CC2 (now possible with the kernel changes here)
- (Patch 4 and 5) The test of the SIGP STOP/RESTART opcodes don't do any additional looping other than that CC2 loop for a SIGP SENSE

Revision history for this message
Frank Heimes (fheimes) wrote :

Many thx, Eric!

Revision history for this message
Frank Heimes (fheimes) wrote (last edit ):

SRU request submitted to the Ubuntu kernel team mailing list for impish and focal:
https://lists.ubuntu.com/archives/kernel-team/2022-March/thread.html#128464
Changing status to 'In Progress' for impish and focal.

Changed in linux (Ubuntu):
status: In Progress → Fix Released
Changed in linux (Ubuntu Focal):
status: New → In Progress
Changed in linux (Ubuntu Impish):
status: New → In Progress
Changed in ubuntu-z-systems:
status: New → In Progress
description: updated
Changed in linux (Ubuntu Focal):
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Changed in linux (Ubuntu Impish):
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
Frank Heimes (fheimes)
description: updated
Revision history for this message
bugproxy (bugproxy) wrote :
Stefan Bader (smb)
Changed in linux (Ubuntu Impish):
importance: Undecided → Medium
Changed in linux (Ubuntu Focal):
importance: Undecided → Medium
Changed in linux (Ubuntu Impish):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Focal):
status: In Progress → Fix Committed
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
bugproxy (bugproxy)
tags: added: targetmilestone-inin2004
removed: targetmilestone-inin---
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/5.13.0-38.43 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-impish' to 'verification-done-impish'. If the problem still exists, change the tag 'verification-needed-impish' to 'verification-failed-impish'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-impish
tags: added: verification-needed-focal
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/5.4.0-106.120 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2022-03-23 11:42 EDT-------
Verified with the assistance of kvm-unit-tests patches described here:
https://<email address hidden>/

...and further examination of the s390dbf/kvm-<pid> contents for a running guest. Rolled back the kernel version on each release to ensure I had a "bad" state, as well as the version in -proposed. Everything behaves as I expected. THANK YOU!

Tested Focal:
5.4.0-100.113 (FAILS)
5.4.0-106.120 (WORKS)

Tested Impish:
5.13.0-30.33 (FAILS)
5.13.0-38.43 (WORKS)

tags: added: verification-done-focal verification-done-impish
removed: verification-needed-focal verification-needed-impish
Revision history for this message
Frank Heimes (fheimes) wrote :

Thank you Eric for the verifications!

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-hwe-5.4/5.4.0-107.121~18.04.1 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
Revision history for this message
Frank Heimes (fheimes) wrote :

This bug was opened for focal and impish, bionic was not really requested and is not marked as affected, so I'm setting the verification-done-bionic tag to unblock further SRU processing.

tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (53.1 KiB)

This bug was fixed in the package linux - 5.13.0-40.45

---------------
linux (5.13.0-40.45) impish; urgency=medium

  * impish/linux: 5.13.0-40.45 -proposed tracker (LP: #1966701)

  * CVE-2022-1016
    - netfilter: nf_tables: initialize registers in nft_do_chain()

  * CVE-2022-1015
    - netfilter: nf_tables: validate registers coming from userspace.

  * audit: improve audit queue handling when "audit=1" on cmdline
    (LP: #1965723) // Impish update: upstream stable patchset 2022-03-22
    (LP: #1966021)
    - audit: improve audit queue handling when "audit=1" on cmdline

  * PS/2 Keyboard wakeup from s2idle not functioning on AMD Yellow Carp platform
    (LP: #1961739)
    - PM: s2idle: ACPI: Fix wakeup interrupts handling

  * Low RX performance for 40G Solarflare NICs (LP: #1964512)
    - SAUCE: sfc: The size of the RX recycle ring should be more flexible

  * [UBUNTU 20.04] Fix SIGP processing on KVM/s390 (LP: #1962578)
    - KVM: s390: Simplify SIGP Set Arch handling
    - KVM: s390: Add a routine for setting userspace CPU state

  * Move virtual graphics drivers from linux-modules-extra to linux-modules
    (LP: #1960633)
    - [Packaging] Move VM DRM drivers into modules

  * Impish update: upstream stable patchset 2022-03-09 (LP: #1964422)
    - bnx2x: Utilize firmware 7.13.21.0
    - bnx2x: Invalidate fastpath HSI version for VFs
    - rcu: Tighten rcu_advance_cbs_nowake() checks
    - select: Fix indefinitely sleeping task in poll_schedule_timeout()
    - drm/amdgpu: Use correct VIEWPORT_DIMENSION for DCN2
    - arm64/bpf: Remove 128MB limit for BPF JIT programs
    - Bluetooth: refactor malicious adv data check
    - net: sfp: ignore disabled SFP node
    - net: stmmac: skip only stmmac_ptp_register when resume from suspend
    - s390/hypfs: include z/VM guests with access control group set
    - bpf: Guard against accessing NULL pt_regs in bpf_get_task_stack()
    - scsi: zfcp: Fix failed recovery on gone remote port with non-NPIV FCP
      devices
    - udf: Restore i_lenAlloc when inode expansion fails
    - udf: Fix NULL ptr deref when converting from inline format
    - efi: runtime: avoid EFIv2 runtime services on Apple x86 machines
    - PM: wakeup: simplify the output logic of pm_show_wakelocks()
    - tracing/histogram: Fix a potential memory leak for kstrdup()
    - tracing: Don't inc err_log entry count if entry allocation fails
    - ceph: properly put ceph_string reference after async create attempt
    - ceph: set pool_ns in new inode layout for async creates
    - fsnotify: fix fsnotify hooks in pseudo filesystems
    - Revert "KVM: SVM: avoid infinite loop on NPF from bad address"
    - perf/x86/intel/uncore: Fix CAS_COUNT_WRITE issue for ICX
    - drm/etnaviv: relax submit size limits
    - KVM: x86: Update vCPU's runtime CPUID on write to MSR_IA32_XSS
    - netfilter: nft_payload: do not update layer 4 checksum when mangling
      fragments
    - serial: 8250: of: Fix mapped region size when using reg-offset property
    - serial: stm32: fix software flow control transfer
    - tty: n_gsm: fix SW flow control encoding/handling
    - tty: Add support for Brainboxes UC cards.
    - usb-storage: Add unusual-devs...

Changed in linux (Ubuntu Impish):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (10.8 KiB)

This bug was fixed in the package linux - 5.4.0-109.123

---------------
linux (5.4.0-109.123) focal; urgency=medium

  * focal/linux: 5.4.0-109.123 -proposed tracker (LP: #1968290)

  * USB devices not detected during boot on USB 3.0 hubs (LP: #1968210)
    - SAUCE: Revert "Revert "xhci: Set HCD flag to defer primary roothub
      registration""
    - SAUCE: Revert "Revert "usb: core: hcd: Add support for deferring roothub
      registration""

linux (5.4.0-108.122) focal; urgency=medium

  * focal/linux: 5.4.0-108.122 -proposed tracker (LP: #1966740)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync dkms-build{,--nvidia-N} from LRMv5
    - debian/dkms-versions -- update from kernel-versions (main/2022.03.21)

  * Low RX performance for 40G Solarflare NICs (LP: #1964512)
    - SAUCE: sfc: The size of the RX recycle ring should be more flexible

  * [UBUNTU 20.04] KVM: Enable storage key checking for intercepted instruction
    (LP: #1962831)
    - selftests: kvm: add _vm_ioctl
    - selftests: kvm: Introduce the TEST_FAIL macro
    - KVM: selftests: Add GUEST_ASSERT variants to pass values to host
    - KVM: s390: gaccess: Refactor gpa and length calculation
    - KVM: s390: gaccess: Refactor access address range check
    - KVM: s390: gaccess: Cleanup access to guest pages
    - s390/uaccess: introduce bit field for OAC specifier
    - s390/uaccess: fix compile error
    - s390/uaccess: Add copy_from/to_user_key functions
    - KVM: s390: Honor storage keys when accessing guest memory
    - KVM: s390: handle_tprot: Honor storage keys
    - KVM: s390: selftests: Test TEST PROTECTION emulation
    - KVM: s390: Add optional storage key checking to MEMOP IOCTL
    - KVM: s390: Add vm IOCTL for key checked guest absolute memory access
    - KVM: s390: Rename existing vcpu memop functions
    - KVM: s390: Add capability for storage key extension of MEM_OP IOCTL
    - KVM: s390: Update api documentation for memop ioctl
    - KVM: s390: Clarify key argument for MEM_OP in api docs
    - KVM: s390: Add missing vm MEM_OP size check

  * 【sec-0911】 fail to reset sec module (LP: #1943301)
    - crypto: hisilicon/sec2 - Add workqueue for SEC driver.
    - crypto: hisilicon/sec2 - update SEC initialization and reset

  * Lots of hisi_qm zombie task slow down system after stress test
    (LP: #1932117)
    - crypto: hisilicon - Use one workqueue per qm instead of per qp

  * Lots of hisi_qm zombie task slow down system after stress test
    (LP: #1932117) // 【sec-0911】 fail to reset sec module (LP: #1943301)
    - crypto: hisilicon - Unify hardware error init/uninit into QM

  * [UBUNTU 20.04] Fix SIGP processing on KVM/s390 (LP: #1962578)
    - KVM: s390: Simplify SIGP Set Arch handling
    - KVM: s390: Add a routine for setting userspace CPU state

  * Move virtual graphics drivers from linux-modules-extra to linux-modules
    (LP: #1960633)
    - [Packaging] Move VM DRM drivers into modules

  * Focal update: v5.4.178 upstream stable release (LP: #1964634)
    - audit: improve audit queue handling when "audit=1" on cmdline
    - ASoC: ops: Reject out of bounds values in snd_soc_put_volsw()
    - ASoC: ops: Reject out of bounds values in snd_...

Changed in linux (Ubuntu Focal):
status: Fix Committed → Fix Released
Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2022-05-06 12:21 EDT-------
Fix released to -updates with last SRU cycle on April 18, hence closing the bug.

Changing Status to: -> CLOSED.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.