[18.04 FEAT] zKVM: Add hardware CPU Model - kernel part

Bug #1836153 reported by bugproxy on 2019-07-11
20
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
High
Unassigned
linux (Ubuntu)
Undecided
Skipper Bug Screeners
Bionic
Medium
Unassigned
Disco
Medium
Unassigned

Bug Description

SRU Justification:
==================

[Impact]

* New hardware is not supported by qemu and won't run run, due to missing hardware CPU model

[Fix]

* 11ba5961a2156a4f210627ed8421387e2531b100 11ba596 "KVM: s390: add debug logging for cpu model subfunctions"

* 346fa2f891c71a9b98014f8f62c15f4c7dd95ec1 346fa2f "KVM: s390: implement subfunction processor calls"

* 7832e91cd33f21f3cf82b003478c292915a1ec14 7832e91 "KVM: s390: add vector enhancements facility 2 to cpumodel"

* d5cb6ab1e3d4d7e0648a167f6290e89f6e86964e d5cb6ab "KVM: s390: add vector BCD enhancements facility to cpumodel"

* 13209ad0395c4de7fa48108b1dac72e341d5c089 13209ad "KVM: s390: add MSA9 to cpumodel"

* d668139718a9e2260702777bd8d86d71c30b6539 d668139 "KVM: s390: provide query function for instructions returning 32 byte"

* 173aec2d5a9fa5f40e462661a8283fcafe04764f 173aec2 "KVM: s390: add enhanced sort facilty to cpu model"

* 4f45b90e1c03466202fca7f62eaf32243f220830 4f45b90 "KVM: s390: add deflate conversion facilty to cpu model"

* 8ec2fa52eac53bff7ef1cedbc4ad8af650ec937c 8ec2fa5 "KVM: s390: enable MSA9 keywrapping functions depending on cpu model"

[Test Case]

* need to be tested by IBM on pre-rel. hardware or simulator

[Regression Potential]

* The regression potential can be considered as low since these changes are limited to arch/s390

* changes are in support for new and upcoming hardware and shouldn't affect existing s390x systems

[Other Info]

* the first 2 commits are the key ones, the other 7 are needed to make them applly cleanly

* these patches are already all included in upstream kernel 5.2, hence they are in eoan's kernel 5.2

* qemu package SRU in LP 1836154 complements this SRU - for testing both need to be in place

* I could apply the commit IDs cleanly with 'cherry-picks --strategy=recursive -X theirs' and checked that no patches on top were pulled in
_________________________

Feature request to be applied to Ubuntu 18.04 - kernel.

The git commit information is already provided within
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1830239

Once this feature is accepted all information for integration will be provided by IBM
kernel 5.2 down to 4.15.

bugproxy (bugproxy) on 2019-07-11
tags: added: architecture-s39064 bugnameltc-179024 severity-high targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)

------- Comment From <email address hidden> 2019-07-11 03:30 EDT-------
To simplify the backport to 4.15 you should add these 2 commits:

11ba5961a2156 KVM: s390: add debug logging for cpu model subfunctions
346fa2f891c71 KVM: s390: implement subfunction processor calls

I have verified that on bionic/master the following works without a problem:

git cherry-pick 346fa2f891c71 11ba5961a2156 7832e91cd33f d5cb6ab1e3d4 13209ad0395c d668139718a9 173aec2d5a9f 4f45b90e1c03 8ec2fa52eac5

Changed in ubuntu-z-systems:
status: New → Triaged
importance: Undecided → High
information type: Private → Public
description: updated
Frank Heimes (frank-heimes) wrote :
Changed in linux (Ubuntu):
status: New → In Progress
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Stefan Bader (smb) on 2019-07-18
Changed in linux (Ubuntu Bionic):
importance: Undecided → Medium
Changed in linux (Ubuntu Disco):
importance: Undecided → Medium
Changed in linux (Ubuntu Bionic):
status: New → Fix Committed
Changed in linux (Ubuntu Disco):
status: New → Fix Committed

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-disco

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-07-26 03:12 EDT-------
I can confirm that the KVM part (this bugzilla) works fine with
a. this kernel from proposed
b. qemu from https://launchpad.net/~paelzer/+archive/ubuntu/bug-1836154-s390x-hwmodels/+packages

Now. The kernel in proposed has a bug which prohibits this kernel to be released.

dmesg is flooded with
[ 417.945769] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.951747] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.955339] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.960318] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 417.961260] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 422.248440] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 430.012698] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 434.051283] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 526.865193] BUG: non-zero pgtables_bytes on freeing mm: -16384
[ 526.873399] BUG: non-zero pgtables_bytes on freeing mm: -16384

messages. Looks like you picked an upstream patch and missed several followup patches in that area.
Thjis is around commit e12e4044aede97974f2222eb7f0ed726a5179a32 ("s390/mm: fix mis-accounting of pgtable_bytes") and later patches on top of this.

(This is independent from the KVM changes which are working perfectly fine).

Frank Heimes (frank-heimes) wrote :

@cborntra many thanks for the verification.

Commit 9967c2a is indeed not (yet) in this 4.15 bionic kernel.
It got upstream accepted with 4.20 and is with that in disco, hence that should all work on disco:

$ git log --oneline | grep -m 1 "s390/mm: fix mis-accounting of pgtable_bytes"
e12e404 s390/mm: fix mis-accounting of pgtable_bytes
$ git tag --contains e12e404
Ubuntu-5.0.0-7.8
...
Ubuntu-5.0.0-22.23
v5.0
$ rmadison --arch=s390x linux-generic | grep disco-proposed
 linux-generic | 5.0.0.22.23 | disco-proposed | s390x

Regarding bionic, the patch you pointed out will be picked up with the upstream stable release update process that is regularly done.
The next update is addressed by ticket LP 1836968 and covers patchset 2019-07-17, that includes "s390/mm: fix mis-accounting of pgtable_bytes"
(Ctrl+f on page https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1836968 will show)
Hence there is no need to open a new LP for this particular patch or to submit it in a separate kernel SRU. Because it will probably come with the next bionic kernel update (since LP 1836968 is already marked as Fix Committed).

I think with strictly looking at the KVM part we can consider this verification as successfully done.

I will btw. mark the 'linux (Ubuntu)' entry that represents eoan and Fix Released, because with it's 5.2 kernel it has already everything need.

Changed in linux (Ubuntu):
status: In Progress → Fix Released
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
tags: added: verification-done-bionic verification-done-disco
removed: verification-needed-bionic verification-needed-disco
Frank Heimes (frank-heimes) wrote :

With "[SRU][Bionic][PULL] Bionic upstream stable patchset 2019-07-30+"
commit "s390/mm: fix mis-accounting of pgtable_bytes" landed in between in bionic master-next.

~/ubuntu-bionic-master-next/ubuntu-bionic$ git log --oneline | grep -m 1 "s390/mm: fix mis-accounting of pgtable_bytes"
76d93ab s390/mm: fix mis-accounting of pgtable_bytes
~/ubuntu-bionic-master-next/ubuntu-bionic$ git describe --tags --contains 76d93ab
Ubuntu-4.15.0-56.62~514

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-08-07 08:50 EDT-------
Due to vacation reason, the verification is delayed by 3 weeks.

Frank Heimes (frank-heimes) wrote :

Verification was already done on disco and bionic, just xenial will be delayed.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 5.0.0-25.26

---------------
linux (5.0.0-25.26) disco; urgency=medium

  * CVE-2019-1125
    - x86/cpufeatures: Carve out CQM features retrieval
    - x86/cpufeatures: Combine word 11 and 12 into a new scattered features word
    - x86/speculation: Prepare entry code for Spectre v1 swapgs mitigations
    - x86/speculation: Enable Spectre v1 swapgs mitigations
    - x86/entry/64: Use JMP instead of JMPQ
    - x86/speculation/swapgs: Exclude ATOMs from speculation through SWAPGS

 -- Kleber Sacilotto de Souza <email address hidden> Thu, 01 Aug 2019 12:04:35 +0200

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (171.3 KiB)

This bug was fixed in the package linux - 4.15.0-58.64

---------------
linux (4.15.0-58.64) bionic; urgency=medium

  * unable to handle kernel NULL pointer dereference at 000000000000002c (IP:
    iget5_locked+0x9e/0x1f0) (LP: #1838982)
    - Revert "ovl: set I_CREATING on inode being created"
    - Revert "new primitive: discard_new_inode()"

linux (4.15.0-57.63) bionic; urgency=medium

  * CVE-2019-1125
    - x86/cpufeatures: Carve out CQM features retrieval
    - x86/cpufeatures: Combine word 11 and 12 into a new scattered features word
    - x86/speculation: Prepare entry code for Spectre v1 swapgs mitigations
    - x86/speculation: Enable Spectre v1 swapgs mitigations
    - x86/entry/64: Use JMP instead of JMPQ
    - x86/speculation/swapgs: Exclude ATOMs from speculation through SWAPGS

  * Packaging resync (LP: #1786013)
    - update dkms package versions

linux (4.15.0-56.62) bionic; urgency=medium

  * bionic/linux: 4.15.0-56.62 -proposed tracker (LP: #1837626)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log
    - [Packaging] update helper scripts

  * CVE-2019-2101
    - media: uvcvideo: Fix 'type' check leading to overflow

  * hibmc-drm Causes Unreadable Display for Huawei amd64 Servers (LP: #1762940)
    - [Config] Set CONFIG_DRM_HISI_HIBMC to arm64 only
    - SAUCE: Make CONFIG_DRM_HISI_HIBMC depend on ARM64

  * Bionic: support for Solarflare X2542 network adapter (sfc driver)
    (LP: #1836635)
    - sfc: make mem_bar a function rather than a constant
    - sfc: support VI strides other than 8k
    - sfc: add Medford2 (SFC9250) PCI Device IDs
    - sfc: improve PTP error reporting
    - sfc: update EF10 register definitions
    - sfc: populate the timer reload field
    - sfc: update MCDI protocol headers
    - sfc: support variable number of MAC stats
    - sfc: expose FEC stats on Medford2
    - sfc: expose CTPIO stats on NICs that support them
    - sfc: basic MCDI mapping of 25/50/100G link speeds
    - sfc: support the ethtool ksettings API properly so that 25/50/100G works
    - sfc: add bits for 25/50/100G supported/advertised speeds
    - sfc: remove tx and MCDI handling from NAPI budget consideration
    - sfc: handle TX timestamps in the normal data path
    - sfc: add function to determine which TX timestamping method to use
    - sfc: use main datapath for HW timestamps if available
    - sfc: only enable TX timestamping if the adapter is licensed for it
    - sfc: MAC TX timestamp handling on the 8000 series
    - sfc: on 8000 series use TX queues for TX timestamps
    - sfc: only advertise TX timestamping if we have the license for it
    - sfc: simplify RX datapath timestamping
    - sfc: support separate PTP and general timestamping
    - sfc: support second + quarter ns time format for receive datapath
    - sfc: support Medford2 frequency adjustment format
    - sfc: add suffix to large constant in ptp
    - sfc: mark some unexported symbols as static
    - sfc: update MCDI protocol headers
    - sfc: support FEC configuration through ethtool
    - sfc: remove ctpio_dmabuf_start from stats
    - sfc: stop the TX queue before pushing new buffers

  * [18.04 FEAT] zKVM: Add hardwar...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-08-15 08:45 EDT-------
IBM Bugzilla status -> closed, Released by all requested Distros

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-09-03 14:58 EDT-------
This still has the verification-needed-xenial tag. Will this be backported or can we remove that tag? xenial-proposed does not seem to contain a kernel update and a backport to xenial was not requested. (It would be possible though).

Dimitri John Ledkov (xnox) wrote :

Please ignore the xenial verification request, as it was generated in error. I can confirm this issue is resolved in bionic and up and calls for no further actions. I apologise for any inconvenience caused.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers