Ubuntu 16.04.6 - Shared CEX7C cards defined in z/VM guest not established by zcrypt device driver

Bug #1848173 reported by bugproxy on 2019-10-15
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Frank Heimes
linux (Ubuntu)
Skipper Bug Screeners

Bug Description

SRU Justification:


* Ubuntu 16.04.6 systems on z15 with crypto CEX7C adapters under z/VM cannot see and make use of their hw crypto resources.

* The patch/backport adds CEX7 toleration support (by mapping it to CEX5) to kernel 4.4.


* Backport: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1848173/+attachment/5297314/+files/s390-zcrypt-CEX7-toleration-support.patch

[Test Case]

* Define a z/VM guest with 'apvirt' (hardware crypto adapter virtualization) having CryptoExpress 7S adapters attached to z/VM LPAR.

* Use lszcrypt command (ideally lszcrypt -VVV) from the s390-tools package to list the detected and available hardware crypto resources.

* Canonical can only do a toleration test: IBM needs to do the functional test (due to hardware availability).

[Regression Potential]

* The regression potential can be considered as moderate since this is purely s390x specific

* and limited to CryptoExpress 7S (CEX7) adapter cards

* and again if they running under z/VM (on z15) with 'apvirt' configured for the guest.

* and again only with 16.04.6's kernel 4.4.

[Other Info]

* The patch was already applied, kernel compiled and things tested on z15 und z/VM.


System: IBM Z15 z/VM with shared CEX7C adapters
OS: Ubuntu 16.04.6 LTS ( 4.4.0-165-generic kernel ) with latest updates
Shared CEX7C adapters are not displayed on Ubuntu even though APAR 66266 had been installed onto the unterlying z/VM system.

Defined shared CEX7C CCA adapters to provide cryptographic accelerators based on CCA cards to a z/VM guest system running Ubuntu 16.04.6 LTS.

The adapters display all right under vm or when running vmcp commands under Linux.

lszcrypt -VVV does not display any adapter.

We observed that zcrypt_cex4 was not automatically loaded via dependency by modprobe ap. Explicitly loading by modprobe zcrypt_cex4 did not change card availability.

Please investigate.


Terminal output
root@system:/sys/bus/ap/devices/card01# ls -l
total 0
-r--r--r-- 1 root root 4096 Oct 8 17:51 ap_functions
-r--r--r-- 1 root root 4096 Oct 8 17:51 depth
-r--r--r-- 1 root root 4096 Oct 8 17:51 hwtype
-r--r--r-- 1 root root 4096 Oct 8 17:51 interrupt
-r--r--r-- 1 root root 4096 Oct 8 17:51 modalias
-r--r--r-- 1 root root 4096 Oct 8 17:51 pendingq_count
drwxr-xr-x 2 root root 0 Oct 8 17:51 power
-r--r--r-- 1 root root 4096 Oct 8 17:51 raw_hwtype
-r--r--r-- 1 root root 4096 Oct 8 17:51 request_count
-r--r--r-- 1 root root 4096 Oct 8 17:51 requestq_count
-r--r--r-- 1 root root 4096 Oct 8 17:51 reset
lrwxrwxrwx 1 root root 0 Oct 8 17:51 subsystem -> ../../../bus/ap
-rw-r--r-- 1 root root 4096 Oct 8 17:50 uevent

# lszcrypt -V // < No output displayed >
# vmcp q v crypto
AP 001 CEX7C Domain 001 shared online

root@system:/sys/bus/ap/devices/card01# cat hwtype
root@system:/sys/bus/ap/devices/card01# cat raw_hwtype

# lsmod
Module Size Used by
ap 36864 0
ghash_s390 16384 0
prng 16384 0
aes_s390 20480 0
des_s390 16384 0
des_generic 28672 1 des_s390
sha512_s390 16384 0
qeth_l2 53248 1
sha256_s390 16384 0
sha1_s390 16384 0
sha_common 16384 3 sha256_s390,sha1_s390,sha512_s390
qeth 151552 1 qeth_l2
vmur 20480 0
ccwgroup 20480 1 qeth
dm_multipath 36864 0
zfcp 143360 0
dasd_eckd_mod 118784 8
qdio 73728 3 qeth,zfcp,qeth_l2
scsi_transport_fc 86016 1 zfcp
dasd_mod 135168 5 dasd_eckd_mod

# modprobe zcrypt_cex4
zcrypt_cex4 16384 0
zcrypt_api 36864 1 zcrypt_cex4
ap 36864 2 zcrypt_cex4,zcrypt_api

Contact Information = <email address hidden>

---uname output---
Linux system 4.4.0-164-generic #192-Ubuntu SMP Fri Sep 13 12:01:28 UTC 2019 s390x s390x s390x GNU/Linux

Machine Type = IBM Type: 8561 Model: 403 T01

A debugger is not configured

---Steps to Reproduce---
 1.) Define shared CEX7 CCA cards to z/VM Guest
2.) boot up Ubuntu 16.04.6 LTS
3.) modprobe ap
4.) lszcrypt -VVV

Stack trace output:

Oops output:

System Dump Info:
  The system is not configured to capture a system dump.

Device driver error code:

*Additional Instructions for <email address hidden>:
-Attach sysctl -a output output to the bug.

lszcrypt returns with

# lszcrypt -VVV ; echo RC=$?

After investigating here a little ...
Ubuntu 16.04 has only toleration support for CEX6 and no support for CEX7.

Here is a patch which maps cex7 cards to cex5 cards.
Have a look into - it is just a 2 line code change which
extends the toleration patch for cex6 (mapped to cex5)
by the cex7 card - also mapped to cex5.

Code compiles and I've tested the kernel on a z15 with
lots of cex6 and cex7 cards - works fine.

Default Comment by Bridge

tags: added: architecture-s39064 bugnameltc-181815 severity-high targetmilestone-inin16046
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Frank Heimes (fheimes) on 2019-10-15
Changed in ubuntu-z-systems:
status: New → Triaged
importance: Undecided → Medium
Frank Heimes (fheimes) on 2019-10-15
summary: - Ubuntu16.04.6 - shared CEX7C cards defined in z/VM guest not established
- by zcrypt device driver
+ Ubuntu 16.04.6 - shared CEX7C cards defined in z/VM guest not
+ established by zcrypt device driver
Frank Heimes (fheimes) on 2019-10-16
summary: - Ubuntu 16.04.6 - shared CEX7C cards defined in z/VM guest not
+ Ubuntu 16.04.6 - Shared CEX7C cards defined in z/VM guest not
established by zcrypt device driver
Frank Heimes (fheimes) wrote :

Kernel SRU request submitted:
Changing status to In Progress.

description: updated
Changed in linux (Ubuntu):
status: New → In Progress
Changed in ubuntu-z-systems:
assignee: nobody → Frank Heimes (frank-heimes)
status: Triaged → In Progress
Changed in linux (Ubuntu Xenial):
status: New → In Progress
Changed in linux (Ubuntu):
status: In Progress → Invalid
Changed in linux (Ubuntu Xenial):
status: In Progress → Fix Committed
Frank Heimes (fheimes) on 2019-10-21
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial

------- Comment From <email address hidden> 2019-10-23 07:52 EDT-------
Verified, with
uname -a
Linux t35lp54 4.4.0-167-generic #196-Ubuntu SMP Mon Oct 21 19:47:50 UTC 2019 s390x s390x s390x GNU/Linux

works - I can see the CEX7 cards in toleration mode as CEX5 cards :-)

Frank Heimes (fheimes) wrote :

Thanks for the verification - adjusting the tags accordingly.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Launchpad Janitor (janitor) wrote :
Download full text (18.6 KiB)

This bug was fixed in the package linux - 4.4.0-168.197

linux (4.4.0-168.197) xenial; urgency=medium

  * CVE-2018-12207
    - KVM: x86: MMU: Encapsulate the type of rmap-chain head in a new struct
    - KVM: x86: MMU: Consolidate quickly_check_mmio_pf() and is_mmio_page_fault()
    - KVM: x86: MMU: Move handle_mmio_page_fault() call to kvm_mmu_page_fault()
    - KVM: MMU: rename has_wrprotected_page to mmu_gfn_lpage_is_disallowed
    - KVM: MMU: introduce kvm_mmu_gfn_{allow,disallow}_lpage
    - KVM: x86: MMU: Make mmu_set_spte() return emulate value
    - KVM: x86: MMU: Move initialization of parent_ptes out from
    - KVM: x86: MMU: always set accessed bit in shadow PTEs
    - KVM: x86: MMU: Move parent_pte handling from kvm_mmu_get_page() to
    - KVM: x86: MMU: Remove unused parameter parent_pte from kvm_mmu_get_page()
    - KVM: x86: simplify ept_misconfig
    - KVM: x86: extend usage of RET_MMIO_PF_* constants
    - KVM: MMU: drop vcpu param in gpte_access
    - kvm: Convert kvm_lock to a mutex
    - kvm: x86: Do not release the page inside mmu_set_spte()
    - KVM: x86: make FNAME(fetch) and __direct_map more similar
    - KVM: x86: remove now unneeded hugepage gfn adjustment
    - KVM: x86: change kvm_mmu_page_get_gfn BUG_ON to WARN_ON
    - KVM: x86: add tracepoints around __direct_map and FNAME(fetch)
    - SAUCE: KVM: vmx, svm: always run with EFER.NXE=1 when shadow paging is
    - SAUCE: x86: Add ITLB_MULTIHIT bug infrastructure
    - SAUCE: kvm: mmu: ITLB_MULTIHIT mitigation
    - SAUCE: kvm: Add helper function for creating VM worker threads
    - SAUCE: kvm: x86: mmu: Recovery of shattered NX large pages
    - SAUCE: cpu/speculation: Uninline and export CPU mitigations helpers
    - SAUCE: kvm: x86: mmu: Apply global mitigations knob to ITLB_MULTIHIT

  * CVE-2019-11135
    - KVM: x86: Emulate MSR_IA32_ARCH_CAPABILITIES on AMD hosts
    - KVM: x86: use Intel speculation bugs and features as derived in generic x86
    - x86/msr: Add the IA32_TSX_CTRL MSR
    - x86/cpu: Add a helper function x86_read_arch_cap_msr()
    - x86/cpu: Add a "tsx=" cmdline option with TSX disabled by default
    - x86/speculation/taa: Add mitigation for TSX Async Abort
    - x86/speculation/taa: Add sysfs reporting for TSX Async Abort
    - kvm/x86: Export MDS_NO=0 to guests when TSX is enabled
    - x86/tsx: Add "auto" option to the tsx= cmdline parameter
    - x86/speculation/taa: Add documentation for TSX Async Abort
    - x86/tsx: Add config options to set tsx=on|off|auto
    - SAUCE: x86/speculation/taa: Call tsx_init()
    - SAUCE: x86/cpu: Include cpu header from bugs.c
    - [Config] Disable TSX by default when possible

  * CVE-2019-0154
    - SAUCE: i915_bpo: drm/i915: Lower RM timeout to avoid DSI hard hangs
    - SAUCE: i915_bpo: drm/i915/gen8+: Add RC6 CTX corruption WA
    - SAUCE: drm/i915/gen8+: Add RC6 CTX corruption WA

  * CVE-2019-0155
    - SAUCE: i915_bpo: drm/i915/gtt: Add read only pages to gen8_pte_encode
    - SAUCE: i915_bpo: drm/i915/gtt: Read-only pages for insert_entries on bdw+
    - SAUCE: i915_bpo: drm/i915/gtt: Disable read-on...

Changed in linux (Ubuntu Xenial):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2019-11-13 04:01 EDT-------
IBM Bugzilla status : Closed, Fix Released with Xenial

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers