[18.04 FEAT] zcrypt DD: introduce APQN tags to support deterministic driver binding

Bug #1799184 reported by bugproxy on 2018-10-22
18
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
High
Canonical Kernel Team
linux (Ubuntu)
High
Joseph Salisbury
Bionic
High
Joseph Salisbury

Bug Description

== SRU Justification ==

APQN tags in the zcrypt device driver are required to support deterministic driver binding

With the introduction of KVM hw crypto virtualization (on s390x) the driver bound to an AP queue device is no longer unique determined.
Therefore a deterministic hot plugging semantics of AP queues that may be bound to multiple drivers is needed.
With the three listed commits here it will be possible to configure an AP queue (APQN) as being bound to a particular driver even if the associate hw gets intermittently lost and reconnected.

== Fixes ==

ac2b96f351d7d222 ("s390/zcrypt: code beautify")
7e0bdbe5c21cb831 ("s390/zcrypt: AP bus support for alternate driver(s)")
3d8f60d38e249f98 ("s390/zcrypt: hex string mask improvements for apmask and aqmask")

== Patches ==

Git-commit: ac2b96f351d7d222
https://github.com/torvalds/linux/commit/ac2b96f351d7d222c46e524feca03005f3fa8d75
Author: Harald Freudenberger <email address hidden>
Date: Fri Aug 17 12:36:01 2018 +0200
s390/zcrypt: code beautify
Code beautify by following most of the checkpatch suggestions:
 - SPDX license identifier line complains by checkpatch
 - missing space or newline complains by checkpatch
 - octal numbers for permssions complains by checkpatch
 - renaming of static sysfs functions complains by checkpatch
 - fix of block comment complains by checkpatch
 - fix printf like calls where function name instead of %s __func__
   was used
 - __packed instead of __attribute__((packed))
 - init to zero for static variables removed
 - use of DEVICE_ATTR_RO and DEVICE_ATTR_RW macros

No functional code changes or API changes!

Signed-off-by: Harald Freudenberger <email address hidden>
Signed-off-by: Martin Schwidefsky <email address hidden>

===

Git-commit 7e0bdbe5c21cb831
https://github.com/torvalds/linux/commit/7e0bdbe5c21cb8316a694e46ad5aad339f6894a6
Author: Harald Freudenberger <email address hidden>
Date: Fri Jul 20 08:36:53 2018 +0200
s390/zcrypt: AP bus support for alternate driver(s)
The current AP bus, AP devices and AP device drivers implementation
uses a clearly defined mapping for binding AP devices to AP device
drivers. So for example a CEX6C queue will always be bound to the
cex4queue device driver.

The Linux Device Driver model has no sensitivity for more than one
device driver eligible for one device type. If there exist more than
one drivers matching to the device type, simple all drivers are tried
consecutively. There is no way to determine and influence the probing
order of the drivers.

With KVM there is a need to provide additional device drivers matching
to the very same type of AP devices. With a simple implementation the
KVM drivers run in competition to the regular drivers. Whichever
'wins' a device depends on build order and implementation details
within the common Linux Device Driver Model and is not
deterministic. However, a userspace process could figure out which
device should be bound to which driver and sort out the correct
binding by manipulating attributes in the sysfs.

If for security reasons a AP device must not get bound to the 'wrong'
device driver the sorting out has to be done within the Linux kernel
by the AP bus code. This patch modifies the behavior of the AP bus
for probing drivers for devices in a way that two sets of drivers are
usable. Two new bitmasks 'apmask' and 'aqmask' are used to mark a
subset of the APQN range for 'usable by the ap bus and the default
drivers' or 'not usable by the default drivers and thus available for
alternate drivers like vfio-xxx'. So an APQN which is addressed by
this masking only the default drivers will be probed. In contrary an
APQN which is not addressed by the masks will never be probed and
bound to default drivers but onny to alternate drivers.

Eventually the two masks give a way to divide the range of APQNs into
two pools: one pool of APQNs used by the AP bus and the default
drivers and thus via zcrypt drivers available to the userspace of the
system. And another pool where no zcrypt drivers are bound to and
which can be used by alternate drivers (like vfio-xxx) for their
needs. This division is hot-plug save and makes sure a APQN assigned
to an alternate driver is at no time somehow exploitable by the wrong
party.

The two masks are located in sysfs at /sys/bus/ap/apmask and
/sys/bus/ap/aqmask. The mask syntax is exactly the same as the
already existing mask attributes in the /sys/bus/ap directory (for
example ap_usage_domain_mask and ap_control_domain_mask).

By default all APQNs belong to the ap bus and the default drivers:

  cat /sys/bus/ap/apmask
  0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
  cat /sys/bus/ap/aqmask
  0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

The masks can be changed at boot time with the kernel command line
like this:

  ... ap.apmask=0xffff ap.aqmask=0x40

This would give these two pools:

  default drivers pool: adapter 0 - 15, domain 1
  alternate drivers pool: adapter 0 - 15, all but domain 1
      adapter 16-255, all domains

The sysfs attributes for this two masks are writeable and an
administrator is able to reconfigure the assignements on the fly by
writing new mask values into. With changing the mask(s) a revision of
the existing queue to driver bindings is done. So all APQNs which are
bound to the 'wrong' driver are reprobed via kernel function
device_reprobe() and thus the new correct driver will be assigned with
respect of the changed apmask and aqmask bits.

The mask values are bitmaps in big endian order starting with bit 0.
So adapter number 0 is the leftmost bit, mask is 0x8000... The sysfs
attributes accept 2 different formats:
- Absolute hex string starting with 0x like "0x12345678" does set
  the mask starting from left to right. If the given string is shorter
  than the mask it is padded with 0s on the right. If the string is
  longer than the mask an error comes back (EINVAL).
- '+' or '-' followed by a numerical value. Valid examples are "+1",
  "-13", "+0x41", "-0xff" and even "+0" and "-0". Only the addressed
  bit in the mask is switched on ('+') or off ('-').

This patch will also be the base for an upcoming extension to the
zcrypt drivers to be able to provide additional zcrypt device nodes
with filtering based on ap and aq masks.

Signed-off-by: Harald Freudenberger <email address hidden>
Signed-off-by: Martin Schwidefsky <email address hidden>

===

Git-commit 3d8f60d38e249f98
https://github.com/torvalds/linux/commit/3d8f60d38e249f989a7fca9c2370c31c3d5487e1
Author: Harald Freudenberger <email address hidden>
Date: Mon Aug 20 15:27:45 2018 +0200
s390/zcrypt: hex string mask improvements for apmask and aqmask
The sysfs attributes /sys/bus/ap/apmask and /sys/bus/ap/aqmask
and the kernel command line arguments ap.apm and ap.aqm get
an improvement of the value parsing with this patch:

The mask values are bitmaps in big endian order starting with bit 0.
So adapter number 0 is the leftmost bit, mask is 0x8000... The sysfs
attributes and the kernel command line accept 2 different formats:
 - Absolute hex string starting with 0x like "0x12345678" does set
   the mask starting from left to right. If the given string is shorter
   than the mask it is padded with 0s on the right. If the string is
   longer than the mask an error comes back (EINVAL).
 - Relative format - a concatenation (done with ',') of the terms
   +<bitnr>[-<bitnr>] or -<bitnr>[-<bitnr>]. <bitnr> may be any
   valid number (hex, decimal or octal) in the range 0...255.
   Here are some examples:
     "+0-15,+32,-128,-0xFF"
     "-0-255,+1-16,+0x128"

Signed-off-by: Harald Freudenberger <email address hidden>
Signed-off-by: Martin Schwidefsky <email address hidden>

== Regression Potential ==

Mid to low, because:

- There are quite some changes, but everything is limited to s390/zcrypt, means:
   arch/s390/include/uapi/asm/*
   and
   drivers/s390/crypto/*
- So everything is s390x specific.
- And larger parts of the changes are beautifications.
- It's all upstream accepted in 4.19
- The code is already successfully test, integrated and running in cosmic.

== Test Case ==

On an LPAR that has access to an CryptoExpress card,

execute the 'lszcrypt --verbose'
and check availability of:
/sys/bus/ap/apmask

General test and verification was already done by IBM.
Please notice that the code already landed in cosmic.
__________

With the introduction of KVM crypto virtualization the driver bound to an AP queue device is no longer unique determined.
This feature provides a deterministic hot plugging semantics of AP queues that may be bound to multiple drivers.
In particular it enables to configure an AP queue (APQN) as being bound to a particular driver even if the associate HW gets intermittently lost and reconnected.

This function has to be applied to kernel 4.15 for Ubuntu 18.04,

Git Commit information available here:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1784331

CVE References

bugproxy (bugproxy) on 2018-10-22
tags: added: architecture-s39064 bugnameltc-172503 severity-high targetmilestone-inin1804
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Frank Heimes (frank-heimes) wrote :

According to
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1784331/comments/5
the three needed commits are:
1) Git-commit: ac2b96f351d7d222
   s390/zcrypt: code beautify"
2) Git-commit 7e0bdbe5c21cb831
   s390/zcrypt: AP bus support for alternate driver(s)"
3) Git-commit 3d8f60d38e249f98 s390/zcrypt:
   hex string mask improvements for apmask and aqmask
Hope they cleanly apply to 4.15, too.

Changed in ubuntu-z-systems:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
information type: Private → Public
Changed in linux (Ubuntu):
status: New → Triaged
importance: Undecided → High
assignee: Skipper Bug Screeners (skipper-screen-team) → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Bionic):
status: New → In Progress
Changed in linux (Ubuntu):
status: Triaged → In Progress
Changed in linux (Ubuntu Bionic):
importance: Undecided → High
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in ubuntu-z-systems:
status: Triaged → In Progress

------- Comment From <email address hidden> 2018-10-31 11:28 EDT-------
In addition s390tools also need an update
Backport from 2.7.0 to 2.3.0

Git commit: for s390-tools 2.7.0
https://github.com/ibm-s390-tools/s390-tools/commit/0a0b4c382693cded5652404e8fa2c0e483aa33df
lszcrypt: support for alternate zcrypt device drivers

For 2.3.0 Ubuntu 18.04, maybe you also need to pull addl. commits within the subdir zconf/zcyrpt.

Joseph Salisbury (jsalisbury) wrote :

I built a test kernel with the commits posed in comment 1. Commits 2c957a8ad45 and 4a07750ba8f3f are needed as a prereqs as well. The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1799184

Can you test this kernel and see if it resolves this bug?

Note about installing test kernels:
* If the test kernel is prior to 4.15(Bionic) you need to install the linux-image and linux-image-extra .deb packages.
* If the test kernel is 4.15(Bionic) or newer, you need to install the linux-modules, linux-modules-extra and linux-image-unsigned .deb packages.

Thanks in advance!

description: updated
description: updated
Frank Heimes (frank-heimes) wrote :

SRU request:
Subject: [SRU][Bionic][PATCH 0/3] Fixes for LP1799184
https://lists.ubuntu.com/archives/kernel-team/2018-November/096438.html

Stefan Bader (smb) on 2018-11-08
Changed in linux (Ubuntu Bionic):
status: In Progress → Fix Committed
Changed in linux (Ubuntu):
status: In Progress → Fix Committed
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
Frank Heimes (frank-heimes) wrote :

I was able to successfully verify the patches and can now see the apmask and aqmask in /sys/bus/ap using the kernel from bionic-proposed (4.15.0-42-generic).
$ cat /sys/bus/ap/aqmask
0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
$ cat /sys/bus/ap/apmask
0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
Adjusting tags accordingly.

tags: added: verification-done-bionic
removed: verification-needed-bionic
Launchpad Janitor (janitor) wrote :
Download full text (3.1 KiB)

This bug was fixed in the package linux - 4.15.0-42.45

---------------
linux (4.15.0-42.45) bionic; urgency=medium

  * linux: 4.15.0-42.45 -proposed tracker (LP: #1803592)

  * [FEAT] Guest-dedicated Crypto Adapters (LP: #1787405)
    - KVM: s390: reset crypto attributes for all vcpus
    - KVM: s390: vsie: simulate VCPU SIE entry/exit
    - KVM: s390: introduce and use KVM_REQ_VSIE_RESTART
    - KVM: s390: refactor crypto initialization
    - s390: vfio-ap: base implementation of VFIO AP device driver
    - s390: vfio-ap: register matrix device with VFIO mdev framework
    - s390: vfio-ap: sysfs interfaces to configure adapters
    - s390: vfio-ap: sysfs interfaces to configure domains
    - s390: vfio-ap: sysfs interfaces to configure control domains
    - s390: vfio-ap: sysfs interface to view matrix mdev matrix
    - KVM: s390: interface to clear CRYCB masks
    - s390: vfio-ap: implement mediated device open callback
    - s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl
    - s390: vfio-ap: zeroize the AP queues
    - s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl
    - KVM: s390: Clear Crypto Control Block when using vSIE
    - KVM: s390: vsie: Do the CRYCB validation first
    - KVM: s390: vsie: Make use of CRYCB FORMAT2 clear
    - KVM: s390: vsie: Allow CRYCB FORMAT-2
    - KVM: s390: vsie: allow CRYCB FORMAT-1
    - KVM: s390: vsie: allow CRYCB FORMAT-0
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1
    - KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2
    - KVM: s390: device attrs to enable/disable AP interpretation
    - KVM: s390: CPU model support for AP virtualization
    - s390: doc: detailed specifications for AP virtualization
    - KVM: s390: fix locking for crypto setting error path
    - KVM: s390: Tracing APCB changes
    - s390: vfio-ap: setup APCB mask using KVM dedicated function
    - s390/zcrypt: Add ZAPQ inline function.
    - s390/zcrypt: Review inline assembler constraints.
    - s390/zcrypt: Integrate ap_asm.h into include/asm/ap.h.
    - s390/zcrypt: fix ap_instructions_available() returncodes
    - s390/zcrypt: remove VLA usage from the AP bus
    - s390/zcrypt: Remove deprecated ioctls.
    - s390/zcrypt: Remove deprecated zcrypt proc interface.
    - s390/zcrypt: Support up to 256 crypto adapters.
    - [Config:] Enable CONFIG_S390_AP_IOMMU and set CONFIG_VFIO_AP to module.

  * Bypass of mount visibility through userns + mount propagation (LP: #1789161)
    - mount: Retest MNT_LOCKED in do_umount
    - mount: Don't allow copying MNT_UNBINDABLE|MNT_LOCKED mounts

  * CVE-2018-18955: nested user namespaces with more than five extents
    incorrectly grant privileges over inode (LP: #1801924) // CVE-2018-18955
    - userns: also map extents in the reverse map to kernel IDs

  * kdump fail due to an IRQ storm (LP: #1797990)
    - SAUCE: x86/PCI: Export find_cap() to be used in early PCI code
    - SAUCE: x86/quirks: Add parameter to clear MSIs early on boot
    - SAUCE: x86/quirks: Scan all busses for early PCI quirks

 -- Thadeu Lima de Souza Cascardo <email address hidden> Thu, 15 Nov 2018 17:01:46 ...

Read more...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
Frank Heimes (frank-heimes) wrote :

Just verified that the 3 patches (bug description / SRU template) are included in kernel 4.19 and since 4.19 laded in disco proposed today, I'm changing the kernel entry to Fix Released (code is available in cosmic, too).
Changing project entry to Fix Released, too.

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers