[FEAT] Guest-dedicated Crypto Adapters

Bug #1787405 reported by bugproxy
30
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
Fix Released
High
Canonical Kernel Team
libvirt (Ubuntu)
Fix Released
Undecided
Christian Ehrhardt 
Bionic
Fix Released
Undecided
Unassigned
Cosmic
Fix Released
Undecided
Unassigned
linux (Ubuntu)
Fix Released
High
Joseph Salisbury
Bionic
Fix Released
High
Joseph Salisbury
Cosmic
Fix Released
High
Joseph Salisbury
Disco
Fix Released
High
Joseph Salisbury
qemu (Ubuntu)
Fix Released
Undecided
Christian Ehrhardt 
Bionic
Fix Released
Undecided
Unassigned
Cosmic
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

 * The ability to pass through more cryptographic capabilities is a very
   important feature for users of s390x as virtualization platform.
   Its availability upstream now and its backport in this bug allows to
   exploit the crypto cards as new HW for these virtualization use
   cases.

 * This falls under both "other safe cases" SRU exceptions:
    - For Long Term Support releases we regularly want to enable new
      hardware ...
    - For Long Term Support releases we sometimes want to introduce new
      features. They must not change the behaviour on existing
      installations ...

 * This bug has three main components:
   - kernel (ability to do all of this)
   - qemu (add feature to exploit the new code)
   - libvirt (make the feature user consumable)

[Test Case]

 * In general this consists of a few steps
   - get the updated kernel/qemu/libvirt
   - mask the card & domains from the usual driver
   - load vfio-ap
   - assign card&domain to vfio-ap
   - prepare a guest
   - configure a guest to use the card

 * See comment #66 how to do all of that in detail

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1787405/comments/66

[Regression Potential]

 * The changes are mostly s390x only and adding a new feature so
   regressions to existing components should be low. But to backport it
   slight changes to the MDEV handling had to be applied as well.
   The potential regressions I can see are in that MDEV handling if one
   of the backports would be bad.
   Fortunately we know that without the related libvirt fixes we added
   here using MDEVs didn't work at all yet, and people very rarely use
   qemu without libvirt for anything else than experiments.
   Therefore I'm confident that even if there would be a flaw in the
   MDEV changes no one is hugely relying on it.

[Other Info]

 * n/a

== SRU Justification ==

(Kernel SRU)

Allow kvm to dedicate crypto adapters (and domains) as passthrough devices to a KVM guest such that the hypervisor cannot observe the communication of the guest with the device.
(Since all kernel patches/commits are from kernel 4.19, they will automagically land in 'Disco'.)

== Fix ==

9ea5972 ("KVM: s390: vsie: simulate VCPU SIE entry/exit")
3194cdb ("KVM: s390: introduce and use KVM_REQ_VSIE_RESTART")
e585b24 ("KVM: s390: refactor crypto initialization")
1fde573 ("s390: vfio-ap: base implementation of VFIO AP device driver")
65f0671 ("s390: vfio-ap: register matrix device with VFIO mdev framework")
96d152b ("s390: vfio-ap: sysfs interfaces to configure adapters")
3211da0 ("s390: vfio-ap: sysfs interfaces to configure domains")
3b1eab7 ("s390: vfio-ap: sysfs interfaces to configure control domains")
81b2b4b ("s390: vfio-ap: sysfs interface to view matrix mdev matrix")
4210459 ("KVM: s390: interface to clear CRYCB masks")
258287c ("s390: vfio-ap: implement mediated device open callback")
e06670c ("s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl")
46a7263 ("s390: vfio-ap: zeroize the AP queues")
cd8a377 ("s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl")
6cc571b ("KVM: s390: Clear Crypto Control Block when using vSIE")
d6f6959 ("KVM: s390: vsie: Do the CRYCB validation first")
3af84de ("KVM: s390: vsie: Make use of CRYCB FORMAT2 clear")
56019f9 ("KVM: s390: vsie: Allow CRYCB FORMAT-2")
19fd83a ("KVM: s390: vsie: allow CRYCB FORMAT-1")
6ee7409 ("KVM: s390: vsie: allow CRYCB FORMAT-0")
c9ba8c2 ("KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1")
6b79de4 ("KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2")
9ee71f2 ("KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2")
37940fb ("KVM: s390: device attrs to enable/disable AP interpretation")
112c24d ("KVM: s390: CPU model support for AP virtualization")
492a6be ("s390: doc: detailed specifications for AP virtualization")

<-- till here in 'kvm/next' (https://git.kernel.org/pub/scm/virt/kvm/kvm.git/) -->

8e41bd5 ("KVM: s390: fix locking for crypto setting error path")
0e237e4 ("KVM: s390: Tracing APCB changes")
76c7829 ("s390: vfio-ap: setup APCB mask using KVM dedicated function")

<-- till here in 'kvms390/next' (https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git/) -->

<-- In addition to that some prereqs for the 'ap/crypto' driver are necessary -->

ea3c418 ("s390/zcrypt: Add ZAPQ inline function.")
df80c03 ("s390/zcrypt: Review inline assembler constraints.")
f1b0a43 ("s390/zcrypt: Integrate ap_asm.h into include/asm/ap.h.")
2395103 ("s390/zcrypt: fix ap_instructions_available() returncodes")
7e0bdbe ("s390/zcrypt: AP bus support for alternate driver(s)")
3d8f60d3 ("s390/zcrypt: hex string mask improvements for apmask and aqmask.")
fa108f9 ("s390/zcrypt: remove VLA usage from the AP bus")

<-- https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1787405/comments/12 -->

== PATCH ==

Above git commits are all from 4.19.
The git commands for 4.18 would be:

$ git cherry-pick <all from 'kvm/next' list>

(112c24d "KVM: s390: CPU model support for AP virtualization" may have a trivial merge conflict with the etoken patch)

$ git cherry-pick <all from 'kvms390/next' list>

$ git cherry-pick <all from 'ap/zcrypt' list>

== Regression Potential ==

Low to mid:

- mid because in summary there are a lot of changes, but low
- they are all limited to the s390x architecture
- and again limited to KVM/s390x, vfio-ap and the zcrypt (aka ap) driver
- Test kernel was built for testting.

== Test Case ==

Setup a system for KVM use on an s390x LPAR that has CryptoExpress (aka crypto-) adapters installed.
Verify that the AP bus created a sysfs device for each APQN, like:
/sys/devices/ap/card04/04.0006
/sys/devices/ap/card04/04.0047
/sys/devices/ap/card0a/0a.0006
/sys/devices/ap/card0a/0a.0047
Verify the APQN range via the following two sysfs files:
/sys/bus/ap/apmask
/sys/bus/ap/aqmask
Configure and start a guest.
More details see: 492a6be ("s390: doc: detailed specifications for AP virtualization")
But for that an updated qemu and libvirt should be in place - that's addressed in LP1787405, too.
(So this is only the kernel part of that ticket.)
__________

Description:
Allow kvm to dedicate crypto adapters (and domains) as passthrough devices to a KVM guest such that the hypervisor cannot observe the communication of the guest with the device.

This functionality will be contribute to following packages.
--kernel, qemu and libvirt.

Currently these functions are not finalized and therefore no git-commit are avalable,
- kernel > 4.19
- libvirt > 4.6.0
- qemu > 3.0

We will provide these as soon as possible.

This request is launched against Ubuntu 18.10 to fulllfil the feature integration process of Canonical.
But the main intention is, to get this integrated into 18.04 LTS !!!!!!

Thererfore, the backports will be required for both distros.!

bugproxy (bugproxy)
tags: added: architecture-s39064 bugnameltc-170621 severity-high targetmilestone-inin1810
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → linux (Ubuntu)
Changed in ubuntu-z-systems:
importance: Undecided → High
Revision history for this message
Andrew Cloke (andrew-cloke) wrote :

Marking as incomplete until the git-commits are available.

Changed in ubuntu-z-systems:
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
status: New → Incomplete
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Kernel backports are part of HWE stack, available from updates.
libvirt/qemu are part of the cloud-archive backports.
Thus things like this will land in 18.04 via above repositories - if this makes it into 18.10, if not later with 19.04 backports.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

Given qemu v3 is targetted to 19.04, shouldn't this ticket be bumped to 19.04 as well?

Changed in linux (Ubuntu):
status: New → Incomplete
Changed in libvirt (Ubuntu):
status: New → Incomplete
Changed in qemu (Ubuntu):
status: New → Incomplete
Revision history for this message
Andrew Cloke (andrew-cloke) wrote :

IBM, could you comment on the upstream status of these changes?

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-08-21 03:51 EDT-------
@Xnox: due to the fact, that this request is mainly for 18.04, but need to be integrated in 18.10 first, as described within the 1st Entry of the ticket, a 19.04 target is definedly too late.
Once these functions are available, backports are required to get this into 18.10 and than into 18.04(main target).

These features is required in support of the Secure Service Container integration!.
And therefore currently the highest priority from IBM currently....

Do I need reflect this priority in a specific way..?

------- Comment From <email address hidden> 2018-08-21 03:55 EDT-------
@Andrew: We will keep Canonical informed once upstream commits are available and able to be used for further processing.... Thx

Revision history for this message
Dimitri John Ledkov (xnox) wrote : Re: [18.10 FEAT] Guest-dedicated Crypto Adapters

Merge window is almost closed. Has this landed into v4.19? Was this part of the:

commit e1dbc5a41051d4791160727829903ec5169c7152
Merge: 40c431a5a5c2 4ec84835900b
Author: Linus Torvalds <email address hidden>
Date: Fri Aug 24 09:31:34 2018 -0700

    Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux

    Pull s390 updates from Martin Schwidefsky:

     - A couple of patches for the zcrypt driver:
         + Add two masks to determine which AP cards and queues are host
           devices, this will be useful for KVM AP device passthrough
         + Add-on patch to improve the parsing of the new apmask and aqmask
         + Some code beautification

and so on?

Do you have the full list of linux kernel commit ids please?

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-08-27 02:21 EDT-------
These commits are only a small part of the feature. The main bulk of the patches is still being discussed, see
https://<email address hidden>/

Revision history for this message
Dimitri John Ledkov (xnox) wrote : Re: [18.10 FEAT] Guest-dedicated Crypto Adapters

@cborntra

Ok, in that case it's not in v4.19 right?

Then @hws should bump this feature to 19.04 target.

We would not really want to backport and integrate things into v4.18 kernel for 18.10 that may change whilst still trying to land into mainline.

And definitely will not backport incomplete/non-mainline things into 18.04 ga kernel.

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-09-03 10:35 EDT-------
Changed distro target to 19.04, will not make it into 18.10....

tags: added: targetmilestone-inin1904
removed: targetmilestone-inin1810
Frank Heimes (fheimes)
summary: - [18.10 FEAT] Guest-dedicated Crypto Adapters
+ [19.04 FEAT] Guest-dedicated Crypto Adapters
Revision history for this message
Dimitri John Ledkov (xnox) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

Please provide us with proprietary early access to:
- linux patches
- qemu patches
- libvirt patches
- user-facing documentation
- feature / PR description

expectations of how far up the stack it is expected to be integrated, to properly discuss, scope, and agree on the plan of action and timelines. for example if it is expected to use supplementary archives and packages (like the cloud-archive, for qemu/libvirt, and hwe kernel flavour - or not) and so on and so forth. Is integration expected in Openstack? MAAS pods? Who will do integration and backports? So on and so forth.

On Canonical side, we will at least need to schedule this work across foundations, kernel, server and openstack teams most likely.

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-10-05 07:42 EDT-------
The kernel part is now part of kvm/next scheduled for 4.20 (or whatever version number that will be)

https://git.kernel.org/pub/scm/virt/kvm/kvm.git/commit/?h=next&id=55d09dd4c86060fbbc74ab2b1bfaed401cd0163a

As of today 3 followup patch are scheduled as well.
see https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git/log/?h=apv11

for these additional patches (and the original ones).

------- Comment From <email address hidden> 2018-10-05 07:44 EDT-------
we are working on qemu patch (currently v9 on the qemu list, with a v10 expected to be merged for 3.1).
libvirt patches are also in preparation.

Revision history for this message
bugproxy (bugproxy) wrote :
Download full text (3.4 KiB)

------- Comment From <email address hidden> 2018-10-08 09:18 EDT-------
For reference, for the kernel on top of v4.18 the following commits are necessary:

9ea597286570 KVM: s390: vsie: simulate VCPU SIE entry/exit
3194cdb71190 KVM: s390: introduce and use KVM_REQ_VSIE_RESTART
e585b24aeb44 KVM: s390: refactor crypto initialization
1fde573413b5 s390: vfio-ap: base implementation of VFIO AP device driver
65f06713d3fa s390: vfio-ap: register matrix device with VFIO mdev framework
96d152bdc987 s390: vfio-ap: sysfs interfaces to configure adapters
3211da0c0b54 s390: vfio-ap: sysfs interfaces to configure domains
3b1eab7fb9da s390: vfio-ap: sysfs interfaces to configure control domains
81b2b4b76a73 s390: vfio-ap: sysfs interface to view matrix mdev matrix
42104598ef2e KVM: s390: interface to clear CRYCB masks
258287c994de s390: vfio-ap: implement mediated device open callback
e06670c5fe3b s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl
46a7263d4746 s390: vfio-ap: zeroize the AP queues
cd8a377e3b40 s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl
6cc571b1b1e8 KVM: s390: Clear Crypto Control Block when using vSIE
d6f6959ac587 KVM: s390: vsie: Do the CRYCB validation first
3af84def9cbf KVM: s390: vsie: Make use of CRYCB FORMAT2 clear
56019f9aca22 KVM: s390: vsie: Allow CRYCB FORMAT-2
19fd83a64718 KVM: s390: vsie: allow CRYCB FORMAT-1
6ee74098201b KVM: s390: vsie: allow CRYCB FORMAT-0
c9ba8c2cd210 KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1
6b79de4b056e KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2
9ee71f20cb8d KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2
37940fb0b6a2 KVM: s390: device attrs to enable/disable AP interpretation
112c24d4dc48 KVM: s390: CPU model support for AP virtualization
492a6be197c0 s390: doc: detailed specifications for AP virtualization
<-- till here in kvm next (https://git.kernel.org/pub/scm/virt/kvm/kvm.git/)

8e41bd54317b KVM: s390: fix locking for crypto setting error path
0e237e446994 KVM: s390: Tracing APCB changes
76c7829f5b8c s390: vfio-ap: setup APCB mask using KVM dedicated function

<-- til here in kvms390/next
(https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git/)

In addition to that some prereqs for the ap/crypto driver are necessary

ea3c4185efb3 s390/zcrypt: Add ZAPQ inline function.
df80c0383133 s390/zcrypt: Review inline assembler constraints.
f1b0a4343c41 s390/zcrypt: Integrate ap_asm.h into include/asm/ap.h.
2395103b3fbf s390/zcrypt: fix ap_instructions_available() returncodes
7e0bdbe5c21c s390/zcrypt: AP bus support for alternate driver(s)
3d8f60d38e24 s390/zcrypt: hex string mask improvements for apmask and aqmask.
fa108f95c676 (tag: s390-4.19-3) s390/zcrypt: remove VLA usage from the AP bus

So the git commands for 4.18 would be:

$ git cherry-pick 9ea597286570 3194cdb71190 e585b24aeb44 1fde573413b5 \
65f06713d3fa 96d152bdc987 3211da0c0b54 3b1eab7fb9da 81b2b4b76a73 42104598ef2e \
258287c994de e06670c5fe3b 46a7263d4746 cd8a377e3b40 6cc571b1b1e8 d6f6959ac587 \
3af84def9cbf 56019f9aca22 19fd83a64718 6ee74098201b c9ba8c2cd210 6b79de4b056e \
9ee71f20cb8d 37940fb0b6a2 112c24d4dc48 492a6be197c0 # kvm/next

(112c24d... KVM: s390: CPU model support fo...

Read more...

Revision history for this message
Dimitri John Ledkov (xnox) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

This is a very useful list of commits, thank you!

All of this will be in v4.20 right? For 19.04 we might even target to land v4.21 (possibly named v5.1).

What about qemu, libvirt patches?
Is everything already in qemu v3.0.0 & libvirt 4.8.0?

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-10-09 13:16 EDT-------
Yes, this should all land in 4.20. qemu will likely be 3.1. libvirt soon

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-10-12 10:10 EDT-------
QEMU patch set has been merged
https://git.qemu.org/?p=qemu.git;a=commit;h=69ac8c4cb93f2685839ff7b857cef306b388ff3c

Frank Heimes (fheimes)
Changed in libvirt (Ubuntu):
status: Incomplete → Invalid
Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

Per Call HWS<->JFH libvirt seems no more need any code, so set it to invalid in the context of this bug.

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: Incomplete → Triaged
Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Frank Heimes (fheimes)
information type: Private → Public
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-10-30 11:51 EDT-------
the qemu patches are (in reverse order)
694a8d703b s390: doc: detailed specifications for AP virtualization
2fe2942cd6 s390x/vfio: ap: Introduce VFIO AP device
a51b31535a s390x/ap: base Adjunct Processor (AP) object model
1d7db85b61 s390x/kvm: enable AP instruction interpretation for guest
c5cd17afdd s390x/cpumodel: Set up CPU model for AP device support
8f3cd250a8 linux-headers: update

There are some minor merge conflicts that are easy to solve when merging into 3.0,2.12 or older

Changed in linux (Ubuntu):
importance: Undecided → High
status: Confirmed → Triaged
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-10-30 12:04 EDT-------
the libvirt patches have landed as well
a017bae1ae news: Update news for vfio-ap support
1170864198 qemu: vfio-ap device support
dc788d2540 qemu: add vfio-ap capability

tags: added: libvirt-19.04 qemu-19.04
Changed in linux (Ubuntu):
assignee: Skipper Bug Screeners (skipper-screen-team) → Joseph Salisbury (jsalisbury)
Revision history for this message
Frank Heimes (fheimes) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

@CB: Hi, does the list of kernel patches for 4.18 (from comment #12) also cleanly apply to 4.15 from bionic (since IBM asks to finally have it in bionic)? Or are additional ones needed?
Can you provide a brief test case description (and maybe some docu snippets) that allows to do a brief test of the guest-dedicated Crypto Adapters once everything is in place? (That's required for the SRU request.)

description: updated
description: updated
Revision history for this message
Frank Heimes (fheimes) wrote :

Just found some docu snippets from commit descriptions, especially the 's390: doc' commit ...

Frank Heimes (fheimes)
description: updated
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-02 10:15 EDT-------
The list is also valid for 4.15. Please note that this still has the same dependencies on the crypto ap driver. (I think there is a separate feature request for that). I already mentioned these commit ids.

There are some more commit in the crypto area between 4.15 and 4.18 but none of thoese seems to be required. Harald, can you confirm?

Revision history for this message
Dimitri John Ledkov (xnox) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

Confirming libvirt task back on, as there are appear to be a few small patches for libvirt.

Changed in libvirt (Ubuntu):
status: Invalid → Confirmed
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-02 10:52 EDT-------
We also need these commits: (before the remaining ones)

20c922f04b17 KVM: s390: reset crypto attributes for all vcpus
to make the KVM commits apply cleanly. (the first one has a simple merge conflict)

The AP patches also have some minor conflicts due to the missing
efda7adec7a5 s390/zcrypt: Make ap init functions static.
d485235b0054 s390: assume diag308 set always works

but it is probably simpler to fixup the patches.

------- Comment From <email address hidden> 2018-11-02 11:03 EDT-------
We also need a define that is added with

af4a72276d49 s390/zcrypt: Support up to 256 crypto adapters.
(to fit on 4.15 you would then need
71cbbff8c4fd s390/zcrypt: Remove deprecated zcrypt proc interface.
2a80786d477a s390/zcrypt: Remove deprecated ioctls.
)

The alternative is to define MAX_ZDEV_ENTRIES_EXT but just cherry-picking these 3 commits is probably less risky.

We then need the following kernel config options.

CONFIG_VFIO_AP
CONFIG_VFIO_MDEV
CONFIG_VFIO_MDEV_DEVICE
CONFIG_S390_AP_IOMMU=y

in the kernel config.

With that I can use crypto cards with the bionic kernel

Revision history for this message
Joseph Salisbury (jsalisbury) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

I tried to apply the first two commits listed in comment #23. Neither apply cleanly and need to be backported. Would it be possible for IBM to provide a backport of any commits that don't apply cleanly?

This bug will be discussed on Monday in the kernel teams work items meeting, due to the number of commits needed in Bionic(4.15), which is a stable release.

tags: added: kernel-key
Revision history for this message
Terry Rudd (terrykrudd) wrote : Re: [Bug 1787405] Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

Hi Joe,

If IBM does not do the backport, just how much work do you think there
is on that whole effort?  That list of patches was pretty impressive in
my eyes.

Terry
On 11/02/2018 10:43 AM, Joseph Salisbury wrote:
> I tried to apply the first two commits listed in comment #23. Neither
> apply cleanly and need to be backported. Would it be possible for IBM
> to provide a backport of any commits that don't apply cleanly?
>
> This bug will be discussed on Monday in the kernel teams work items
> meeting, due to the number of commits needed in Bionic(4.15), which is a
> stable release.
>
> ** Tags added: kernel-key
>

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-05 04:04 EDT-------
I gave this a quick spin. The resulting backport on top of the bionic master branch is at

https://git.kernel.org/pub/scm/linux/kernel/git/borntraeger/linux.git/log/?h=apbionic

Feel free to use this branch as a "cheat sheet" for the patches that need backport.

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-05 04:58 EDT-------
FWIW, parts of the commits mentioned here are already part of IBM Bug 172503 - LP1799184.

Revision history for this message
Joseph Salisbury (jsalisbury) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

Thanks for the backports! Were you able to get a kernel to build with these commits? I applied the 40 commits to a Bionic tree, but it fails to build.

I'll see if I can figure out why my build failed.

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-05 13:57 EDT-------
I build the kernel on a different system (not bionic) but yes it built fine. What config and what compile error do you have?

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-05 14:25 EDT-------
so I retried on bionic:
# cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.1 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.1 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

git head= 3dfd30e6cf9cbd8dcac852f959d08eeba0e0fafd (branch apbionic from my tree)

# diff -u /boot/config-4.15.0-29-generic .config | grep AP
+CONFIG_VFIO_AP=m
+CONFIG_S390_AP_IOMMU=y

This kernel builds fine

Can you maybe compare your branch against my branch (folders arch/s390/kvm/ and drivers/s390/crypto/ )

Revision history for this message
Joseph Salisbury (jsalisbury) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

Yeah, it looks like it's failing due to that config option:
IOMMU Hardware Support (IOMMU_SUPPORT) [Y/?] y
  S390 CCW IOMMU Support (S390_CCW_IOMMU) [Y/n/?] y
  S390 AP IOMMU Support (S390_AP_IOMMU) [N/y/?] (NEW) aborted!

Console input/output is redirected. Run 'make oldconfig' to update configuration.

I should be able to fix this and have a test kernel shortly.

Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

I built a test kernel with those two config options enabled and the 40 commits. The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1787405

Can you test this kernel and see if it resolves this bug?

Note about installing test kernels:
• If the test kernel is prior to 4.15(Bionic) you need to install the linux-image and linux-image-extra .deb packages.
• If the test kernel is 4.15(Bionic) or newer, you need to install the linux-modules, linux-modules-extra and linux-image-unsigned .deb packages.

Thanks in advance!

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-06 11:01 EDT-------
Thanks for providing the kernel image so quickly.
I successfully tested the AP passthrough function using the following components:

Distribution:
Ubuntu 18.04 LTS
Host kernel:
Linux KVMCrypto 4.15.0-38-generic #42~lp1787405 SMP Mon Nov 5 21:13:01 UTC 2018 s390x s390x s390x GNU/Linux
KVM guest kernel:
Linux f6c59abfb01a 4.15.0-38-generic #41-Ubuntu SMP Wed Oct 10 10:57:21 UTC 2018 s390x Linux
Qemu:
QEMU emulator version 3.0.50 (v3.0.0-1732-gef30274865-dirty)

Thanks!

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-07 10:16 EDT-------
I installed a fresh Ubuntu 18.04.1 on a LPAR and after booting
these both packages on top:

linux-image-4.15.0-38-generic_4.15.0-38.42~lp1787405_s390x.deb
linux-modules-4.15.0-38-generic_4.15.0-38.42~lp1787405_s390x.deb

then I needed to configure zipl to something usefull as the modified zipl.conf obviously is somewhat broken after package install:

--------------------------------------------------------------------------------
[defaultboot]
defaultmenu=menu

[UBUNTU18.04.1]
target=/boot
image=/boot/vmlinuz.old
parameters="scsi_mod.scsi_logging_level=4605 printk.time=1 zfcp.dbfsize=100 root=/dev/disk/by-path/ccw-0.0.e96b-part1"
ramdisk=/boot/initrd.img.old

[newkernel]
target=/boot
image=/boot/vmlinuz
parameters="scsi_mod.scsi_logging_level=4605 printk.time=1 zfcp.dbfsize=100 root=/dev/disk/by-path/ccw-0.0.e96b-part1"
ramdisk=/boot/initrd.img

:menu
target=/boot
1 = UBUNTU18.04.1
2 = newkernel
default = 2
prompt = 1
timeout = 10

--------------------------------------------------------------------------------

after boot the new kernel is active:

uname -a
Linux s83lp75 4.15.0-38-generic #42~lp1787405 SMP Mon Nov 5 21:13:01 UTC 2018 s390x s390x s390x GNU/Linux

then I ran my brand new developed zcrypttest and all the testcases ran fine.
This is at least an indication that the zcrypt dd is not broken, multi domain and multi adapter works and all the 3 kinds of adapters can get addressed with all the different cprbs and work as expected. Even more some basic assumptions about request scheduling memory consumptions are tested.

What's not covered is the new functionallity coming with the apmask and aqmask. I'll do this later as I'd like to devel some testcases for this feature in the next days.

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-08 10:35 EDT-------
I successfully tested the guest support on backport for ap on the 18.04 ubuntu kernel

Revision history for this message
Frank Heimes (fheimes) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters
Changed in linux (Ubuntu):
status: Triaged → In Progress
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

The SRU request has been submitted. However, due to the number of changes, and this being a stable release, a code review of these patches should be done. Is it possible for you to do that and post the outcome here?

https://lists.ubuntu.com/archives/kernel-team/2018-November/096675.html

no longer affects: qemu (Ubuntu Bionic)
no longer affects: qemu (Ubuntu Cosmic)
no longer affects: qemu (Ubuntu Disco)
no longer affects: libvirt (Ubuntu Disco)
no longer affects: libvirt (Ubuntu Cosmic)
no longer affects: libvirt (Ubuntu Bionic)
Changed in linux (Ubuntu Cosmic):
status: New → Triaged
Changed in linux (Ubuntu Bionic):
status: New → Triaged
Changed in linux (Ubuntu Cosmic):
importance: Undecided → High
Changed in linux (Ubuntu Bionic):
importance: Undecided → High
Changed in linux (Ubuntu Cosmic):
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Bionic):
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Cosmic):
status: Triaged → In Progress
status: In Progress → Triaged
Changed in linux (Ubuntu Bionic):
status: Triaged → In Progress
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-09 11:53 EDT-------
Some comments:
1. the majority of the code is in one new device driver and a Documentation file
2. The code review was done upstream. All commits are part of linux 4.19 or 4.20-rc1 so it will hit disco soon
3. most commits contain one or more reviews. Almost all commits are almost identical to the relevant upstream commit with only minimal changes during the backport so I would consider that the original review still holds

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-09 13:00 EDT-------
Another thing: There is currently this SRU on the list and acked.
[SRU][Bionic][PATCH 0/5] Fixes for LP1799184 [v2]

This will reduce the size of this pull request

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [19.04 FEAT] Guest-dedicated Crypto Adapters

While the target is 18.04 SRU Policy implies this has to be available in all later releases to avoid upgrade regressions.
OTOH we planned libvirt 5.0 and qemu 3.1 which both are released in late December/early January for 19.04 instead of starting now (partially on your request for those versions if you remember).

That would mean I could only SRU (for real) this ~"late January".
Until then I could of course work on it in advance and provide a testing PPA for your verification.
Since the Kernel is currently only available in a PPA as well - and the next Kernel SRU cycle (which is a prerequisite to it anyway) will finalize late December - there is not much to gain by pulling qemu/libvirt forward IMHO.

If that is not early enough I'd have to work on pushing it to 19.04 almost right now which will imply other things have to drop from this cycles plan. That is very unfortunate and needs some acks which I'll start to ask for - but really please consider if late January would just be enough to avoid this major hickup. So would late January be ok for you?

TL;DR:
- this is very costly to do asap
- what is the latest date this can land in 18.04?

Changed in libvirt (Ubuntu):
status: Confirmed → Triaged
Changed in qemu (Ubuntu):
status: Incomplete → Triaged
Revision history for this message
Andrew Cloke (andrew-cloke) wrote :

(Minor update: removed "19.04" from the bug title as it is misleading. 18.04 is the ultimate target.)

summary: - [19.04 FEAT] Guest-dedicated Crypto Adapters
+ [FEAT] Guest-dedicated Crypto Adapters
Changed in linux (Ubuntu Disco):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Cosmic):
status: Triaged → Fix Committed
Revision history for this message
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-cosmic' to 'verification-done-cosmic'. If the problem still exists, change the tag 'verification-needed-cosmic' to 'verification-failed-cosmic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-cosmic
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-15 06:17 EDT-------
Our latest date to have the kernel patches applied to 18.04.1(kernel 4.15) is the December SRU, because we have long lasting customer service durations as well as an ongoing customer PoC.
For testing reason , we do need this made available mid of December - so 18.04.2 is too late.
And the HWE is unfortunately no alternative for us, due to the testing that we intensively do on kernel 4.15, hence kernel upgrades via the HWE kernel would require much more (re-)testing on our side and the alignment to our product release would become to complex to manage.

Please confirm, that the patch will be made available with the next SRU.

The additianally req. patches for qemu can be applied to 2.11 (hence no need to wait for 3.1 availability, this should make things simpler).

This all is required due to our Test start for not later than beginning CW04-2019.

tags: removed: verification-needed-cosmic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks for the not-too conflicting patch series.
I gave it a try and must agree that it seems backportable reasonably (checked only qemu for now).

To me it looks safe in terms of not affecting other use cases, nor other architectures.
It also looks safe for upgrades/migrations not affecting active machine state structs.
I'll give it some testing after I have looked into a libvirt backport as well.
But you could already use the code from the ppa [1] to verify if that is doing what you expected.

If libvirt is as-easy then I think we can shove that in between without the long delays I was afraid of. I could just push it on top of 2.12 in Disco and SRU 2.12/2.11 from there.
If you could help

One question already thou - is there any bug if e.g. any of the libvirt/qemu/kernel code is released without the other? So for example if the qemu update is installed but not the upgraded kernel (or any other combination of the three) would it crash? Or would is just say, sorry not supported on probing or so?
I ask you to think about that so that (if required) dependencies will be bumped accordingly.

Summary:
- please test [1]
- any hard version dependencies needed?
- I'll ping once also libvirt is in that PPA (or if I have any issues backporting it)

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3520

Changed in qemu (Ubuntu):
status: Triaged → In Progress
Changed in libvirt (Ubuntu):
status: Triaged → In Progress
assignee: nobody →  Christian Ehrhardt  (paelzer)
Changed in qemu (Ubuntu):
assignee: nobody →  Christian Ehrhardt  (paelzer)
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-15 07:35 EDT-------
Re 2: there are no hard dependency. If any of the component is on an old level you can of course not use the new feature, but no existing feature should break.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Some noise in libvirt, I'm not entirely sure if VIR_ENUM_IMPL would need all the bumps up to 415 or if inserting it at 282 (next) is safe. I thnik as long as it is the same enum at virQEMUCapsFlags in the header it should be ok right?

Some more missing bits since vfio-ccw isn't available in libvirt 4.0, but it seemed doable as well.
Some more around qemuDomainPrimeVfioDeviceAddresses not yet existing. In general vfio-ccw and some later Mdev code is missing there, I don't think that will work without backporting some more.
The series around 72241444002678f7a8e2f423ff14fcbc27ab0fa5 in particular might be needed - but is that too much?

I'd highly appreciate if Boris (or whoever can afford the time, but was working on this and knows the context) could give that a review. The Debianized and backported changes can be found at [1] - also [3] for qemu, but that was a much safer backport as mentioned before.

Lets see what build and test will say, but maybe here some help to backport it to 4.0 the way you want it might be appreciated - at least in identifying the extended series.
It is now in the same PPA [2] that I referred to before.
[...]
Yeah it failed to build.
I could now just pick above referenced commit on top of what I have, but I think giving you a chance to look at it what you think will be required for a 4.0 backport is probably much more efficient.

[1]: https://git.launchpad.net/~paelzer/libvirt/commit/?id=19e25b48b31f8d717ea466ce2eb9537f5c5b07ea
[2]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3520
[3]: https://git.launchpad.net/~paelzer/ubuntu/+source/qemu/commit/?id=912b6b5ba1c774eb1251f06a3ffb8ba0cf1c1ea2

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-15 08:40 EDT-------
Boris can you have a look and comment on libvirt?

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Changed in linux (Ubuntu Cosmic):
status: Fix Committed → In Progress
Changed in linux (Ubuntu Cosmic):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Bionic):
status: In Progress → Fix Committed
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-16 04:33 EDT-------
(In reply to comment #65)
> FYI: build log of the current incomplete backport:
> https://launchpadlibrarian.net/397706595/buildlog_ubuntu-bionic-s390x.
> libvirt_4.0.0-1ubuntu8.6~ppa1_BUILDING.txt.gz

The build error is due to the changes in the enum virMediatedDeviceModelType
Change the assignment from 2 to 1 for VIR_MDEV_MODEL_TYPE_VFIO_AP.
--- a/src/util/virmdev.h
+++ b/src/util/virmdev.h
@@ -26,6 +26,7 @@
typedef enum {
VIR_MDEV_MODEL_TYPE_VFIO_PCI = 0,
+ VIR_MDEV_MODEL_TYPE_VFIO_AP = 2,
VIR_MDEV_MODEL_TYPE_LAST
} virMediatedDeviceModelType;

There is most likely also trouble ahead regarding the use of the macro virReportEnumRangeError. This needs to be replaced with
virReportError(VIR_ERR_INTERNAL_ERROR,
_("Unexpected enum value %d for "
"virMediatedDeviceModelType"),
mdevsrc->model);

I will try to create a patch series based on v4.0.0

Revision history for this message
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-19 02:56 EDT-------
Question for canonical: What combinations (linux,qemu,libvirt) and from where are we supposed to test?
The bug covers multiple components.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

@cborntra

At the moment, fix committed is the kernel only for bionic & cosmic. Please check and comment if the kernels look sane to be released into -updates.

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-19 07:37 EDT-------
On bionic I tested the kernel from proposed together with the qemu from the ppa 3520. The vfio-ap functionality works. Can somebody else change the state on the launchpad site? The ibm bugzilla mirror does not allow me to do this.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@cborntra: done for you - tags updated - thanks for testing.

On the libvirt side I wait for a series by Boris atm, let me know if the expectations are otherwise.

tags: added: verification-done-bionic
removed: verification-needed-bionic
tags: added: kernel-da-key
removed: kernel-key
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-19 11:04 EDT-------
@paelzer
I am still trying to sort out the vfio-ap required patches for libvirt. I hope to get it done by tomorrow.

tags: added: verification-done-cosmic
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-20 02:26 EDT-------
I forgot to mention that with the patches provided in the tar.gz I was able to successfully run a guest with guest-dedicated crypto adapters.

Revision history for this message
bugproxy (bugproxy) wrote : libvirt patches for vfio-ap support

------- Comment on attachment From <email address hidden> 2018-11-20 02:21 EDT-------

This file contains the changes required for the vfio-ap support in libvirt based on the code level Christian Erhard provided in comment 62.
The list of required backported code needed to be extended and the second patch Christian provided required some changes.

It also turned out that appamor prevents access to the vfio-ap devices located in /dev/vfio/*. I changed (hacked) the patch for LP1775777 to bypass the problem temporally. This certainly needs to be adjusted in the correct way which I am not certain of.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Arr, thanks chrome :-/
Such a nice update lost, well let me rewrite it in a shorter fashion:

1. the patches seem good, thanks for the effort to help backporting
2. the extra changes seem safe to me
3. I added a patch on top to get it working with virt-aa-helper
   That is essentially an extension to [1], while hotplug is already covered by [2] (always
   easier as at this time of the livecycle it can inspect the attributes).
4. Now please try the PPA at [3]

If confirmed working I think the next steps are:
- bring the virt-aa-helper change upstream
- some more testing and regression check
- get all of this into current Ubuntu development release
- plan SRU releases to 18.10/18.04 then

[1]: https://libvirt.org/git/?p=libvirt.git;a=commit;h=74e86b6b2521881808bb93290bcebcb469ab7820
[2]: https://libvirt.org/git/?p=libvirt.git;a=commit;h=606afafba4054d275ffaa4d9afa78c35e2366571
[3]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3520

Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-11-21 03:06 EDT-------
I successfully tested on s390 the provided libvirt packages as requested in point 4 of paelzer last comment.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI: https://www.redhat.com/archives/libvir-list/2018-November/msg00827.html
for the MDEV-vfio apparmor fix upstreaming.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

In preparation I did prepare the series for Disco/Cosmic:

For libvirt 4.6 compared to our series to 4.0:
- drop three being upstream in 4.4 and 4.6
  2b9690b62d01bb0b8555764e2365976b98fe4d47 v4.4.0
  21442874cf61ce61c7e0f8bcd616641f35adda2b v4.4.0
  d54e45b6edd7623e488a19e30bc4148a21fa8b03 v4.6.0
- old lp1787405-0006-conf-Move-VFIO-AP-validation-from-post-parse-to-QEMU.patch backport can now use the upstream versions of 208d6e6f5aafa102d04ce300c6338b0736bb52df and faab373b53e1a4eacf0d6f524eb47df243f21fac instead
- we can now use the upstram patch for f865d58028ccd568b6e7909608678584b12d3c90 as-is
- context updates for debian/patches/ubuntu/lp1787405-0003-qemu-add-vfio-ap-capability.patch
- also updated the Bionic branch as I realized patch 6 had actually two upstream patches as source (only meta data).

For qemu:
- patch debian/patches/ubuntu/lp1787405-0001-linux-headers-update.patch had some minor context updates
- patch ubuntu/lp1787405-0002-s390x-cpumodel-Set-up-CPU-model-for-AP-device-suppor.patch and ubuntu/lp1787405-0004-s390x-ap-base-Adjunct-Processor-AP-object-model.patch can now use the upstream version as-is
- some minor header updates for the Bionic branch

Both branches are built for Disco in the PPA we already used [1].

I'll wait another day for the libvirt upstreaming - there were a few reviews, but no formal ack's yet. I'll ping via IRC if nothing more is happening until tomorrow.

FYI: To add more fun to all the code-porting I just happened to realize that there is also a bunch of CVE fixes incoming (I don't know the content yet). But that might force us to bump these branches once more.

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3520

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Actually Eric beat me, so while I rebased here the patch for lbivirt vfio MDEV for virt-aa-helper was merged.

That said I can finalize the branches for Disco tomorrow and run a round of regression tests before an upload to Disco.

In the meantime I got a few cards to my lpar, maybe I can also verify the feature on my own now.

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-11-22 12:21 EDT-------
@Christian E.:
You listed two libvirt commit IDs
https://libvirt.org/git/?p=libvirt.git;a=commit;h=faab373b53e1a4eacf0d6f524eb47df243f21fac
https://libvirt.org/git/?p=libvirt.git;a=commit;h=f865d58028ccd568b6e7909608678584b12d3c90
that I cannot find in libvirt. Maybe it's just a copy&paste error.

I just looked at patch 6 again and it is correct that I have included code from another commit (most likely d54e45b6edd7623e488a19e30bc4148a21fa8b03) to make the refactoring work and compile without noting it down as origin in the commit message. Sorry about that.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote : Re: [Bug 1787405] Comment bridged from LTC Bugzilla

On Thu, Nov 22, 2018 at 6:35 PM bugproxy <email address hidden> wrote:

> ------- Comment From <email address hidden> 2018-11-22 12:21
> EDT-------
> @Christian E.:
> You listed two libvirt commit IDs
>
> https://libvirt.org/git/?p=libvirt.git;a=commit;h=faab373b53e1a4eacf0d6f524eb47df243f21fac
>
> https://libvirt.org/git/?p=libvirt.git;a=commit;h=f865d58028ccd568b6e7909608678584b12d3c90
> that I cannot find in libvirt. Maybe it's just a copy&paste error.
>

Hmm, yeah that was copy-pasta :-/
The files in my branch are actually good already for completeness here on
the bug the patches that work unmodified on 4.6 are:
https://libvirt.org/git/?p=libvirt.git;a=commit;h=11708641983e9107a129c62fd343d0fec228342f
https://libvirt.org/git/?p=libvirt.git;a=commit;h=208d6e6f5aafa102d04ce300c6338b0736bb52df
https://libvirt.org/git/?p=libvirt.git;a=commit;h=25dde373730545894f60ce5b1497f19d61714c69

I just looked at patch 6 again and it is correct that I have included
> code from another commit (most likely
> d54e45b6edd7623e488a19e30bc4148a21fa8b03) to make the refactoring work
> and compile without noting it down as origin in the commit message.
> Sorry about that.
>

No problem at all.
d54e45b6 is qemuDomainMdevDefValidate which is in patch #5 actually already.
But #6 is is qemuDomainMdevDefVFIOAPValidate from 25dde373 and the
extension for AP in 208d6e6f fused into one.
But that is ok, an SRU wants to only pick what is needed and not rework all
the rest - I just wanted to make sure references are ok.
It is mostly for housekeeping and to make it "traceable" for the upcoming
SRU review.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Integrated the upcoming qemu CVE fixes as well as another SRU fix going on currently into the builds of our PPA. Also I forked a branch for 18.10 for the same changes.

With that I started the cross arch regression check (still ongoing - and will for a while)

Changed in libvirt (Ubuntu Bionic):
status: New → Triaged
Changed in libvirt (Ubuntu Cosmic):
status: New → Triaged
Changed in qemu (Ubuntu Bionic):
status: New → Triaged
Changed in qemu (Ubuntu Cosmic):
status: New → Triaged
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Download full text (3.6 KiB)

Used the Kernel from Proposed:
apt-cache policy linux-image-4.15.0-42-generic
linux-image-4.15.0-42-generic:
  Installed: 4.15.0-42.45
  Candidate: 4.15.0-42.45

Libvirt/Qemu from PPA [1]

Having one device assigned to my LPAR atm:
$ ll /sys/bus/ap/devices/
total 0
drwxr-xr-x 2 root root 0 Nov 23 03:29 ./
drwxr-xr-x 4 root root 0 Nov 23 03:29 ../
lrwxrwxrwx 1 root root 0 Nov 23 03:29 00.0016 -> ../../../devices/ap/card00/00.0016/
lrwxrwxrwx 1 root root 0 Nov 23 03:29 card00 -> ../../../devices/ap/card00/

# mask out the adapters/queues of your choice that you want to virtualize
# In my case i have card 0 queue 16 (hex 16 dec 22 to match HMC config)
$ lszcrypt
CARD.DOMAIN TYPE MODE STATUS REQUEST_CNT
-------------------------------------------------
00 CEX5C CCA-Coproc online 5
00.0016 CEX5C CCA-Coproc online 5
# so lets assign that to vfio-ap instead of zcrypt use

# Adapter
$ cat /sys/bus/ap/apmask
0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
$ echo -0x0 | sudo tee /sys/bus/ap/apmask
$ cat /sys/bus/ap/apmask
0x7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
# Domain
$ cat /sys/bus/ap/aqmask
0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
$ echo -0x16 | sudo tee /sys/bus/ap/aqmask
$ cat /sys/bus/ap/aqmask
0xfffffdffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

$ sudo modprobe vfio_ap
$ dmesg | tail
...
[272006.492864] vfio_ap matrix: MDEV: Registered

# create a new MDEV
$ uuid=$(uuidgen)
$ echo ${uuid} | sudo tee /sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/create

in Dmesg:
[272197.818811] iommu: Adding device 24f952b3-03d1-4df2-9967-0d5f7d63d5f2 to group 0
[272197.818815] vfio_mdev 24f952b3-03d1-4df2-9967-0d5f7d63d5f2: MDEV: group_id = 0

# Assign adapter 0 to vfio-ap
echo +0x0 > /sys/devices/vfio_ap/matrix/${uuid}/assign_adapter
# Assign domain 16 (22) to vfio-ap
$ echo +0x16 | sudo tee /sys/devices/vfio_ap/matrix/${uuid}/assign_domain
$ echo +0x16 | sudo tee /sys/devices/vfio_ap/matrix/${uuid}/assign_control_domain

Check the matrix you have set up
$ cat /sys/devices/vfio_ap/matrix/${uuid}/matrix
00.0016

Get something to bootable to then start it with the MDEV assigned:
$ uvt-kvm create --memory=1024 --password=ubuntu bionic-vfio-ap arch=s390x label=daily release=bionic
# wait until initialized and shut it down
$ virsh shutdown bionic-vfio-ap

# Modify to also use the MDEV
$ virsh edit bionic-vfio-ap
# add a snippet matching your UUID like:
<hostdev mode='subsystem' type='mdev' model='vfio-ap'>
  <source>
    <address uuid='24f952b3-03d1-4df2-9967-0d5f7d63d5f2'/>
  </source>
</hostdev>

When restarting the guest this correctly adds the commandline argument:
  -device vfio-ap,id=hostdev0,sysfsdev=/sys/bus/mdev/devices/24f952b3-03d1-4df2-9967-0d5f7d63d5f2
We also see virt-aa helper generting vfio rules
$ grep '/dev/vfio' /etc/apparmor.d/libvirt/$(virsh dominfo bionic-vfio-ap | awk '/^Security label/ {print $3}').files
  "/dev/vfio/vfio" rw,
  "/dev/vfio/[0-9]*" rw,

And most importantly in the guest the adapter is present:
$ lszcrypt
CARD.DOMAIN TYPE MODE STATUS REQUEST_CNT
----------...

Read more...

description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 4.6.0-2ubuntu4

---------------
libvirt (4.6.0-2ubuntu4) disco; urgency=medium

  * debian/patches/ubuntu/lp1787405-*: Support guest dedicated Crypto
    Adapters on s390x (LP: #1787405)
  * d/p/ubuntu/lp-1802727-netdevbridge-fall-back-to-ioctl-from-sysfs.patch:
    fix libvirt bridge handling in unprivileged containers (LP: #1802906)

 -- Christian Ehrhardt <email address hidden> Fri, 09 Nov 2018 07:42:01 +0100

Changed in libvirt (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package qemu - 1:2.12+dfsg-3ubuntu9

---------------
qemu (1:2.12+dfsg-3ubuntu9) disco; urgency=medium

  [ Marc Deslauriers ]
  * SECURITY UPDATE: integer overflow in NE2000 NIC emulation
    - debian/patches/CVE-2018-10839.patch: use proper type in
      hw/net/ne2000.c.
    - CVE-2018-10839
  * SECURITY UPDATE: integer overflow via crafted QMP command
    - debian/patches/CVE-2018-12617.patch: check bytes count read by
      guest-file-read in qga/commands-posix.c.
    - CVE-2018-12617
  * SECURITY UPDATE: OOB heap buffer r/w access in NVM Express Controller
    - debian/patches/CVE-2018-16847.patch: check size in hw/block/nvme.c.
    - CVE-2018-16847
  * SECURITY UPDATE: buffer overflow in rtl8139
    - debian/patches/CVE-2018-17958.patch: use proper type in
      hw/net/rtl8139.c.
    - CVE-2018-17958
  * SECURITY UPDATE: buffer overflow in pcnet
    - debian/patches/CVE-2018-17962.patch: use proper type in
      hw/net/pcnet.c.
    - CVE-2018-17962
  * SECURITY UPDATE: DoS via large packet sizes
    - debian/patches/CVE-2018-17963.patch: check size in net/net.c.
    - CVE-2018-17963
  * SECURITY UPDATE: DoS in lsi53c895a
    - debian/patches/CVE-2018-18849.patch: check message length value is
      valid in hw/scsi/lsi53c895a.c.
    - CVE-2018-18849
  * SECURITY UPDATE: Out-of-bounds r/w stack access in ppc64
    - debian/patches/CVE-2018-18954.patch: check size before data buffer
      access in hw/ppc/pnv_lpc.c.
    - CVE-2018-18954
  * SECURITY UPDATE: race condition in 9p
    - debian/patches/CVE-2018-19364-1.patch: use write lock in
      hw/9pfs/cofile.c.
    - debian/patches/CVE-2018-19364-2.patch: use write lock in
      hw/9pfs/9p.c.
    - CVE-2018-19364

  [ Christian Ehrhardt]
  * debian/patches/ubuntu/lp1787405-*: Support guest dedicated Crypto
    Adapters on s390x (LP: #1787405)
  * enable opengl for vfio-MDEV support (LP: #1804766)
    - d/control-in: set --enable-opengl
    - d/control-in: add gl related build-dependencies

 -- Christian Ehrhardt <email address hidden> Wed, 21 Nov 2018 13:17:01 -0500

Changed in qemu (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The migration into 19.04 is complete and the CVE fixes completed as well.
I updated all related repositories and the content 100% matches what we have pre-tested.

Just a small delay to make sure the bundled opengl enablement works as well (or to unbundle it).

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I found some more things that I'd want for the opengl changes and decided to unbundle it from this SRU.
That said the qemu and libvirt code for this bug here is now uploaded to bionic-/cosmic-unapproved for review by the SRU Team.

Revision history for this message
Robie Basak (racb) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted qemu into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.12+dfsg-3ubuntu8.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in qemu (Ubuntu Cosmic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-cosmic
removed: verification-done-cosmic
Changed in qemu (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed-bionic
removed: verification-done-bionic
Revision history for this message
Robie Basak (racb) wrote :

Hello bugproxy, or anyone else affected,

Accepted qemu into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.11+dfsg-1ubuntu7.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in libvirt (Ubuntu Cosmic):
status: Triaged → Fix Committed
Revision history for this message
Robie Basak (racb) wrote :

Hello bugproxy, or anyone else affected,

Accepted libvirt into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/4.6.0-2ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in libvirt (Ubuntu Bionic):
status: Triaged → Fix Committed
Revision history for this message
Robie Basak (racb) wrote :

Hello bugproxy, or anyone else affected,

Accepted libvirt into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/4.0.0-1ubuntu8.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Frank Heimes (fheimes)
Changed in ubuntu-z-systems:
status: In Progress → Fix Committed
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@IBM - overall that means, please test from proposed:
libvirt 4.6.0-2ubuntu3.1 + qemu 1:2.12+dfsg-3ubuntu8.2 on 18.10
libvirt 4.0.0-1ubuntu8.6 + qemu 1:2.11+dfsg-1ubuntu7.9 on 18.04

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (39.7 KiB)

This bug was fixed in the package linux - 4.18.0-12.13

---------------
linux (4.18.0-12.13) cosmic; urgency=medium

  * linux: 4.18.0-12.13 -proposed tracker (LP: #1802743)

  * [FEAT] Guest-dedicated Crypto Adapters (LP: #1787405)
    - s390/zcrypt: Add ZAPQ inline function.
    - s390/zcrypt: Review inline assembler constraints.
    - s390/zcrypt: Integrate ap_asm.h into include/asm/ap.h.
    - s390/zcrypt: fix ap_instructions_available() returncodes
    - KVM: s390: vsie: simulate VCPU SIE entry/exit
    - KVM: s390: introduce and use KVM_REQ_VSIE_RESTART
    - KVM: s390: refactor crypto initialization
    - s390: vfio-ap: base implementation of VFIO AP device driver
    - s390: vfio-ap: register matrix device with VFIO mdev framework
    - s390: vfio-ap: sysfs interfaces to configure adapters
    - s390: vfio-ap: sysfs interfaces to configure domains
    - s390: vfio-ap: sysfs interfaces to configure control domains
    - s390: vfio-ap: sysfs interface to view matrix mdev matrix
    - KVM: s390: interface to clear CRYCB masks
    - s390: vfio-ap: implement mediated device open callback
    - s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl
    - s390: vfio-ap: zeroize the AP queues
    - s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl
    - KVM: s390: Clear Crypto Control Block when using vSIE
    - KVM: s390: vsie: Do the CRYCB validation first
    - KVM: s390: vsie: Make use of CRYCB FORMAT2 clear
    - KVM: s390: vsie: Allow CRYCB FORMAT-2
    - KVM: s390: vsie: allow CRYCB FORMAT-1
    - KVM: s390: vsie: allow CRYCB FORMAT-0
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1
    - KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2
    - KVM: s390: device attrs to enable/disable AP interpretation
    - KVM: s390: CPU model support for AP virtualization
    - s390: doc: detailed specifications for AP virtualization
    - KVM: s390: fix locking for crypto setting error path
    - KVM: s390: Tracing APCB changes
    - s390: vfio-ap: setup APCB mask using KVM dedicated function
    - [Config:] Enable CONFIG_S390_AP_IOMMU and set CONFIG_VFIO_AP to module.

  * Bypass of mount visibility through userns + mount propagation (LP: #1789161)
    - mount: Retest MNT_LOCKED in do_umount
    - mount: Don't allow copying MNT_UNBINDABLE|MNT_LOCKED mounts

  * CVE-2018-18955: nested user namespaces with more than five extents
    incorrectly grant privileges over inode (LP: #1801924) // CVE-2018-18955
    - userns: also map extents in the reverse map to kernel IDs

  * kdump fail due to an IRQ storm (LP: #1797990)
    - SAUCE: x86/PCI: Export find_cap() to be used in early PCI code
    - SAUCE: x86/quirks: Add parameter to clear MSIs early on boot
    - SAUCE: x86/quirks: Scan all busses for early PCI quirks

  * crash in ENA driver on removing an interface (LP: #1802341)
    - SAUCE: net: ena: fix crash during ena_remove()

  * Ubuntu 18.04.1 - [s390x] Kernel panic while stressing network bonding
    (LP: #1797367)
    - s390/qeth: reduce hard-coded access to ccw channels
    - s390/qeth: sanitize strings in debug messages

  * Add checksum offload and T...

Changed in linux (Ubuntu Cosmic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (3.1 KiB)

This bug was fixed in the package linux - 4.15.0-42.45

---------------
linux (4.15.0-42.45) bionic; urgency=medium

  * linux: 4.15.0-42.45 -proposed tracker (LP: #1803592)

  * [FEAT] Guest-dedicated Crypto Adapters (LP: #1787405)
    - KVM: s390: reset crypto attributes for all vcpus
    - KVM: s390: vsie: simulate VCPU SIE entry/exit
    - KVM: s390: introduce and use KVM_REQ_VSIE_RESTART
    - KVM: s390: refactor crypto initialization
    - s390: vfio-ap: base implementation of VFIO AP device driver
    - s390: vfio-ap: register matrix device with VFIO mdev framework
    - s390: vfio-ap: sysfs interfaces to configure adapters
    - s390: vfio-ap: sysfs interfaces to configure domains
    - s390: vfio-ap: sysfs interfaces to configure control domains
    - s390: vfio-ap: sysfs interface to view matrix mdev matrix
    - KVM: s390: interface to clear CRYCB masks
    - s390: vfio-ap: implement mediated device open callback
    - s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl
    - s390: vfio-ap: zeroize the AP queues
    - s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl
    - KVM: s390: Clear Crypto Control Block when using vSIE
    - KVM: s390: vsie: Do the CRYCB validation first
    - KVM: s390: vsie: Make use of CRYCB FORMAT2 clear
    - KVM: s390: vsie: Allow CRYCB FORMAT-2
    - KVM: s390: vsie: allow CRYCB FORMAT-1
    - KVM: s390: vsie: allow CRYCB FORMAT-0
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1
    - KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2
    - KVM: s390: device attrs to enable/disable AP interpretation
    - KVM: s390: CPU model support for AP virtualization
    - s390: doc: detailed specifications for AP virtualization
    - KVM: s390: fix locking for crypto setting error path
    - KVM: s390: Tracing APCB changes
    - s390: vfio-ap: setup APCB mask using KVM dedicated function
    - s390/zcrypt: Add ZAPQ inline function.
    - s390/zcrypt: Review inline assembler constraints.
    - s390/zcrypt: Integrate ap_asm.h into include/asm/ap.h.
    - s390/zcrypt: fix ap_instructions_available() returncodes
    - s390/zcrypt: remove VLA usage from the AP bus
    - s390/zcrypt: Remove deprecated ioctls.
    - s390/zcrypt: Remove deprecated zcrypt proc interface.
    - s390/zcrypt: Support up to 256 crypto adapters.
    - [Config:] Enable CONFIG_S390_AP_IOMMU and set CONFIG_VFIO_AP to module.

  * Bypass of mount visibility through userns + mount propagation (LP: #1789161)
    - mount: Retest MNT_LOCKED in do_umount
    - mount: Don't allow copying MNT_UNBINDABLE|MNT_LOCKED mounts

  * CVE-2018-18955: nested user namespaces with more than five extents
    incorrectly grant privileges over inode (LP: #1801924) // CVE-2018-18955
    - userns: also map extents in the reverse map to kernel IDs

  * kdump fail due to an IRQ storm (LP: #1797990)
    - SAUCE: x86/PCI: Export find_cap() to be used in early PCI code
    - SAUCE: x86/quirks: Add parameter to clear MSIs early on boot
    - SAUCE: x86/quirks: Scan all busses for early PCI quirks

 -- Thadeu Lima de Souza Cascardo <email address hidden> Thu, 15 Nov 2018 17:01:46 ...

Read more...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Verified Bionic with the libvirt+qemu setup outlined in comment #66

Guest starting fine and in the guest:
$ lszcrypt
CARD.DOMAIN TYPE MODE STATUS REQUEST_CNT
-------------------------------------------------
00 CEX5C CCA-Coproc online 1
00.0016 CEX5C CCA-Coproc online 1

qemu cmdline that was generated:
  -device vfio-ap,id=hostdev0,sysfsdev=/sys/bus/mdev/devices/24f952b3-03d1-4df2-9967-0d5f7d63d5f2

Cosmic is still a todo (but not today)

@IBM - will you also test and re-verify this or don't you have time/motivation as the PPAs were essentially the same?

tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-12-05 12:34 EDT-------
Confirmed. I installed bionic,
add bionic-proposed
used virt-install to install a guest
shut down guest
and added a hostdev to guest
started guest:
[root@localhost ~]# lszcrypt
CARD.DOMAIN TYPE MODE STATUS REQUEST_CNT
-------------------------------------------------
06 CEX6A Accelerator online 0
06.001a CEX6A Accelerator online 0
08 CEX6C CCA-Coproc online 1
08.001a CEX6C CCA-Coproc online 1
0a CEX6P EP11-Coproc online 0
0a.001a CEX6P EP11-Coproc online 0

Good to go from proposed into updates.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Did cosmic as well now.
First verified that with the non-proposed version it fails (on 1:2.12+dfsg-3ubuntu8.1 / 4.6.0-2ubuntu3)
=> fails as expected (can't even define the XML)

Then upgraded to 1:2.12+dfsg-3ubuntu8.2 and 4.6.0-2ubuntu3.1 from cosmic proposed.

With that qemu works as intended and gets the ap passed through.
But libvirt in 4.6 has gained the (unwelcome) smartness to add display=off which is useful for other mdevs but breaks vfio-ap usage.
That causes this:
error: internal error: qemu unexpectedly closed the monitor: 2018-12-06T07:48:27.407849Z qemu-system-s390x: -device vfio-ap,id=hostdev0,sysfsdev=/sys/bus/mdev/devices/24f952b3-03d1-4df2-9967-0d5f7d63d5f2,display=off: Property '.display' not found

This is still no regression (only the new feature is incomplete on cosmic).
We can either release 4.6.0-2ubuntu3.1 or wait for 4.6.0-2ubuntu3.2 which I start to prep now.
Yet I need to find the right fix first ...

Setting c-verified as well (for qemu and kernel to get their SRU queues flushed at least).

Summarizing:
- kernel verified B&C
- qemu verified B&C
- libvirt verified B
- libvirt will get a follow on fix for C to handle display
- Setting the libvirt task back to in progress

Changed in libvirt (Ubuntu Cosmic):
status: Fix Committed → In Progress
tags: added: verification-done verification-done-cosmic
removed: verification-needed verification-needed-cosmic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Looking back I'm afraid comment #59 was not testing "all" releases when it said "I successfully tested on s390 the provided libvirt packages as requested in point 4 of paelzer last comment".

There really is more than just the latest LTS :-)
Now lets find the patch that makes it stop adding that silly display attribute.

Can even be found with just defining a vfio-ap hostdev as outlined in comment #66 which after a safe&re-edit will have that display attribute in xml.

Changed in libvirt (Ubuntu):
status: Fix Released → In Progress
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Seems to me like:
4.7 d6f97d13 "qemu: mdev: Use vfio-pci 'display' property only with vfio-pci mdevs"

Related:
4.6 d48813e8 conf: Introduce new video type 'none'
4.6 c0ca6dcf qemu: command: Enable formatting vfio-pci.display option onto cmdline
4.6 d54e45b6 conf: Introduce new <hostdev> attribute 'display'
4.6 11c7bdac qemu: caps: Add vfio-pci.display capability

I think the appearance of the 4.6 changes I referred made the 4.7 change above needed.
Lets spin a PPA with just this on top and retest this.

If good we can push it to disco and on top the cosmic SRU (and release it as one update).

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Respins with that fix on top build now for Bionic and Disco in the PPA [1] (same we used so far).
Lets see if they build and work as expected ...

[1]: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3520

Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-12-06 03:54 EDT-------
@paelzer
You picked the correct commit to resolve the display property problem which you encountered.
https://libvirt.org/git/?p=libvirt.git;a=commit;h=d6f97d1338ba9470f7c745fab317d272cde84d38

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Tested 4.6.0-2ubuntu3.2~ppa1 from the PPA.
I can now add

    <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-ap'>
      <source>
        <address uuid='24f952b3-03d1-4df2-9967-0d5f7d63d5f2'/>
      </source>
    </hostdev>

without getting display='off' added

- no new/different related apparmor issues
- starting the guest works fine
- generated qemu line LGTM
  -device vfio-ap,id=hostdev0,sysfsdev=/sys/bus/mdev/devices/24f952b3-03d1-4df2-9967-0d5f7d63d5f2
- guest sees AP adapter
- did a few start/stop cycles to check stability of getting the mdev back (to other guest after
  first shut down for example)

I have the same ready for Disco as 4.6.0-2ubuntu5 and will upload that now.

Once complete the 4.6.0-2ubuntu3.2 shall replace the libvirt currently in cosmic-proposed.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 4.6.0-2ubuntu5

---------------
libvirt (4.6.0-2ubuntu5) disco; urgency=medium

  * d/p/ubuntu/lp1787405-0008-qemu-mdev-Use-vfio-pci-display-property-only
    -with-vf.patch: fix handling of non PCI vfio display propery (part
    of LP: #1787405)

 -- Christian Ehrhardt <email address hidden> Thu, 06 Dec 2018 09:20:39 +0100

Changed in libvirt (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The mini-fox on top for disco was delayed by some openstack/nova/sqlite tests.
I debugged and fixed them together with coreycb, uploading the Cosmic fix on top of the current one in proposed.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Uploaded libvirt_4.6.0-2ubuntu3.2_source.changes @SRU Team please accept that into C-proposed over the one we currently have. Then I can re-confirm it there.

Revision history for this message
Andy Whitcroft (apw) wrote : Please test proposed package

Hello bugproxy, or anyone else affected,

Accepted libvirt into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/4.6.0-2ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in libvirt (Ubuntu Cosmic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-cosmic
removed: verification-done verification-done-cosmic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks for accepting that follow-on fix.
With that it now fully works on cosmic as well following the howto on comment #66

tags: added: verification-done verification-done-cosmic
removed: verification-needed verification-needed-cosmic
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI: We also resolved the systemd test issues in Bionic (unrelated, but needed a fix).
Lets see how they will look like in Cosmic after the tests on the new upload complete.

Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for libvirt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 4.0.0-1ubuntu8.6

---------------
libvirt (4.0.0-1ubuntu8.6) bionic; urgency=medium

  * d/control: explicitly Build-dep on libwiretap-dev to fix FTBFS since
    libwireshark 2.6.x SRU upload (LP: #1801666)
  * debian/patches/ubuntu/lp1787405-*: Support guest dedicated Crypto
    Adapters on s390x (LP: #1787405)

 -- Christian Ehrhardt <email address hidden> Fri, 09 Nov 2018 07:42:01 +0100

Changed in libvirt (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package qemu - 1:2.12+dfsg-3ubuntu8.2

---------------
qemu (1:2.12+dfsg-3ubuntu8.2) cosmic; urgency=medium

  * debian/patches/ubuntu/lp1787405-*: Support guest dedicated Crypto
    Adapters on s390x (LP: #1787405)

 -- Christian Ehrhardt <email address hidden> Fri, 23 Nov 2018 08:39:19 +0100

Changed in qemu (Ubuntu Cosmic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package qemu - 1:2.11+dfsg-1ubuntu7.9

---------------
qemu (1:2.11+dfsg-1ubuntu7.9) bionic; urgency=medium

  * debian/patches/ubuntu/lp1787405-*: Support guest dedicated Crypto
    Adapters on s390x (LP: #1787405)

 -- Christian Ehrhardt <email address hidden> Thu, 15 Nov 2018 12:29:56 +0100

Changed in qemu (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 4.6.0-2ubuntu3.2

---------------
libvirt (4.6.0-2ubuntu3.2) cosmic; urgency=medium

  * d/p/ubuntu/lp1787405-0008-qemu-mdev-Use-vfio-pci-display-property-only
    -with-vf.patch: fix handling of non PCI vfio display propery (part
    of LP: #1787405)

libvirt (4.6.0-2ubuntu3.1) cosmic; urgency=medium

  * debian/patches/ubuntu/lp1787405-*: Support guest dedicated Crypto
    Adapters on s390x (LP: #1787405)

 -- Christian Ehrhardt <email address hidden> Thu, 06 Dec 2018 09:16:13 +0100

Changed in libvirt (Ubuntu Cosmic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (14.1 KiB)

This bug was fixed in the package linux - 4.19.0-12.13

---------------
linux (4.19.0-12.13) disco; urgency=medium

  * linux: 4.19.0-12.13 -proposed tracker (LP: #1813664)

  * kernel oops in bcache module (LP: #1793901)
    - SAUCE: bcache: never writeback a discard operation

  * Disco update: 4.19.18 upstream stable release (LP: #1813611)
    - ipv6: Consider sk_bound_dev_if when binding a socket to a v4 mapped address
    - mlxsw: spectrum: Disable lag port TX before removing it
    - mlxsw: spectrum_switchdev: Set PVID correctly during VLAN deletion
    - net: dsa: mv88x6xxx: mv88e6390 errata
    - net, skbuff: do not prefer skb allocation fails early
    - qmi_wwan: add MTU default to qmap network interface
    - ipv6: Take rcu_read_lock in __inet6_bind for mapped addresses
    - net: clear skb->tstamp in bridge forwarding path
    - netfilter: ipset: Allow matching on destination MAC address for mac and
      ipmac sets
    - gpio: pl061: Move irq_chip definition inside struct pl061
    - drm/amd/display: Guard against null stream_state in set_crc_source
    - drm/amdkfd: fix interrupt spin lock
    - ixgbe: allow IPsec Tx offload in VEPA mode
    - platform/x86: asus-wmi: Tell the EC the OS will handle the display off
      hotkey
    - e1000e: allow non-monotonic SYSTIM readings
    - usb: typec: tcpm: Do not disconnect link for self powered devices
    - selftests/bpf: enable (uncomment) all tests in test_libbpf.sh
    - of: overlay: add missing of_node_put() after add new node to changeset
    - writeback: don't decrement wb->refcnt if !wb->bdi
    - serial: set suppress_bind_attrs flag only if builtin
    - bpf: Allow narrow loads with offset > 0
    - ALSA: oxfw: add support for APOGEE duet FireWire
    - x86/mce: Fix -Wmissing-prototypes warnings
    - MIPS: SiByte: Enable swiotlb for SWARM, LittleSur and BigSur
    - crypto: ecc - regularize scalar for scalar multiplication
    - arm64: perf: set suppress_bind_attrs flag to true
    - drm/atomic-helper: Complete fake_commit->flip_done potentially earlier
    - clk: meson: meson8b: fix incorrect divider mapping in cpu_scale_table
    - samples: bpf: fix: error handling regarding kprobe_events
    - usb: gadget: udc: renesas_usb3: add a safety connection way for
      forced_b_device
    - fpga: altera-cvp: fix probing for multiple FPGAs on the bus
    - selinux: always allow mounting submounts
    - ASoC: pcm3168a: Don't disable pcm3168a when CONFIG_PM defined
    - scsi: qedi: Check for session online before getting iSCSI TLV data.
    - drm/amdgpu: Reorder uvd ring init before uvd resume
    - rxe: IB_WR_REG_MR does not capture MR's iova field
    - efi/libstub: Disable some warnings for x86{,_64}
    - jffs2: Fix use of uninitialized delayed_work, lockdep breakage
    - clk: imx: make mux parent strings const
    - pstore/ram: Do not treat empty buffers as valid
    - media: uvcvideo: Refactor teardown of uvc on USB disconnect
    - powerpc/xmon: Fix invocation inside lock region
    - powerpc/pseries/cpuidle: Fix preempt warning
    - media: firewire: Fix app_info parameter type in avc_ca{,_app}_info
    - ASoC: use dma_ops of parent device for acp_audio_dma
    - media: ve...

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
Changed in ubuntu-z-systems:
status: Fix Committed → Fix Released
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2019-02-05 03:48 EDT-------
IBM Bugzilla status -> closed, Fix Released for all distros

Brad Figg (brad-figg)
tags: added: cscc
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.