[Hyper-V] Additional patches for Lv2 storage performance

[Joshua R. Poulson]

After analysis of the first 4.15 kernel for Lv2 performance, and while we are delayed getting to 4.18, we have identified and backported the following patches for the 4.15 linux-azure kernel:

commit 1268ed0c474a5c8f165ef386f3310521b5e00e27
Author: K. Y. Srinivasan <email address hidden>
Date: Tue Jul 3 16:01:55 2018 -0700
    x86/hyper-v: Fix the circular dependency in IPI enlightenment
linux-next: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=1268ed0c474a5c8f165ef386f3310521b5e00e27

commit 366f03b0cf90ef55f063d4a54cf62b0ac9b6da9d
Author: K. Y. Srinivasan <email address hidden>
Date: Wed May 16 14:53:32 2018 -0700
    X86/Hyper-V: Enhanced IPI enlightenment
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=366f03b0cf90ef55f063d4a54cf62b0ac9b6da9d

commit 68bb7bfb7985df2bd15c2dc975cb68b7a901488a
Author: K. Y. Srinivasan <email address hidden>
Date: Wed May 16 14:53:31 2018 -0700
    X86/Hyper-V: Enable IPI enlightenments
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=68bb7bfb7985df2bd15c2dc975cb68b7a901488a

commit 6b48cb5f8347bc0153ff1d7b075db92e6723ffdb
Author: K. Y. Srinivasan <email address hidden>
Date: Wed May 16 14:53:30 2018 -0700
    X86/Hyper-V: Enlighten APIC access
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=6b48cb5f8347bc0153ff1d7b075db92e6723ffdb

commit 68d1eb72ee99e26576913aa6824f7a703ca06b90
Author: Vitaly Kuznetsov <email address hidden>
Date: Tue Mar 20 15:02:09 2018 +0100
    x86/hyper-v: define struct hv_enlightened_vmcs and clean field bits
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=68d1eb72ee99e26576913aa6824f7a703ca06b90

commit a46d15cc1ae5af905afac2af4cc0c188c2eb59b0
Author: Vitaly Kuznetsov <email address hidden>
Date: Tue Mar 20 15:02:08 2018 +0100
    x86/hyper-v: allocate and use Virtual Processor Assist Pages
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=a46d15cc1ae5af905afac2af4cc0c188c2eb59b0

commit 415bd1cd3a42897f61a92cda0a9f9d7b04c28fb7
Author: Vitaly Kuznetsov <email address hidden>
Date: Tue Mar 20 15:02:06 2018 +0100
    x86/hyper-v: move definitions from TLFS to hyperv-tlfs.h
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=415bd1cd3a42897f61a92cda0a9f9d7b04c28fb7

commit 5a485803221777013944cbd1a7cd5c62efba3ffa
Author: Vitaly Kuznetsov <email address hidden>
Date: Tue Mar 20 15:02:05 2018 +0100
    x86/hyper-v: move hyperv.h out of uapi
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=5a485803221777013944cbd1a7cd5c62efba3ffa

commit e7c4e36c447daca2b7df49024f6bf230871cb155
Author: Vitaly Kuznetsov <email address hidden>
Date: Wed Jan 24 14:23:34 2018 +0100
    x86/hyperv: Redirect reenlightment notifications on CPU offlining
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=e7c4e36c447daca2b7df49024f6bf230871cb155

commit 93286261de1b46339aa27cd4c639b21778f6cade
Author: Vitaly Kuznetsov <email address hidden>
Date: Wed Jan 24 14:23:33 2018 +0100
    x86/hyperv: Reenlightenment notifications support
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=93286261de1b46339aa27cd4c639b21778f6cade

commit e2768eaa1ca4fbb7b778da5615cce3dd310352e6
Author: Vitaly Kuznetsov <email address hidden>
Date: Wed Jan 24 14:23:32 2018 +0100
    x86/hyperv: Add a function to read both TSC and TSC page value simulateneously
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20181126&id=e2768eaa1ca4fbb7b778da5615cce3dd310352e6

commit 4a5f3cde4d51c7afce859aed9d74d197751896d5
Author: Michael Kelley <email address hidden>
Date: Fri Dec 22 11:19:02 2017 -0700
    Drivers: hv: vmbus: Remove x86-isms from arch independent drivers
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/drivers/hv?h=next-20181126&id=4a5f3cde4d51c7afce859aed9d74d197751896d5

From: Dexuan Cui <email address hidden>

We can concurrently try to open the same sub-channel from 2 paths:

path #1: vmbus_onoffer() -> vmbus_process_offer() -> handle_sc_creation().
path #2: storvsc_probe() -> storvsc_connect_to_vsp() ->
  -> storvsc_channel_init() -> handle_multichannel_storage() ->
  -> vmbus_are_subchannels_present() -> handle_sc_creation().

They conflict with each other, but it was not an issue before the recent
commit ae6935ed7d42 ("vmbus: split ring buffer allocation from open"),
because at the beginning of vmbus_open() we checked newchannel->state so
only one path could succeed, and the other would return with -EINVAL.

After ae6935ed7d42, the failing path frees the channel's ringbuffer by
vmbus_free_ring(), and this causes a panic later.

Commit ae6935ed7d42 itself is good, and it just reveals the longstanding
race. We can resolve the issue by removing path #2, i.e. removing the
second vmbus_are_subchannels_present() in handle_multichannel_storage().

BTW, the comment "Check to see if sub-channels have already been created"
in handle_multichannel_storage() is incorrect: when we unload the driver,
we first close the sub-channel(s) and then close the primary channel, next
the host sends rescind-offer message(s) so primary->sc_list will become
empty. This means the first vmbus_are_subchannels_present() in
handle_multichannel_storage() is never useful.

Fixes: ae6935ed7d42 ("vmbus: split ring buffer allocation from open")

[Joshua R. Poulson (jrp)]

Sorry, the name for the last patch on the list will be "scsi: storvsc: Fix a race in sub-channel creation that can cause panic"

[Dexuan Cui (decui)]

The link to "[PATCH] scsi: storvsc: Fix a race in sub-channel creation that can cause panic" is

https://lkml.org/lkml/2018/11/26/159
or
https://lore.kernel.org/patchwork/patch/1016903/

[Joshua R. Poulson (jrp)]

Awesome, thanks Dexuan.

[Joshua R. Poulson (jrp)]

Issue found: on L80v2 the kernel is allocating IRQs outside of NUMA0, but all of the NVME disks on L80v2 are on NUMA0. Looking for a relevant patch for this.

[Joshua R. Poulson (jrp)]

@mhcerri looking at the changelog I don't see "[PATCH] scsi: storvsc: Fix a race in sub-channel creation that can cause panic" in 4.15.0-1034... https://launchpad.net/ubuntu/+source/linux-azure/4.15.0-1034.35

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-azure - 4.15.0-1035.36~14.04.2

---------------
linux-azure (4.15.0-1035.36~14.04.2) trusty; urgency=medium

  * linux-azure: 4.15.0-1035.36~14.04.2 -proposed tracker (LP: #1806063)

  [ Ubuntu: 4.15.0-1035.36 ]

  * linux-azure: 4.15.0-1035.36 -proposed tracker (LP: #1806021)
  * [Hyper-V] Additional patches for Lv2 storage performance (LP: #1805304)
    - SAUCE: scsi: storvsc: Fix a race in sub-channel creation that can cause
      panic

linux-azure (4.15.0-1034.35~14.04.2) trusty; urgency=medium

  * linux-azure: 4.15.0-1034.35~14.04.2 -proposed tracker (LP: #1805474)

  [ Ubuntu: 4.15.0-1034.35 ]

  * linux-azure: 4.15.0-1034.35 -proposed tracker (LP: #1805412)
  * [Hyper-V] Additional patches for Lv2 storage performance (LP: #1805304)
    - SAUCE: Drivers: hv: vmbus: Remove x86-isms from arch independent drivers
    - SAUCE: x86/hyperv: Add a function to read both TSC and TSC page value
      simulateneously
    - SAUCE: x86/hyperv: Reenlightenment notifications support
    - SAUCE: x86/hyperv: Redirect reenlightment notifications on CPU offlining
    - SAUCE: x86/hyper-v: move hyperv.h out of uapi
    - SAUCE: x86/hyper-v: move definitions from TLFS to hyperv-tlfs.h
    - SAUCE: x86/hyper-v: allocate and use Virtual Processor Assist Pages
    - SAUCE: x86/hyper-v: define struct hv_enlightened_vmcs and clean field bits
    - SAUCE: X86/Hyper-V: Enlighten APIC access
    - SAUCE: X86/Hyper-V: Enable IPI enlightenments
    - SAUCE: X86/Hyper-V: Enhanced IPI enlightenment
    - SAUCE: x86/hyper-v: Fix the circular dependency in IPI enlightenment

linux-azure (4.15.0-1033.34~14.04.2) trusty; urgency=medium

  * linux-azure: 4.15.0-1033.34~14.04.2 -proposed tracker (LP: #1802564)

  [ Ubuntu: 4.15.0-1033.34 ]

  * linux-azure: 4.15.0-1033.34 -proposed tracker (LP: #1802559)
  * linux: 4.15.0-42.45 -proposed tracker (LP: #1803592)
  * [FEAT] Guest-dedicated Crypto Adapters (LP: #1787405)
    - KVM: s390: reset crypto attributes for all vcpus
    - KVM: s390: vsie: simulate VCPU SIE entry/exit
    - KVM: s390: introduce and use KVM_REQ_VSIE_RESTART
    - KVM: s390: refactor crypto initialization
    - s390: vfio-ap: base implementation of VFIO AP device driver
    - s390: vfio-ap: register matrix device with VFIO mdev framework
    - s390: vfio-ap: sysfs interfaces to configure adapters
    - s390: vfio-ap: sysfs interfaces to configure domains
    - s390: vfio-ap: sysfs interfaces to configure control domains
    - s390: vfio-ap: sysfs interface to view matrix mdev matrix
    - KVM: s390: interface to clear CRYCB masks
    - s390: vfio-ap: implement mediated device open callback
    - s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl
    - s390: vfio-ap: zeroize the AP queues
    - s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl
    - KVM: s390: Clear Crypto Control Block when using vSIE
    - KVM: s390: vsie: Do the CRYCB validation first
    - KVM: s390: vsie: Make use of CRYCB FORMAT2 clear
    - KVM: s390: vsie: Allow CRYCB FORMAT-2
    - KVM: s390: vsie: allow CRYCB FORMAT-1
    - KVM: s390: vsie: allow CRYCB FORMAT-0
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1
    - KVM: ...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-azure - 4.15.0-1035.36

---------------
linux-azure (4.15.0-1035.36) bionic; urgency=medium

  * linux-azure: 4.15.0-1035.36 -proposed tracker (LP: #1806021)

  * [Hyper-V] Additional patches for Lv2 storage performance (LP: #1805304)
    - SAUCE: scsi: storvsc: Fix a race in sub-channel creation that can cause
      panic

linux-azure (4.15.0-1034.35) bionic; urgency=medium

  * linux-azure: 4.15.0-1034.35 -proposed tracker (LP: #1805412)

  * [Hyper-V] Additional patches for Lv2 storage performance (LP: #1805304)
    - SAUCE: Drivers: hv: vmbus: Remove x86-isms from arch independent drivers
    - SAUCE: x86/hyperv: Add a function to read both TSC and TSC page value
      simulateneously
    - SAUCE: x86/hyperv: Reenlightenment notifications support
    - SAUCE: x86/hyperv: Redirect reenlightment notifications on CPU offlining
    - SAUCE: x86/hyper-v: move hyperv.h out of uapi
    - SAUCE: x86/hyper-v: move definitions from TLFS to hyperv-tlfs.h
    - SAUCE: x86/hyper-v: allocate and use Virtual Processor Assist Pages
    - SAUCE: x86/hyper-v: define struct hv_enlightened_vmcs and clean field bits
    - SAUCE: X86/Hyper-V: Enlighten APIC access
    - SAUCE: X86/Hyper-V: Enable IPI enlightenments
    - SAUCE: X86/Hyper-V: Enhanced IPI enlightenment
    - SAUCE: x86/hyper-v: Fix the circular dependency in IPI enlightenment

linux-azure (4.15.0-1033.34) bionic; urgency=medium

  * linux-azure: 4.15.0-1033.34 -proposed tracker (LP: #1802559)

  [ Ubuntu: 4.15.0-42.45 ]

  * linux: 4.15.0-42.45 -proposed tracker (LP: #1803592)
  * [FEAT] Guest-dedicated Crypto Adapters (LP: #1787405)
    - KVM: s390: reset crypto attributes for all vcpus
    - KVM: s390: vsie: simulate VCPU SIE entry/exit
    - KVM: s390: introduce and use KVM_REQ_VSIE_RESTART
    - KVM: s390: refactor crypto initialization
    - s390: vfio-ap: base implementation of VFIO AP device driver
    - s390: vfio-ap: register matrix device with VFIO mdev framework
    - s390: vfio-ap: sysfs interfaces to configure adapters
    - s390: vfio-ap: sysfs interfaces to configure domains
    - s390: vfio-ap: sysfs interfaces to configure control domains
    - s390: vfio-ap: sysfs interface to view matrix mdev matrix
    - KVM: s390: interface to clear CRYCB masks
    - s390: vfio-ap: implement mediated device open callback
    - s390: vfio-ap: implement VFIO_DEVICE_GET_INFO ioctl
    - s390: vfio-ap: zeroize the AP queues
    - s390: vfio-ap: implement VFIO_DEVICE_RESET ioctl
    - KVM: s390: Clear Crypto Control Block when using vSIE
    - KVM: s390: vsie: Do the CRYCB validation first
    - KVM: s390: vsie: Make use of CRYCB FORMAT2 clear
    - KVM: s390: vsie: Allow CRYCB FORMAT-2
    - KVM: s390: vsie: allow CRYCB FORMAT-1
    - KVM: s390: vsie: allow CRYCB FORMAT-0
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-1
    - KVM: s390: vsie: allow guest FORMAT-1 CRYCB on host FORMAT-2
    - KVM: s390: vsie: allow guest FORMAT-0 CRYCB on host FORMAT-2
    - KVM: s390: device attrs to enable/disable AP interpretation
    - KVM: s390: CPU model support for AP virtualization
    - s390: doc: detailed specifications for AP virtualization
    ...