[Hyper-V] Hyper-V Sockets

[Joshua R. Poulson]

Dexuan Cui of Microsoft has submitted upstream the a VM Sockets for Hyper-V feature. The V6 series of this feature set was submitted on January 26th so it seems likely that the discussion is winding down and the feature is ripe for inclusion. We'd like it considered for Ubuntu 16.04.

Here's the latest Part 0 description from lkml:

From: Dexuan Cui <email address hidden> via vger.kernel.org

 Changes since v1:
 - updated "[PATCH 6/7] hvsock: introduce Hyper-V VM Sockets feature"
 - added __init and __exit for the module init/exit functions
 - net/hv_sock/Kconfig: "default m" -> "default m if HYPERV"
 - MODULE_LICENSE: "Dual MIT/GPL" -> "Dual BSD/GPL"

Changes since v2:
 - fixed various coding issue pointed out by David Miller
 - fixed indentation issues
 - removed pr_debug in net/hv_sock/af_hvsock.c
 - used reverse-Chrismas-tree style for local variables.
 - EXPORT_SYMBOL -> EXPORT_SYMBOL_GPL

Changes since v3:
 - fixed a few coding issue pointed by Vitaly Kuznetsov and Dan Carpenter
 - fixed the ret value in vmbus_recvpacket_hvsock on error
 - fixed the style of multi-line comment: vmbus_get_hvsock_rw_status()

Changes since v4 (https://lkml.org/lkml/2015/7/28/404):
 - addressed all the comments about V4.
 - treat the hvsock offers/channels as special VMBus devices
 - add a mechanism to pass hvsock events to the hvsock driver
 - fixed some corner cases with proper locking when a connection is closed
 - rebased to the latest Greg's tree

Changes since v5 (https://lkml.org/lkml/2015/12/24/103):
 - addressed the coding style issues (Vitaly Kuznetsov & David Miller, thanks!)
 - used a better coding for the per-channel rescind callback (Thank Vitaly!)
 - avoided the introduction of new VMBUS driver APIs vmbus_sendpacket_hvsock()
 and vmbus_recvpacket_hvsock() and used vmbus_sendpacket()/vmbus_recvpacket()
 in the higher level (i.e., the vmsock driver). Thank Vitaly!

Hyper-V VM Socket (hv_sock) is a byte-stream based communication mechanism
 between Windowsd 10 (or later) host and a guest. It's kind of TCP over
 VMBus, but the transportation layer (VMBus) is much simpler than IP.
 With Hyper-V VM Sockets, applications between the host and a guest can
 talk with each other directly by the traditional BSD-style socket APIs.

The patchset implements the necessary support in the guest side by adding
 the necessary new APIs in the vmbus driver, and introducing a new driver
 hv_sock.ko, which implements_a new socket address family AF_HYPERV.

I know the kernel has already had a VM Sockets driver (AF_VSOCK) based
 on VMware's VMCI (net/vmw_vsock/, drivers/misc/vmw_vmci), and KVM is
 proposing AF_VSOCK of virtio version:
http://thread.gmane.org/gmane.linux.network/365205.

However, though Hyper-V VM Sockets may seem conceptually similar to
 AF_VOSCK, there are differences in the transportation layer, and IMO these
 make the direct code reusing impractical:

1. In AF_VSOCK, the endpoint type is: <u32 ContextID, u32 Port>, but in
 AF_HYPERV, the endpoint type is: <GUID VM_ID, GUID ServiceID>. Here GUID
 is 128-bit.

2. AF_VSOCK supports SOCK_DGRAM, while AF_HYPERV doesn't.

3. AF_VSOCK supports some special sock opts, like SO_VM_SOCKETS_BUFFER_SIZE,
 SO_VM_SOCKETS_BUFFER_MIN/MAX_SIZE and SO_VM_SOCKETS_CONNECT_TIMEOUT.
 These are meaningless to AF_HYPERV.

4. Some AF_VSOCK's VMCI transportation ops are meanless to AF_HYPERV/VMBus,
 like .notify_recv_init
         .notify_recv_pre_block
         .notify_recv_pre_dequeue
         .notify_recv_post_dequeue
         .notify_send_init
         .notify_send_pre_block
         .notify_send_pre_enqueue
         .notify_send_post_enqueue
 etc.

So I think we'd better introduce a new address family: AF_HYPERV.

Please review the patchset.

Looking forward to your comments!

Dexuan Cui (9):
   Drivers: hv: vmbus: add a helper function to set a channel's pending
     send size
   Drivers: hv: vmbus: define the new offer type for Hyper-V socket
     (hvsock)
   Drivers: hv: vmbus: define a new VMBus message type for hvsock
   Drivers: hv: ring_buffer: enhance hv_ringbuffer_read() to support
     hvsock
   Drivers: hv: vmbus: add APIs to send/recv hvsock packets
   Drivers: hv: vmbus: add a hvsock flag in struct hv_driver
   Drivers: hv: vmbus: add a mechanism to pass hvsock events to the
     hvsock driver
   Drivers: hv: vmbus: add an API vmbus_hvsock_device_unregister()
   hvsock: introduce Hyper-V VM Sockets feature

 MAINTAINERS | 2 +
  drivers/hv/channel.c | 84 ++-
  drivers/hv/channel_mgmt.c | 53 +-
  drivers/hv/connection.c | 4 +-
  drivers/hv/hyperv_vmbus.h | 13 +-
  drivers/hv/ring_buffer.c | 54 +-
  drivers/hv/vmbus_drv.c | 4 +
  include/linux/hyperv.h | 88 +++
  include/linux/socket.h | 4 +-
  include/net/af_hvsock.h | 44 ++
  include/uapi/linux/hyperv.h | 16 +
  net/Kconfig | 1 +
  net/Makefile | 1 +
  net/hv_sock/Kconfig | 10 +
  net/hv_sock/Makefile | 3 +
  net/hv_sock/af_hvsock.c | 1473 +++++++++++++++++++++++++++++++++++++++++++
  16 files changed, 1830 insertions(+), 24 deletions(-)
  create mode 100644 include/net/af_hvsock.h
  create mode 100644 net/hv_sock/Kconfig
  create mode 100644 net/hv_sock/Makefile
  create mode 100644 net/hv_sock/af_hvsock.c

--
 2.1.4

Dexuan Cui (8):
   Drivers: hv: vmbus: add a helper function to set a channel's pending
     send size
   Drivers: hv: vmbus: define the new offer type for Hyper-V socket
     (hvsock)
   Drivers: hv: vmbus: vmbus_sendpacket_ctl: hvsock: avoid unnecessary
     signaling
   Drivers: hv: vmbus: define a new VMBus message type for hvsock
   Drivers: hv: vmbus: add a hvsock flag in struct hv_driver
   Drivers: hv: vmbus: add a per-channel rescind callback
   Drivers: hv: vmbus: add an API vmbus_hvsock_device_unregister()
   hvsock: introduce Hyper-V Socket feature

 MAINTAINERS | 2 +
  drivers/hv/channel.c | 21 +-
  drivers/hv/channel_mgmt.c | 46 +-
  drivers/hv/connection.c | 4 +-
  drivers/hv/vmbus_drv.c | 4 +
  include/linux/hyperv.h | 67 ++
  include/linux/socket.h | 4 +-
  include/net/af_hvsock.h | 51 ++
  include/uapi/linux/hyperv.h | 16 +
  net/Kconfig | 1 +
  net/Makefile | 1 +
  net/hv_sock/Kconfig | 10 +
  net/hv_sock/Makefile | 3 +
  net/hv_sock/af_hvsock.c | 1480 +++++++++++++++++++++++++++++++++++++++++++
  14 files changed, 1702 insertions(+), 8 deletions(-)
  create mode 100644 include/net/af_hvsock.h
  create mode 100644 net/hv_sock/Kconfig
  create mode 100644 net/hv_sock/Makefile
  create mode 100644 net/hv_sock/af_hvsock.c

There are, as yet, no pieces of this upstream submitted committed to Linus's tree.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1541585

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Joshua R. Poulson (jrp)]

This bug is a feature request to pull a submission to upstream Linux-next. No logs required.

[Tim Gardner (timg-tpi)]

Joshua - these patches would be easier to review if you had them hosted in a public git repository, preferably on top of Xenial.

[Joshua R. Poulson (jrp)]

While we do carry a git repository at https://github.com/LIS/lis-next I admit it's not based on Xenial.

[Tim Gardner (timg-tpi)]

Joshua - this isn't very digestible.

[Tim Gardner (timg-tpi)]

Joshua - I cherry-picked these commits from 4.6-rc1:

Drivers: hv: vmbus: Cleanup vmbus_set_event()
Drivers: hv: vmbus: Add vendor and device atttributes
Drivers: hv: vmbus: avoid infinite loop in init_vp_index()
Drivers: hv: vmbus: avoid scheduling in interrupt context in vmbus_initiate_unload()
Drivers: hv: vmbus: don't manipulate with clocksources on crash
Drivers: hv: vmbus: add a helper function to set a channel's pending send size
Drivers: hv: vmbus: define the new offer type for Hyper-V socket (hvsock)
Drivers: hv: vmbus: vmbus_sendpacket_ctl: hvsock: avoid unnecessary signaling
Drivers: hv: vmbus: define a new VMBus message type for hvsock
Drivers: hv: vmbus: add a hvsock flag in struct hv_driver
Drivers: hv: vmbus: add a per-channel rescind callback
Drivers: hv: vmbus: add an API vmbus_hvsock_device_unregister()
Drivers: hv: vmbus: Eliminate the spin lock on the read path
Drivers: hv: vmbus: Give control over how the ring access is serialized
drivers/hv: Move VMBus hypercall codes into Hyper-V UAPI header
Drivers: hv: vmbus: don't loose HVMSG_TIMER_EXPIRED messages
Drivers: hv: vmbus: avoid wait_for_completion() on crash
Drivers: hv: vmbus: remove code duplication in message handling
Drivers: hv: vmbus: avoid unneeded compiler optimizations in vmbus_wait_for_unload()
Drivers: hv: util: Pass the channel information during the init call
Drivers: hv: utils: Remove util transport handler from list if registration fails
Revert "Drivers: hv: vmbus: Support handling messages on multiple CPUs"
Drivers: hv: vmbus: Support handling messages on multiple CPUs

Most of them look like good bug fixes. However, they don't implement the hv_sock feature (which I don't see even in linux-next).

[Joshua R. Poulson (jrp)]

Last week David Miller came back with some comments on the upstream submission. Dexuan is going to resubmit soon.

The original 8-part series is represented in your pull with the following:
[1/8] Drivers: hv: vmbus: add a helper function to set a channel's pending send size
[2/8] Drivers: hv: vmbus: define the new offer type for Hyper-V socket (hvsock)
[3/8] Drivers: hv: vmbus: vmbus_sendpacket_ctl: hvsock: avoid unnecessary signaling
[4/8] Drivers: hv: vmbus: define a new VMBus message type for hvsock
[5/8] Drivers: hv: vmbus: add a hvsock flag in struct hv_driver
[6/8] Drivers: hv: vmbus: add a per-channel rescind callback
[7/8] Drivers: hv: vmbus: add an API vmbus_hvsock_device_unregister()

What I don't see in the above is

[8/8] hvsock: introduce Hyper-V Socket feature

I don't see a commit in Linus's tree for that, yet. I think that's because Olaf, had some feedback, and Dexuan submitted a new series:

[1/3] net: add the AF_KCM entries to family name tables *needs a fix*
[2/3] hv_sock: introduce Hyper-V Sockets
[3/3] net: add the AF_HYPERV entries to family name tables

It's the comments on the first item that Dexuan will resubmit for.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.4.0-17.33

---------------
linux (4.4.0-17.33) xenial; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #1563441

  * ISST-LTE: pVM:high cpus number need a high crashkernel value in kdump
    (LP: #1560552)
    - SAUCE: (noup) ppc64 boot: Wait for boot cpu to show up if nr_cpus limit is
      about to hit.

  * Predictable naming mechanism is leading to issues in DLPAR operations of
    NICs (LP: #1560514)
    - SAUCE: (noup) powerpc/pci: Assign fixed PHB number based on device-tree
      properties

  * ThunderX: support alternative phy implementations (LP: #1562968)
    - net: thunderx: Cleanup PHY probing code.
    - [Config] CONFIG_MDIO_CAVIUM=m
    - phy: mdio-octeon: Refactor into two files/modules
    - [Config] CONFIG_MDIO_THUNDER=m
    - phy: mdio-thunder: Add driver for Cavium Thunder SoC MDIO buses.
    - phy: mdio-cavium: Add missing MODULE_* annotations.
    - net: cavium: For Kconfig THUNDER_NIC_BGX, select MDIO_THUNDER.
    - phy: mdio-thunder: Fix some Kconfig typos
    - [d-i] Add phy drivers for Cavium ThunderX to nic-modules udeb

  * linux: exclude ZONE_DEVICE from GFP_ZONE_TABLE (LP: #1563293)
    - Revert "mm: CONFIG_NR_ZONES_EXTENDED"
    - mm: exclude ZONE_DEVICE from GFP_ZONE_TABLE

  * lots of printk to serial console can hang system for long time
    (LP: #1534216)
    - printk: set may_schedule for some of console_trylock() callers

  * [i915_bpo] Update i915 backport driver (LP: #1560395)
    - SAUCE: i915_bpo: Update to drm-intel-next-fixes-2016-03-16
    - PM / runtime: Add new helper for conditional usage count incrementation
    - drm/core: Add drm_for_each_encoder_mask, v2.
    - drm/atomic-helper: Implement subsystem-level suspend/resume

  * [Hyper-V] VM Sockets (LP: #1541585)
    - Drivers: hv: vmbus: Cleanup vmbus_set_event()
    - Drivers: hv: vmbus: Add vendor and device atttributes
    - Drivers: hv: vmbus: avoid infinite loop in init_vp_index()
    - Drivers: hv: vmbus: avoid scheduling in interrupt context in vmbus_initiate_unload()
    - Drivers: hv: vmbus: don't manipulate with clocksources on crash
    - Drivers: hv: vmbus: add a helper function to set a channel's pending send size
    - Drivers: hv: vmbus: define the new offer type for Hyper-V socket (hvsock)
    - Drivers: hv: vmbus: vmbus_sendpacket_ctl: hvsock: avoid unnecessary signaling
    - Drivers: hv: vmbus: define a new VMBus message type for hvsock
    - Drivers: hv: vmbus: add a hvsock flag in struct hv_driver
    - Drivers: hv: vmbus: add a per-channel rescind callback
    - Drivers: hv: vmbus: add an API vmbus_hvsock_device_unregister()
    - Drivers: hv: vmbus: Eliminate the spin lock on the read path
    - Drivers: hv: vmbus: Give control over how the ring access is serialized
    - drivers/hv: Move VMBus hypercall codes into Hyper-V UAPI header
    - Drivers: hv: vmbus: don't loose HVMSG_TIMER_EXPIRED messages
    - Drivers: hv: vmbus: avoid wait_for_completion() on crash
    - Drivers: hv: vmbus: remove code duplication in message handling
    - Drivers: hv: vmbus: avoid unneeded compiler optimizations in vmbus_wait_for_unload()
    - Drivers: hv: util: Pass the chann...