[Hyper-V] Rebase Hyper-V to upstream 4.11 kernel plus some cherry-picks from linux-next

Bug #1690177 reported by Joshua R. Poulson on 2017-05-11
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux-azure (Ubuntu)
Undecided
Marcelo Cerri
Xenial
Undecided
Marcelo Cerri

Bug Description

Catching up with upstream 4.11 Hyper-V files:
arch/x86/kernel/cpu/mshyperv.c
arch/x86/include/asm/mshyperv.h
arch/x86/include/uapi/asm/hyperv.h
include/linux/hyperv.h
drivers/hv/channel.c
drivers/hv/channel_mgmt.c
drivers/hv/connection.c
drivers/hv/hv_balloon.c
drivers/hv/hv.c
drivers/hv/hv_fcopy.c
drivers/hv/hv_kvp.c
drivers/hv/hv_snapshot.c
drivers/hv/hv_util.c
drivers/hv/hv_utils_transport.c
drivers/hv/hv_utils_transport.h
drivers/hv/hyperv_vmbus.h
drivers/hv/ring_buffer.c
drivers/hv/vmbus_drv.c
tools/hv/hv_fcopy_daemon.c
tools/hv/hv_get_dhcp_info.sh
tools/hv/hv_get_dns_info.sh
tools/hv/hv_kvp_daemon.c
tools/hv/hv_set_ifconfig.sh
tools/hv/hv_vss_daemon.c
tools/hv/lsvmbus
drivers/input/serio/hyperv-keyboard.c
drivers/net/hyperv/hyperv_net.h
drivers/net/hyperv/netvsc.c
drivers/net/hyperv/netvsc_drv.c
drivers/net/hyperv/rndis_filter.c
drivers/scsi/storvsc_drv.c
drivers/hid/hid-hyperv.c
drivers/pci/host/pci-hyperv.c
drivers/video/fbdev/hyperv_fb.c
drivers/uio/uio_hv_generic.c

This is the delta from 4.11 to 4.10.3:
mshyperv.c : commit 63ed4e0c67df332681ebfef6eca6852da28d6300 : Drivers: hv: vmbus: Consolidate all Hyper-V specific clocksource code
mshyperv.c : commit 8de8af7e0873c4fdac2205327dff922819e16657 : Drivers: hv: vmbus: Move the extracting of Hypervisor version information
mshyperv.c : commit 8730046c1498e8fb8c9a124789893944e8ce8220 : Drivers: hv vmbus: Move Hypercall page setup out of common code
hyperv.h : commit d058fa7e98ff01a4b4750a2210fc19906db3cbe1 : Drivers: hv: vmbus: Move the crash notification function
hyperv.h : commit e4165a0fad0963bf8b4a59f54d3360ccb6a6d1ea : vmbus: constify parameters where possible
hyperv.h : commit f6c4391553573d592212e6624cfba5e2752cd5c8 : vmbus: remove no longer used signal_policy
hyperv.h : commit 3454323c954775098871559b5c23d877c3e23f77 : vmbus: remove unused kickq argument to sendpacket
hyperv.h : commit 6e47dd3e2938f41d75045bbcb64aa9df3a463b2a : vmbus: expose hv_begin/end_read
channel.c : commit 5c1bec61fdfcd056df909a712e2a86bbaeb0f942 : vmbus: use kernel bitops for traversing interrupt mask
channel.c : commit 5e030d5ce9d99a899b648413139ff65bab12b038 : Drivers: hv: vmbus: Don't leak memory when a channel is rescinded
channel_mgmt.c : commit e810e48c0c9a1a1ebb90cfe966bce6dc80ce08e7 : Drivers: hv: vmbus: Move the code to signal end of message
channel_mgmt.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
connection.c : commit 37cdd991fac810a727cd285629d1640fcf53cd19 : vmbus: put related per-cpu variable together
connection.c : commit 8de8af7e0873c4fdac2205327dff922819e16657 : Drivers: hv: vmbus: Move the extracting of Hypervisor version information
connection.c : commit a100d88df1e924e5c9678fabf054d1bae7ab74fb : hv: use correct order when freeing monitor_pages
hv_balloon.c : commit ad6d41253bf91eabb41626683c35a712ba27a20c : Drivers: hv: balloon: add a fall through comment to hv_memory_notifier()
hv.c : commit 6ab42a66d2cc10afefea9f9e5d9a5ad5a836d254 : Drivers: hv: vmbus: Move Hypercall invocation code out of common code
hv.c : commit d5116b4091ecca271c249ede43a49c1245920558 : Drivers: hv: vmbus: Restructure the clockevents code
hv_fcopy.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_fcopy.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_kvp.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_kvp.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_snapshot.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_snapshot.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_util.c : commit 3716a49a81ba19dda7202633a68b28564ba95eb5 : hv_utils: implement Hyper-V PTP source
hv_util.c : commit a1656454131880980bc3a5313c8bf66ef5990c91 : Drivers: hv: vmbus: Use all supported IC versions to negotiate
hv_utils_transport.c : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hv_utils_transport.h : commit e9c18ae6eb2b312f16c63e34b43ea23926daa398 : Drivers: hv: util: move waiting for release to hv_utils_transport itself
hyperv_vmbus.h : commit 37cdd991fac810a727cd285629d1640fcf53cd19 : vmbus: put related per-cpu variable together
hyperv_vmbus.h : commit 63ed4e0c67df332681ebfef6eca6852da28d6300 : Drivers: hv: vmbus: Consolidate all Hyper-V specific clocksource code
hyperv_vmbus.h : commit d6f3609d2b4c6d0eec01f398cb685e50da3e6013 : Drivers: hv: restore hypervcall page cleanup before kexec
hyperv_vmbus.h : commit e810e48c0c9a1a1ebb90cfe966bce6dc80ce08e7 : Drivers: hv: vmbus: Move the code to signal end of message
ring_buffer.c : commit 6e47dd3e2938f41d75045bbcb64aa9df3a463b2a : vmbus: expose hv_begin/end_read
ring_buffer.c : commit b103a56f80878b4370522ec26017fcadd526f5ad : vmbus: drop no longer used kick_q argument
ring_buffer.c : commit e4165a0fad0963bf8b4a59f54d3360ccb6a6d1ea : vmbus: constify parameters where possible
vmbus_drv.c : commit 68db0cf10678630d286f4bbbbdfa102951a35faa : sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task_stack.h>
vmbus_drv.c : commit 37cdd991fac810a727cd285629d1640fcf53cd19 : vmbus: put related per-cpu variable together
vmbus_drv.c : commit 5c1bec61fdfcd056df909a712e2a86bbaeb0f942 : vmbus: use kernel bitops for traversing interrupt mask
vmbus_drv.c : commit d6f3609d2b4c6d0eec01f398cb685e50da3e6013 : Drivers: hv: restore hypervcall page cleanup before kexec
hyperv-keyboard.c : commit f20b161f59b73d40043199b3bff362e656a9c673 : Input: serio - drop unnecessary calls to device_init_wakeup
hyperv_net.h : commit 23312a3be999c22ec70c0fdf9f01cdee05fac986 : netvsc: negotiate checksum and segmentation parameters
hyperv_net.h : commit dc54a08cd3620e6457382c0cd0c8f03513dd749a : netvsc: optimize receive path
hyperv_net.h : commit 793e39555511bccd73308c41205b72448d0077db : netvsc: account for packets/bytes transmitted after completion
hyperv_net.h : commit 962f3fee83a4ef9010ae84dc43ae7aecb572e2a9 : netvsc: add ethtool ops to get/set RSS key
hyperv_net.h : commit 2b01888d1b453096b5b13c0d4b73e630411198b4 : netvsc: allow more flexible setting of number of channels
hyperv_net.h : commit fdfb70d275223b9d69d5d3abe1f88507da579139 : netvsc: fix calculation of available send sections
netvsc.c : commit fdfb70d275223b9d69d5d3abe1f88507da579139 : netvsc: fix calculation of available send sections
netvsc.c : commit 3454323c954775098871559b5c23d877c3e23f77 : vmbus: remove unused kickq argument to sendpacket
netvsc.c : commit 7ce101246655935b014b11d81f815342921f5654 : netvsc: handle select_queue when device is being removed
netvsc.c : commit 2c7f83ca713fa0ac5c4698e4134b09a355f60263 : netvsc: don't pass void * to internal device_add
netvsc_drv.c : commit b8b835a89b2f7a7fe681983dfe5c489cb9ad9500 : netvsc: group all per-channel state together
netvsc_drv.c : commit dc54a08cd3620e6457382c0cd0c8f03513dd749a : netvsc: optimize receive path
netvsc_drv.c : commit 1130383c174499826a3f01486e574e89be17e2d2 : netvsc: call netif_receive_skb
netvsc_drv.c : commit ceaaea0483cf82a2c817781f3cbacf29f2c5cc97 : netvsc: remove unused variables
netvsc_drv.c : commit d0d7b10b05945f40fefd4e60f457c61aefa3e9a9 : net-next: treewide use is_vlan_dev() helper function.
rndis_filter.c : commit 962f3fee83a4ef9010ae84dc43ae7aecb572e2a9 : netvsc: add ethtool ops to get/set RSS key
rndis_filter.c : commit dc54a08cd3620e6457382c0cd0c8f03513dd749a : netvsc: optimize receive path
rndis_filter.c : commit 23312a3be999c22ec70c0fdf9f01cdee05fac986 : netvsc: negotiate checksum and segmentation parameters
rndis_filter.c : commit ceaaea0483cf82a2c817781f3cbacf29f2c5cc97 : netvsc: remove unused variables
rndis_filter.c : commit 46b4f7f5d1f7410de48128540ef2d1aab913a619 : netvsc: eliminate per-device outstanding send counter
rndis_filter.c : commit 2289f0aa706e5160e078f73c32fcbfb56a3ff1e2 : netvsc: simplify rndis_filter_remove
storvsc_drv.c : commit 183b8021fc0a5fadecdf9c0ccac1f48b5c326278 : scripts/spelling.txt: add "intialization" pattern and fix typo instances
storvsc_drv.c : commit f1c635b439a5c01776fe3a25b1e2dc546ea82e6f : scsi: storvsc: Workaround for virtual DVD SCSI version
storvsc_drv.c : commit d86adf482b843b3a58a9ec3b7c1ccdbf7c705db1 : scsi: storvsc: Enable multi-queue support
storvsc_drv.c : commit f64dad2628bdf62eac7ac145a6e31430376b65e4 : scsi: storvsc: Enable tracking of queue depth
storvsc_drv.c : commit b6a05c823fc573a65efc4466f174abf05f922e0f : scsi: remove eh_timed_out methods in the transport template
pci-hyperv.c : commit 4a9b0933bdfcd85da840284bf5a0eb17b654b9c2 : PCI: hv: Use device serial number as PCI domain
uio_hv_generic.c : commit b71e328297a3a578c482fb4814e737a0ec185839 : vmbus: add direct isr callback mode

In addition, the following patches are in linux-next that are needed for larger VM sizes, network performance, and PCI passthrough fixes found in SR-IOV testing:

mshyperv.c : commit a33fd4c27b3ad11c66bdadc5fe6075297ca87a6d : Drivers: hv: Issue explicit EOI when autoeoi is not enabled
hyperv.h : commit 4539673a6ac027e3ba95901379b5f36c5eb3c9dc : Drivers: hv: Fix a typo
hyperv.h : commit 6c248aad81c8903981db9f12fd74189fa01b3789 : Drivers: hv: Base autoeoi enablement based on hypervisor hints
hyperv.h : commit 4827ee1dca5691c9fc568883170a568db94f9b38 : vmbus: expose debug info for drivers
hyperv.h : commit 2a9d7de2038e87bb2a1085ac73c4246c260263f0 : vmbus: cleanup header file style
channel.c : commit bdc1dd47dbcd39f266f0a6c8727a2b1995b36ef2 : vmbus: fix spelling errors
channel_mgmt.c : commit e6242fa0fb50570acb6364446dec369d7b9b5b2c : vmbus: make channel_message table constant
connection.c : commit ada6eb11137be0baf846e00fca40fc096ba85a28 : vmbus: only reschedule tasklet if time limit exceeded
hv_balloon.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hv.c : commit 6c248aad81c8903981db9f12fd74189fa01b3789 : Drivers: hv: Base autoeoi enablement based on hypervisor hints
hv_fcopy.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hv_kvp.c : commit bdc1dd47dbcd39f266f0a6c8727a2b1995b36ef2 : vmbus: fix spelling errors
hv_kvp.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hv_snapshot.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
hyperv_vmbus.h : commit 2a9d7de2038e87bb2a1085ac73c4246c260263f0 : vmbus: cleanup header file style
hyperv_vmbus.h : commit 4827ee1dca5691c9fc568883170a568db94f9b38 : vmbus: expose debug info for drivers
hyperv_vmbus.h : commit b103a56f80878b4370522ec26017fcadd526f5ad : vmbus: drop no longer used kick_q argument
hyperv_vmbus.h : commit e6242fa0fb50570acb6364446dec369d7b9b5b2c : vmbus: make channel_message table constant
ring_buffer.c : commit f3dd3f4797652c311df9c074436d420f1ad3566e : vmbus: introduce in-place packet iterator
ring_buffer.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
ring_buffer.c : commit 4827ee1dca5691c9fc568883170a568db94f9b38 : vmbus: expose debug info for drivers
ring_buffer.c : commit 2c616a8b6bd3b51b560d994f3412da998fe7ee48 : vmbus: remove unnecessary initialization
vmbus_drv.c : commit 8b1f91fb4c1a8a860b8edc0c383821b2ff8a1ece : vmbus: remove useless return's
vmbus_drv.c : commit e6242fa0fb50570acb6364446dec369d7b9b5b2c : vmbus: make channel_message table constant
hyperv_net.h : commit 15a863bf7436124e799ba175a801e25f7b57191e : netvsc: implement NAPI
hyperv_net.h : commit 8db91f6a9b2ff2bb5355ad11c668fe63eb8ae0c3 : hv_netvsc: Fix the queue index computation in forwarding case
hyperv_net.h : commit 545a8e79bd1cc8774877a26275171a2ec8881c9e : netvsc: use RCU to protect inner device structure
hyperv_net.h : commit a0be450e19d397e9ff215e32ed31bc51339b460a : netvsc: uses RCU instead of removal flag
hyperv_net.h : commit 43c7bd1ffcd1621c64cedf1be52156e2f95bba9b : netvsc: use refcount_t for keeping track of sub channels
hyperv_net.h : commit 76f5ed881c8de0264283c5b934deb33bf87209a8 : netvsc: remove unused #define
netvsc.c : commit 545a8e79bd1cc8774877a26275171a2ec8881c9e : netvsc: use RCU to protect inner device structure
netvsc.c : commit 73e64fa4f417b22d8d5521999a631ced8e2d442e : netvsc: Deal with rescinded channels correctly
netvsc.c : commit ebc1dcf6008e562a8f88a6b1f4a4705f4d4c4fdd : netvsc: eliminate unnecessary skb == NULL checks
netvsc.c : commit f4f1c23d6e41f5ee4973a6da65cba1e1c536ec29 : netvsc: fix NAPI performance regression
netvsc_drv.c : commit 545a8e79bd1cc8774877a26275171a2ec8881c9e : netvsc: use RCU to protect inner device structure
netvsc_drv.c : commit e91e7dd71dedbf4508513c8a66212248a853ecbe : netvsc: replace netdev_alloc_skb_ip_align with napi_alloc_skb
netvsc_drv.c : commit a0be450e19d397e9ff215e32ed31bc51339b460a : netvsc: uses RCU instead of removal flag
netvsc_drv.c : commit 5e8456fdd23c23bbfa06e0d647c90758102a4410 : net: hyperv: use new api ethtool_{get|set}_link_ksettings
netvsc_drv.c : commit 8db91f6a9b2ff2bb5355ad11c668fe63eb8ae0c3 : hv_netvsc: Fix the queue index computation in forwarding case
netvsc_drv.c : commit f3c9d40ee12926f330a1dfebce0bebadd1406ba6 : hv_netvsc: change netvsc device default duplex to FULL
netvsc_drv.c : commit 3071ada4916e26a8961c1b99f7766a73b9007bfc : netvsc: change max channel calculation
rndis_filter.c : commit ce12b81061a0a2647ca90c04d131d90edd5c5063 : netvsc: fix and cleanup rndis_filter_set_packet_filter
rndis_filter.c : commit 00ecfb3b34b69dd702dee1bd6de6fc100be384db : netvsc: remove unnecessary lock on shutdown
storvsc_drv.c : commit daf0cd445a218314f9461d67d4f2b9c24cdd534b : scsi: storvsc: Add support for FC rport.
storvsc_drv.c : commit e04085285b829dc922e78768a9abb390a8caed31 : scsi: storvsc: Prefer kcalloc over kzalloc with multiply
storvsc_drv.c : commit 8d4208c1a73ab30e05da142e0f05b2b02fc26fc9 : scsi: storvsc: remove return at end of void function
pci-hyperv.c : commit 24196f0c7d4bba093dfa8074507f31509970319f : PCI: hv: Convert hv_pci_dev.refs from atomic_t to refcount_t
pci-hyperv.c : commit 433fcf6b7b31f1f233dd50aeb9d066a0f6ed4b9d : PCI: hv: Specify CPU_AFFINITY_ALL for MSI affinity when >= 32 CPUs
pci-hyperv.c : commit d3a78d8bf759d8848339dcc367c4c1678b57a08b : PCI: hv: Properly handle PCI bus remove
pci-hyperv.c : commit 59c58ceeea9cdc6144d7b0303753e6bd26d87455 : PCI: hv: Allocate interrupt descriptors with GFP_ATOMIC
pci-hyperv.c : commit 414428c5da1c71986727c2fa5cdf1ed071e398d7 : PCI: hv: Lock PCI bus on device eject

Joshua R. Poulson (jrp) wrote :

Some of these commits have already been applied, but are included for completeness.

Changed in linux (Ubuntu):
status: New → Confirmed
Joshua R. Poulson (jrp) wrote :

Since 4.10 did not apply cleanly to 4.4, I'm requesting these for 4.8, 4.10, and Azure kernel.

tags: added: kernel-da-key kernel-hyper-v
Marcelo Cerri (mhcerri) on 2017-05-18
affects: linux (Ubuntu) → linux-azure (Ubuntu)
Marcelo Cerri (mhcerri) wrote :

I submitted the first version of the backport for review:

https://lists.ubuntu.com/archives/kernel-team/2017-May/084242.html

I also compiled it and I made the deb files available for testing on the following link:

http://kernel.ubuntu.com/~mhcerri/azure/lp1690177-hyperv-backport/

I boot tested it on azure with a "Standard DS1 v2" instance (with 1 core and 3.5 GB of memory). Can you perform some additional tests with it to ensure it's working properly?

Changed in linux-azure (Ubuntu):
status: Confirmed → In Progress
Joshua R. Poulson (jrp) wrote :

First cut doesn't boot on local hyper-v still, but I'll try in Azure soon.

Joshua R. Poulson (jrp) wrote :

I'm definitely seeing issues with PCI interrupts with PCI passthrough, probably related to

pci-hyperv.c : commit 24196f0c7d4bba093dfa8074507f31509970319f : PCI: hv: Convert hv_pci_dev.refs from atomic_t to refcount_t
pci-hyperv.c : commit 59c58ceeea9cdc6144d7b0303753e6bd26d87455 : PCI: hv: Allocate interrupt descriptors with GFP_ATOMIC

I'm okay with leaving those two out but keeping the others. I suspect that since my first request was for a 4.11 base for everything we have a mismatch.

Symptom:
May 19 17:54:04 jrpcudau kernel: [ 87.156879] hv_pci f4d80956-1c02-4ac3-9063-2708944eab59: Request for interrupt failed: 0xc0350005

Marcelo Cerri (mhcerri) wrote :

I removed these two commits and prepared a new kernel for testing:

http://kernel.ubuntu.com/~mhcerri/azure/lp1690177-hyperv-backport_v2+local-hyperv/

Investigating the issue with local hyper-v, that seems to be caused by the missing EFI support. I re-enabled it in this kernel so you probably can test it locally.

I created the bug #1692920 to track the issue with local hyper-v.

Marcelo Cerri (mhcerri) wrote :

Second version of the pull request submitted:

https://lists.ubuntu.com/archives/kernel-team/2017-May/084389.html

Changed in linux-azure (Ubuntu):
assignee: nobody → Marcelo Cerri (mhcerri)
Marcelo Cerri (mhcerri) on 2017-05-31
Changed in linux-azure (Ubuntu):
status: In Progress → Fix Committed
Marcelo Cerri (mhcerri) on 2017-05-31
Changed in linux-azure (Ubuntu Xenial):
status: New → Fix Committed
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-xenial
Marcelo Cerri (mhcerri) on 2017-06-08
Changed in linux-azure (Ubuntu Xenial):
assignee: nobody → Marcelo Cerri (mhcerri)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers