nvme: improve performance for virtual Google NVMe devices

[Tim Gardner]

This change provides a mechanism to reduce the number of MMIO doorbell writes for the NVMe driver. When running in a virtualized environment like QEMU, the cost of an MMIO is quite hefy here. The main idea for the patch is provide the device two memory location locations:
     1) to store the doorbell values so they can be lookup without the doorbell
        MMIO write
     2) to store an event index.
I believe the doorbell value is obvious, the event index not so much. Similar to the virtio specificaiton, the virtual device can tell the driver (guest OS) not to write MMIO unless you are writing past this value.

FYI: doorbell values are written by the nvme driver (guest OS) and the event index is written by the virtual device (host OS).

The patch implements a new admin command that will communicate where these two memory locations reside. If the command fails, the nvme driver will work as before without any optimizations.

[Tim Gardner (timg-tpi)]

Yakkety patch submitted - https://lists.ubuntu.com/archives/kernel-team/2016-November/080751.html
Test image results were, "That image looks good, IOPs goes up, up, up, and nothing blows up.".

[Tim Gardner (timg-tpi)]

Xenial test kernel at http://people.canonical.com/~rtg/nvme-google/

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-yakkety' to 'verification-done-yakkety'. If the problem still exists, change the tag 'verification-needed-yakkety' to 'verification-failed-yakkety'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Tim Gardner (timg-tpi)]

An out of band email response indicated that Yakkety works as expected. See comment #1

[Luis Henriques (henrix)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.4.0-57.78

---------------
linux (4.4.0-57.78) xenial; urgency=low

  * Release Tracking Bug
    - LP: #1648867

  * Miscellaneous Ubuntu changes
    - SAUCE: Do not build the xr-usb-serial driver for s390

linux (4.4.0-56.77) xenial; urgency=low

  * Release Tracking Bug
    - LP: #1648867

  * Release Tracking Bug
    - LP: #1648579

  * CONFIG_NR_CPUS=256 is too low (LP: #1579205)
    - [Config] Increase the NR_CPUS to 512 for amd64 to support systems with a
      large number of cores.

  * NVMe drives in Amazon AWS instance fail to initialize (LP: #1648449)
    - SAUCE: (no-up) NVMe: only setup MSIX once

linux (4.4.0-55.76) xenial; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1648503

  * NVMe driver accidentally reverted to use GSI instead of MSIX (LP: #1647887)
    - (fix) NVMe: restore code to always use MSI/MSI-x interrupts

linux (4.4.0-54.75) xenial; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1648017

  * Update hio driver to 2.1.0.28 (LP: #1646643)
    - SAUCE: hio: update to Huawei ES3000_V2 (2.1.0.28)

  * linux: Enable live patching for all supported architectures (LP: #1633577)
    - [Config] CONFIG_LIVEPATCH=y for s390x

  * Botched backport breaks level triggered EOIs in QEMU guests with --machine
    kernel_irqchip=split (LP: #1644394)
    - kvm/irqchip: kvm_arch_irq_routing_update renaming split

  * Xenial update to v4.4.35 stable release (LP: #1645453)
    - x86/cpu/AMD: Fix cpu_llc_id for AMD Fam17h systems
    - KVM: x86: fix missed SRCU usage in kvm_lapic_set_vapic_addr
    - KVM: Disable irq while unregistering user notifier
    - fuse: fix fuse_write_end() if zero bytes were copied
    - mfd: intel-lpss: Do not put device in reset state on suspend
    - can: bcm: fix warning in bcm_connect/proc_register
    - i2c: mux: fix up dependencies
    - kbuild: add -fno-PIE
    - scripts/has-stack-protector: add -fno-PIE
    - x86/kexec: add -fno-PIE
    - kbuild: Steal gcc's pie from the very beginning
    - ext4: sanity check the block and cluster size at mount time
    - crypto: caam - do not register AES-XTS mode on LP units
    - drm/amdgpu: Attach exclusive fence to prime exported bo's. (v5)
    - clk: mmp: pxa910: fix return value check in pxa910_clk_init()
    - clk: mmp: pxa168: fix return value check in pxa168_clk_init()
    - clk: mmp: mmp2: fix return value check in mmp2_clk_init()
    - rtc: omap: Fix selecting external osc
    - iwlwifi: pcie: fix SPLC structure parsing
    - mfd: core: Fix device reference leak in mfd_clone_cell
    - uwb: fix device reference leaks
    - PM / sleep: fix device reference leak in test_suspend
    - PM / sleep: don't suspend parent when async child suspend_{noirq, late}
      fails
    - IB/mlx4: Check gid_index return value
    - IB/mlx4: Fix create CQ error flow
    - IB/mlx5: Use cache line size to select CQE stride
    - IB/mlx5: Fix fatal error dispatching
    - IB/core: Avoid unsigned int overflow in sg_alloc_table
    - IB/uverbs: Fix leak of XRC target QPs
    - IB/cm: Mark stale CM id's whenever the mad agent was unregistered
    - netfilter: nft_dynset: fix element timeou...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.8.0-32.34

---------------
linux (4.8.0-32.34) yakkety; urgency=low

  [ Thadeu Lima de Souza Cascardo ]

  * Release Tracking Bug
    - LP: #1649358

  * Vulnerability picked up from 4.8.10 stable kernel (LP: #1648662)
    - net: handle no dst on skb in icmp6_send

linux (4.8.0-31.33) yakkety; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1648034

  * Update hio driver to 2.1.0.28 (LP: #1646643)
    - SAUCE: hio: update to Huawei ES3000_V2 (2.1.0.28)

  * Yakkety update to v4.8.11 stable release (LP: #1645421)
    - x86/cpu/AMD: Fix cpu_llc_id for AMD Fam17h systems
    - KVM: x86: fix missed SRCU usage in kvm_lapic_set_vapic_addr
    - KVM: Disable irq while unregistering user notifier
    - arm64: KVM: pmu: Fix AArch32 cycle counter access
    - KVM: arm64: Fix the issues when guest PMCCFILTR is configured
    - ftrace: Ignore FTRACE_FL_DISABLED while walking dyn_ftrace records
    - ftrace: Add more checks for FTRACE_FL_DISABLED in processing ip records
    - genirq: Use irq type from irqdata instead of irqdesc
    - fuse: fix fuse_write_end() if zero bytes were copied
    - IB/rdmavt: rdmavt can handle non aligned page maps
    - IB/hfi1: Fix rnr_timer addition
    - mfd: intel-lpss: Do not put device in reset state on suspend
    - mfd: stmpe: Fix RESET regression on STMPE2401
    - can: bcm: fix warning in bcm_connect/proc_register
    - gpio: do not double-check direction on sleeping chips
    - ALSA: usb-audio: Fix use-after-free of usb_device at disconnect
    - ALSA: hda - add a new condition to check if it is thinkpad
    - ALSA: hda - Fix mic regression by ASRock mobo fixup
    - i2c: mux: fix up dependencies
    - i2c: i2c-mux-pca954x: fix deselect enabling for device-tree
    - Disable the __builtin_return_address() warning globally after all
    - kbuild: add -fno-PIE
    - scripts/has-stack-protector: add -fno-PIE
    - x86/kexec: add -fno-PIE
    - kbuild: Steal gcc's pie from the very beginning
    - ext4: sanity check the block and cluster size at mount time
    - ARM: dts: imx53-qsb: Fix regulator constraints
    - crypto: caam - do not register AES-XTS mode on LP units
    - powerpc/64: Fix setting of AIL in hypervisor mode
    - drm/amdgpu: Attach exclusive fence to prime exported bo's. (v5)
    - drm/i915: Refresh that status of MST capable connectors in ->detect()
    - drm/i915: Assume non-DP++ port if dvo_port is HDMI and there's no AUX ch
      specified in the VBT
    - virtio-net: drop legacy features in virtio 1 mode
    - clk: mmp: pxa910: fix return value check in pxa910_clk_init()
    - clk: mmp: pxa168: fix return value check in pxa168_clk_init()
    - clk: mmp: mmp2: fix return value check in mmp2_clk_init()
    - clk: imx: fix integer overflow in AV PLL round rate
    - rtc: omap: Fix selecting external osc
    - iwlwifi: pcie: fix SPLC structure parsing
    - iwlwifi: pcie: mark command queue lock with separate lockdep class
    - iwlwifi: mvm: fix netdetect starting/stopping for unified images
    - iwlwifi: mvm: fix d3_test with unified D0/D3 images
    - iwlwifi: mvm: wake the wait queue when the RX sync counter is zero
    - mfd: cor...