[linux-azure] Commit To Improve NVMe Performance

[Joseph Salisbury]

In bug 1818138, the config option CONFIG_NO_HZ_FULL was requested to be enabled in the linux-azure kernels(currently 4.15 and 4.18) in order to increase NVME disks performance.

To achieve the peak IOPs we are seeing in testing, the following commit is also needed:
7ac257b862f2c (“blk-mq: remove the request_list usage”)

This commit cleans up unused code in block-mq (mq uses pre-allocated tags to allocate request, not request_list as it's used only in legacy queue). The commit saves a bunch of rcu locks on I/O queuing path, which increases the achievable IOPs seen in testing.

[Marcelo Cerri (mhcerri)]

Hi, Joe.

Can you confirm the patch is also necessary for 4.15?

I backported the commit to xenial/linux-azure and it just required a bit of fixing due to context differences. I also built a test kernel if you want to test it:

https://kernel.ubuntu.com/~mhcerri/azure/xenial-linux-azure-4.15.0-1042.46+lp1819689/

[Marcelo Cerri (mhcerri)]

Hi, Joe. I'm including the patch for 4.15. Let me know if you find any issues on the test kernel.

[Marcelo Cerri (mhcerri)]

https://lists.ubuntu.com/archives/kernel-team/2019-April/100272.html

[Joseph Salisbury (jsalisbury)]

It was found that 4.15 has a couple more patches missing for NVMe, those patches are required to get to the target number of IOPS.

The commits are:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/nvme/host/pci.c?h=v5.1&id=f9dde187fa921c12a8680089a77595b866e65455

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/nvme/host/pci.c?h=v5.1&id=1ab0cd6966fc4a7e9dfbd7c6eda917ae9c977f42

Would it be possible to pick up those two patches for 4.15. The 1st patch is a clean apply. The 2nd patch needs back porting. We can help, if it is needed for the second patch.

[Joseph Salisbury (jsalisbury)]

If it would be best, I can open a new bug for those two additional commits.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-azure - 4.18.0-1018.18

---------------
linux-azure (4.18.0-1018.18) cosmic; urgency=medium

  [ Ubuntu: 4.18.0-20.21 ]

  * CVE-2018-12126 // CVE-2018-12127 // CVE-2018-12130
    - Documentation/l1tf: Fix small spelling typo
    - x86/cpu: Sanitize FAM6_ATOM naming
    - kvm: x86: Report STIBP on GET_SUPPORTED_CPUID
    - locking/atomics, asm-generic: Move some macros from <linux/bitops.h> to a
      new <linux/bits.h> file
    - tools include: Adopt linux/bits.h
    - x86/msr-index: Cleanup bit defines
    - x86/speculation: Consolidate CPU whitelists
    - x86/speculation/mds: Add basic bug infrastructure for MDS
    - x86/speculation/mds: Add BUG_MSBDS_ONLY
    - x86/kvm: Expose X86_FEATURE_MD_CLEAR to guests
    - x86/speculation/mds: Add mds_clear_cpu_buffers()
    - x86/speculation/mds: Clear CPU buffers on exit to user
    - x86/kvm/vmx: Add MDS protection when L1D Flush is not active
    - x86/speculation/mds: Conditionally clear CPU buffers on idle entry
    - x86/speculation/mds: Add mitigation control for MDS
    - x86/speculation/mds: Add sysfs reporting for MDS
    - x86/speculation/mds: Add mitigation mode VMWERV
    - Documentation: Move L1TF to separate directory
    - Documentation: Add MDS vulnerability documentation
    - x86/speculation/mds: Add mds=full,nosmt cmdline option
    - x86/speculation: Move arch_smt_update() call to after mitigation decisions
    - x86/speculation/mds: Add SMT warning message
    - x86/speculation/mds: Fix comment
    - x86/speculation/mds: Print SMT vulnerable on MSBDS with mitigations off
    - x86/speculation/mds: Add 'mitigations=' support for MDS
  * CVE-2017-5715 // CVE-2017-5753
    - s390/speculation: Support 'mitigations=' cmdline option
  * CVE-2017-5715 // CVE-2017-5753 // CVE-2017-5754 // CVE-2018-3639
    - powerpc/speculation: Support 'mitigations=' cmdline option
  * CVE-2017-5715 // CVE-2017-5754 // CVE-2018-3620 // CVE-2018-3639 //
    CVE-2018-3646
    - cpu/speculation: Add 'mitigations=' cmdline option
    - x86/speculation: Support 'mitigations=' cmdline option
  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

linux-azure (4.18.0-1017.17) cosmic; urgency=medium

  * linux-azure: 4.18.0-1017.17 -proposed tracker (LP: #1826166)

  * [linux-azure] Include mainline commits fc96df16a1ce and ba50bf1ce9a5 in
    Azure kernel (LP: #1821378)
    - Drivers: hv: vmbus: Return -EINVAL for the sys files for unopened channels
    - Drivers: hv: vmbus: Check for ring when getting debug info

  * [linux-azure] Commit To Improve NVMe Performance (LP: #1819689)
    - blk-mq: remove the request_list usage

  [ Ubuntu: 4.18.0-19.20 ]

  * linux: 4.18.0-19.20 -proposed tracker (LP: #1826171)
  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log
  * autopkgtests run too often, too much and don't skip enough (LP: #1823056)
    - [Debian] Set +x on rebuild testcase.
    - [Debian] Skip rebuild test, for regression-suite deps.
    - [Debian] Make ubuntu-regression-suite skippable on unbootable kernels.
    - [Debian] make rebuild use skippable error codes when skipping.
    - [Debian] Only run regression-suite...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-azure - 4.15.0-1045.49

---------------
linux-azure (4.15.0-1045.49) xenial; urgency=medium

  * [linux-azure] Storage performance drop on RAID (LP: #1828248)
    - Revert "blk-mq: remove the request_list usage"

  [ Ubuntu: 4.15.0-50.54 ]

  * CVE-2018-12126 // CVE-2018-12127 // CVE-2018-12130
    - Documentation/l1tf: Fix small spelling typo
    - x86/cpu: Sanitize FAM6_ATOM naming
    - kvm: x86: Report STIBP on GET_SUPPORTED_CPUID
    - locking/atomics, asm-generic: Move some macros from <linux/bitops.h> to a
      new <linux/bits.h> file
    - tools include: Adopt linux/bits.h
    - x86/msr-index: Cleanup bit defines
    - x86/speculation: Consolidate CPU whitelists
    - x86/speculation/mds: Add basic bug infrastructure for MDS
    - x86/speculation/mds: Add BUG_MSBDS_ONLY
    - x86/kvm: Expose X86_FEATURE_MD_CLEAR to guests
    - x86/speculation/mds: Add mds_clear_cpu_buffers()
    - x86/speculation/mds: Clear CPU buffers on exit to user
    - x86/kvm/vmx: Add MDS protection when L1D Flush is not active
    - x86/speculation/mds: Conditionally clear CPU buffers on idle entry
    - x86/speculation/mds: Add mitigation control for MDS
    - x86/speculation/mds: Add sysfs reporting for MDS
    - x86/speculation/mds: Add mitigation mode VMWERV
    - Documentation: Move L1TF to separate directory
    - Documentation: Add MDS vulnerability documentation
    - x86/speculation/mds: Add mds=full,nosmt cmdline option
    - x86/speculation: Move arch_smt_update() call to after mitigation decisions
    - x86/speculation/mds: Add SMT warning message
    - x86/speculation/mds: Fix comment
    - x86/speculation/mds: Print SMT vulnerable on MSBDS with mitigations off
    - x86/speculation/mds: Add 'mitigations=' support for MDS
  * CVE-2017-5715 // CVE-2017-5753
    - s390/speculation: Support 'mitigations=' cmdline option
  * CVE-2017-5715 // CVE-2017-5753 // CVE-2017-5754 // CVE-2018-3639
    - powerpc/speculation: Support 'mitigations=' cmdline option
  * CVE-2017-5715 // CVE-2017-5754 // CVE-2018-3620 // CVE-2018-3639 //
    CVE-2018-3646
    - cpu/speculation: Add 'mitigations=' cmdline option
    - x86/speculation: Support 'mitigations=' cmdline option
  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

linux-azure (4.15.0-1044.48) xenial; urgency=medium

  * linux-azure: 4.15.0-1044.48 -proposed tracker (LP: #1826354)

  * [linux-azure] Include mainline commits fc96df16a1ce and ba50bf1ce9a5 in
    Azure kernel (LP: #1821378)
    - Drivers: hv: vmbus: Return -EINVAL for the sys files for unopened channels
    - Drivers: hv: vmbus: Check for ring when getting debug info

  * [linux-azure] Commit To Improve NVMe Performance (LP: #1819689)
    - blk-mq: remove the request_list usage

  [ Ubuntu: 4.15.0-49.53 ]

  * linux: 4.15.0-49.53 -proposed tracker (LP: #1826358)
  * Backport support for software count cache flush Spectre v2 mitigation. (CVE)
    (required for POWER9 DD2.3) (LP: #1822870)
    - powerpc/64s: Add support for ori barrier_nospec patching
    - powerpc/64s: Patch barrier_nospec in modules
    - powerpc/64s: Enable barrier_nospec based on firmware settings
    - powerpc...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-azure - 4.15.0-1045.49~14.04.1

---------------
linux-azure (4.15.0-1045.49~14.04.1) trusty; urgency=medium

  [ Ubuntu: 4.15.0-1045.49 ]

  * [linux-azure] Storage performance drop on RAID (LP: #1828248)
    - Revert "blk-mq: remove the request_list usage"
  * CVE-2018-12126 // CVE-2018-12127 // CVE-2018-12130
    - Documentation/l1tf: Fix small spelling typo
    - x86/cpu: Sanitize FAM6_ATOM naming
    - kvm: x86: Report STIBP on GET_SUPPORTED_CPUID
    - locking/atomics, asm-generic: Move some macros from <linux/bitops.h> to a
      new <linux/bits.h> file
    - tools include: Adopt linux/bits.h
    - x86/msr-index: Cleanup bit defines
    - x86/speculation: Consolidate CPU whitelists
    - x86/speculation/mds: Add basic bug infrastructure for MDS
    - x86/speculation/mds: Add BUG_MSBDS_ONLY
    - x86/kvm: Expose X86_FEATURE_MD_CLEAR to guests
    - x86/speculation/mds: Add mds_clear_cpu_buffers()
    - x86/speculation/mds: Clear CPU buffers on exit to user
    - x86/kvm/vmx: Add MDS protection when L1D Flush is not active
    - x86/speculation/mds: Conditionally clear CPU buffers on idle entry
    - x86/speculation/mds: Add mitigation control for MDS
    - x86/speculation/mds: Add sysfs reporting for MDS
    - x86/speculation/mds: Add mitigation mode VMWERV
    - Documentation: Move L1TF to separate directory
    - Documentation: Add MDS vulnerability documentation
    - x86/speculation/mds: Add mds=full,nosmt cmdline option
    - x86/speculation: Move arch_smt_update() call to after mitigation decisions
    - x86/speculation/mds: Add SMT warning message
    - x86/speculation/mds: Fix comment
    - x86/speculation/mds: Print SMT vulnerable on MSBDS with mitigations off
    - x86/speculation/mds: Add 'mitigations=' support for MDS
  * CVE-2017-5715 // CVE-2017-5753
    - s390/speculation: Support 'mitigations=' cmdline option
  * CVE-2017-5715 // CVE-2017-5753 // CVE-2017-5754 // CVE-2018-3639
    - powerpc/speculation: Support 'mitigations=' cmdline option
  * CVE-2017-5715 // CVE-2017-5754 // CVE-2018-3620 // CVE-2018-3639 //
    CVE-2018-3646
    - cpu/speculation: Add 'mitigations=' cmdline option
    - x86/speculation: Support 'mitigations=' cmdline option
  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log

linux-azure (4.15.0-1044.48~14.04.1) trusty; urgency=medium

  * linux-azure: 4.15.0-1044.48~14.04.1 -proposed tracker (LP: #1826352)

  * Packaging resync (LP: #1786013)
    - [Packaging] resync git-ubuntu-log
    - [Packaging] update helper scripts

  [ Ubuntu: 4.15.0-1044.48 ]

  * linux-azure: 4.15.0-1044.48 -proposed tracker (LP: #1826354)
  * [linux-azure] Include mainline commits fc96df16a1ce and ba50bf1ce9a5 in
    Azure kernel (LP: #1821378)
    - Drivers: hv: vmbus: Return -EINVAL for the sys files for unopened channels
    - Drivers: hv: vmbus: Check for ring when getting debug info
  * [linux-azure] Commit To Improve NVMe Performance (LP: #1819689)
    - blk-mq: remove the request_list usage
  * linux: 4.15.0-49.53 -proposed tracker (LP: #1826358)
  * Backport support for software count cache flush Spectre v2 mitigation. (CVE)
    (required for POWER9 DD...

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for linux-azure has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.