fdatasync performance regression on 5.0 kernels

Bug #1847641 reported by Khaled El Mously on 2019-10-10
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Medium
Khaled El Mously
Disco
Medium
Khaled El Mously
Eoan
Medium
Khaled El Mously
linux-gke-5.0 (Ubuntu)
Undecided
Unassigned
Bionic
Undecided
Unassigned
Eoan
Undecided
Unassigned

Bug Description

no longer affects: linux-gcp (Ubuntu)
Changed in linux-gke-5.0 (Ubuntu Bionic):
status: New → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux-gke-5.0 - 5.0.0-1022.22~18.04.3

---------------
linux-gke-5.0 (5.0.0-1022.22~18.04.3) bionic; urgency=medium

  * bionic/linux-gke-5.0: 5.0.0-1022.22~18.04.3 -proposed tracker (LP: #1847644)

  * fdatasync performance regression on 5.0 kernels (LP: #1847641)
    - blk-wbt: fix performance regression in wbt scale_up/scale_down

 -- Khalid Elmously <email address hidden> Thu, 10 Oct 2019 14:25:06 -0400

Changed in linux-gke-5.0 (Ubuntu Bionic):
status: Fix Committed → Fix Released
no longer affects: linux (Ubuntu Bionic)
no longer affects: linux-gke-5.0 (Ubuntu Disco)

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1847641

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Connor Kuehl (connork) on 2019-10-15
Changed in linux (Ubuntu Eoan):
status: Incomplete → In Progress
Changed in linux (Ubuntu Disco):
importance: Undecided → Medium
status: New → In Progress
Changed in linux (Ubuntu Eoan):
importance: Undecided → Medium
Changed in linux (Ubuntu Disco):
assignee: nobody → Khaled El Mously (kmously)
Changed in linux (Ubuntu Eoan):
assignee: nobody → Khaled El Mously (kmously)
Changed in linux-gke-5.0 (Ubuntu Eoan):
status: New → Invalid
Changed in linux (Ubuntu Disco):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Eoan):
status: In Progress → Fix Committed

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-disco

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-eoan' to 'verification-done-eoan'. If the problem still exists, change the tag 'verification-needed-eoan' to 'verification-failed-eoan'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-eoan
Chris Newcomer (cnewcomer) wrote :

Eoan Verification: SUCCESS

Eoan:
I1107 01:13:14.719551 1 iotest.go:34] Starting loop...
I1107 01:13:14.719636 1 iotest.go:38] Generating chunk done. Took: 29.737µs
I1107 01:13:14.719693 1 iotest.go:45] Writing chunk done. Took: 44.206µs
I1107 01:13:14.723320 1 iotest.go:49] Fdatasync done. Took: 3.616042ms
I1107 01:13:15.723595 1 iotest.go:34] Starting loop...
I1107 01:13:15.723664 1 iotest.go:38] Generating chunk done. Took: 26.125µs
I1107 01:13:15.723709 1 iotest.go:45] Writing chunk done. Took: 32.533µs
I1107 01:13:15.726895 1 iotest.go:49] Fdatasync done. Took: 3.177235ms
I1107 01:13:16.727150 1 iotest.go:34] Starting loop...
I1107 01:13:16.727263 1 iotest.go:38] Generating chunk done. Took: 17.628µs
I1107 01:13:16.727331 1 iotest.go:45] Writing chunk done. Took: 37.676µs
I1107 01:13:16.730810 1 iotest.go:49] Fdatasync done. Took: 3.468056ms
I1107 01:13:17.731056 1 iotest.go:34] Starting loop...
I1107 01:13:17.731140 1 iotest.go:38] Generating chunk done. Took: 36.62µs
I1107 01:13:17.731186 1 iotest.go:45] Writing chunk done. Took: 33.776µs
I1107 01:13:17.734490 1 iotest.go:49] Fdatasync done. Took: 3.294855ms
I1107 01:13:18.734697 1 iotest.go:34] Starting loop...
I1107 01:13:18.734782 1 iotest.go:38] Generating chunk done. Took: 28.668µs
I1107 01:13:18.734832 1 iotest.go:45] Writing chunk done. Took: 36.553µs
I1107 01:13:18.737912 1 iotest.go:49] Fdatasync done. Took: 3.070391ms

# uname -a
Linux cnewcomer-1573088148 5.3.0-21-generic #22-Ubuntu SMP Tue Oct 29 22:55:51 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Chris Newcomer (cnewcomer) wrote :

Disco verification: SUCCESS (except for crypt error not seen on Eoan when running update-initramfs)

The package did experience a crypt error while the finish scripts were running update-initramfs:
update-initramfs: Generating /boot/initrd.img-5.0.0-34-generic
cryptsetup: WARNING: The initramfs image may not contain cryptsetup binaries
    nor crypto modules. If that's on purpose, you may want to uninstall the
    'cryptsetup-initramfs' package in order to disable the cryptsetup initramfs
    integration and avoid this warning.

I1107 01:20:47.416048 1 iotest.go:34] Starting loop...
I1107 01:20:47.416130 1 iotest.go:38] Generating chunk done. Took: 35.664µs
I1107 01:20:47.416180 1 iotest.go:45] Writing chunk done. Took: 39.66µs
I1107 01:20:47.418845 1 iotest.go:49] Fdatasync done. Took: 2.655269ms
I1107 01:20:48.419035 1 iotest.go:34] Starting loop...
I1107 01:20:48.419118 1 iotest.go:38] Generating chunk done. Took: 35.264µs
I1107 01:20:48.419179 1 iotest.go:45] Writing chunk done. Took: 48.533µs
I1107 01:20:48.421913 1 iotest.go:49] Fdatasync done. Took: 2.717937ms
I1107 01:20:49.422100 1 iotest.go:34] Starting loop...
I1107 01:20:49.422178 1 iotest.go:38] Generating chunk done. Took: 28.414µs
I1107 01:20:49.422222 1 iotest.go:45] Writing chunk done. Took: 31.93µs
I1107 01:20:49.424807 1 iotest.go:49] Fdatasync done. Took: 2.5748ms
I1107 01:20:50.425013 1 iotest.go:34] Starting loop...
I1107 01:20:50.425100 1 iotest.go:38] Generating chunk done. Took: 26.333µs
I1107 01:20:50.425153 1 iotest.go:45] Writing chunk done. Took: 35.647µs
I1107 01:20:50.429562 1 iotest.go:49] Fdatasync done. Took: 4.399009ms
I1107 01:20:51.429771 1 iotest.go:34] Starting loop...
I1107 01:20:51.429857 1 iotest.go:38] Generating chunk done. Took: 35.83µs
I1107 01:20:51.429900 1 iotest.go:45] Writing chunk done. Took: 30.646µs
I1107 01:20:51.432736 1 iotest.go:49] Fdatasync done. Took: 2.828024ms

# uname -a
Linux cnewcomer-1573089286 5.0.0-34-generic #36-Ubuntu SMP Wed Oct 30 05:16:14 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

tags: added: verification-bionic-disco verification-bionic-eoan
removed: verification-needed-disco verification-needed-eoan
tags: added: verification-needed-disco verification-needed-eoan
removed: verification-bionic-disco verification-bionic-eoan
tags: added: verification-done-disco verification-done-eoan
removed: verification-needed-disco verification-needed-eoan

All autopkgtests for the newly accepted linux-gcp-5.3 (5.3.0-1008.9~18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

linux-gcp-5.3/unknown (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-gcp-5.3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Launchpad Janitor (janitor) wrote :
Download full text (53.1 KiB)

This bug was fixed in the package linux - 5.3.0-22.24

---------------
linux (5.3.0-22.24) eoan; urgency=medium

  * [REGRESSION] md/raid0: cannot assemble multi-zone RAID0 with default_layout
    setting (LP: #1849682)
    - Revert "md/raid0: avoid RAID0 data corruption due to layout confusion."

  * refcount underflow and type confusion in shiftfs (LP: #1850867) // CVE-2019-15793
    - SAUCE: shiftfs: Correct id translation for lower fs operations
    - SAUCE: shiftfs: prevent type confusion
    - SAUCE: shiftfs: Fix refcount underflow in btrfs ioctl handling

  * CVE-2018-12207
    - kvm: x86, powerpc: do not allow clearing largepages debugfs entry
    - SAUCE: KVM: vmx, svm: always run with EFER.NXE=1 when shadow paging is
      active
    - SAUCE: x86: Add ITLB_MULTIHIT bug infrastructure
    - SAUCE: kvm: mmu: ITLB_MULTIHIT mitigation
    - SAUCE: kvm: Add helper function for creating VM worker threads
    - SAUCE: kvm: x86: mmu: Recovery of shattered NX large pages
    - SAUCE: cpu/speculation: Uninline and export CPU mitigations helpers
    - SAUCE: kvm: x86: mmu: Apply global mitigations knob to ITLB_MULTIHIT

  * CVE-2019-11135
    - x86/msr: Add the IA32_TSX_CTRL MSR
    - x86/cpu: Add a helper function x86_read_arch_cap_msr()
    - x86/cpu: Add a "tsx=" cmdline option with TSX disabled by default
    - x86/speculation/taa: Add mitigation for TSX Async Abort
    - x86/speculation/taa: Add sysfs reporting for TSX Async Abort
    - kvm/x86: Export MDS_NO=0 to guests when TSX is enabled
    - x86/tsx: Add "auto" option to the tsx= cmdline parameter
    - x86/speculation/taa: Add documentation for TSX Async Abort
    - x86/tsx: Add config options to set tsx=on|off|auto
    - [Config] Disable TSX by default when possible

  * CVE-2019-0154
    - SAUCE: drm/i915: Lower RM timeout to avoid DSI hard hangs
    - SAUCE: drm/i915/gen8+: Add RC6 CTX corruption WA

  * CVE-2019-0155
    - SAUCE: drm/i915: Rename gen7 cmdparser tables
    - SAUCE: drm/i915: Disable Secure Batches for gen6+
    - SAUCE: drm/i915: Remove Master tables from cmdparser
    - SAUCE: drm/i915: Add support for mandatory cmdparsing
    - SAUCE: drm/i915: Support ro ppgtt mapped cmdparser shadow buffers
    - SAUCE: drm/i915: Allow parsing of unsized batches
    - SAUCE: drm/i915: Add gen9 BCS cmdparsing
    - SAUCE: drm/i915/cmdparser: Use explicit goto for error paths
    - SAUCE: drm/i915/cmdparser: Add support for backward jumps
    - SAUCE: drm/i915/cmdparser: Ignore Length operands during command matching

linux (5.3.0-21.22) eoan; urgency=medium

  * eoan/linux: 5.3.0-21.22 -proposed tracker (LP: #1850486)

  * Fix signing of staging modules in eoan (LP: #1850234)
    - [Packaging] Leave unsigned modules unsigned after adding .gnu_debuglink

linux (5.3.0-20.21) eoan; urgency=medium

  * eoan/linux: 5.3.0-20.21 -proposed tracker (LP: #1849064)

  * eoan: alsa/sof: Enable SOF_HDA link and codec (LP: #1848490)
    - [Config] Enable SOF_HDA link and codec

  * Eoan update: 5.3.7 upstream stable release (LP: #1848750)
    - panic: ensure preemption is disabled during panic()
    - [Config] updateconfigs for USB_RIO500
    - USB: rio500: Remove Rio 500 kernel driver
   ...

Changed in linux (Ubuntu Eoan):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (38.6 KiB)

This bug was fixed in the package linux - 5.0.0-35.38

---------------
linux (5.0.0-35.38) disco; urgency=medium

  * [REGRESSION] md/raid0: cannot assemble multi-zone RAID0 with default_layout
    setting (LP: #1849682)
    - SAUCE: Fix revert "md/raid0: avoid RAID0 data corruption due to layout
      confusion."

  * refcount underflow and type confusion in shiftfs (LP: #1850867) // CVE-2019-15793
    - SAUCE: shiftfs: Correct id translation for lower fs operations
    - SAUCE: shiftfs: prevent type confusion
    - SAUCE: shiftfs: Fix refcount underflow in btrfs ioctl handling

  * CVE-2018-12207
    - kvm: Convert kvm_lock to a mutex
    - kvm: x86: Do not release the page inside mmu_set_spte()
    - KVM: x86: make FNAME(fetch) and __direct_map more similar
    - KVM: x86: remove now unneeded hugepage gfn adjustment
    - KVM: x86: change kvm_mmu_page_get_gfn BUG_ON to WARN_ON
    - KVM: x86: add tracepoints around __direct_map and FNAME(fetch)
    - kvm: x86, powerpc: do not allow clearing largepages debugfs entry
    - SAUCE: KVM: vmx, svm: always run with EFER.NXE=1 when shadow paging is
      active
    - SAUCE: x86: Add ITLB_MULTIHIT bug infrastructure
    - SAUCE: kvm: mmu: ITLB_MULTIHIT mitigation
    - SAUCE: kvm: Add helper function for creating VM worker threads
    - SAUCE: kvm: x86: mmu: Recovery of shattered NX large pages
    - SAUCE: cpu/speculation: Uninline and export CPU mitigations helpers
    - SAUCE: kvm: x86: mmu: Apply global mitigations knob to ITLB_MULTIHIT

  * CVE-2019-11135
    - KVM: x86: use Intel speculation bugs and features as derived in generic x86
      code
    - x86/msr: Add the IA32_TSX_CTRL MSR
    - x86/cpu: Add a helper function x86_read_arch_cap_msr()
    - x86/cpu: Add a "tsx=" cmdline option with TSX disabled by default
    - x86/speculation/taa: Add mitigation for TSX Async Abort
    - x86/speculation/taa: Add sysfs reporting for TSX Async Abort
    - kvm/x86: Export MDS_NO=0 to guests when TSX is enabled
    - x86/tsx: Add "auto" option to the tsx= cmdline parameter
    - x86/speculation/taa: Add documentation for TSX Async Abort
    - x86/tsx: Add config options to set tsx=on|off|auto
    - SAUCE: x86/speculation/taa: Call tsx_init()
    - [Config] Disable TSX by default when possible

  * CVE-2019-0154
    - SAUCE: drm/i915: Lower RM timeout to avoid DSI hard hangs
    - SAUCE: drm/i915/gen8+: Add RC6 CTX corruption WA

  * CVE-2019-0155
    - SAUCE: drm/i915: Rename gen7 cmdparser tables
    - SAUCE: drm/i915: Disable Secure Batches for gen6+
    - SAUCE: drm/i915: Remove Master tables from cmdparser
    - SAUCE: drm/i915: Add support for mandatory cmdparsing
    - SAUCE: drm/i915: Support ro ppgtt mapped cmdparser shadow buffers
    - SAUCE: drm/i915: Allow parsing of unsized batches
    - SAUCE: drm/i915: Add gen9 BCS cmdparsing
    - SAUCE: drm/i915/cmdparser: Use explicit goto for error paths
    - SAUCE: drm/i915/cmdparser: Add support for backward jumps
    - SAUCE: drm/i915/cmdparser: Ignore Length operands during command matching

linux (5.0.0-34.36) disco; urgency=medium

  * disco/linux: <version to be filled> -proposed tracker (LP: #1850574)

  * [REGRESSION] md/raid0: cannot as...

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :
Download full text (33.2 KiB)

This bug was fixed in the package linux - 5.3.0-24.26

---------------
linux (5.3.0-24.26) eoan; urgency=medium

  * eoan/linux: 5.3.0-24.26 -proposed tracker (LP: #1852232)

  * Eoan update: 5.3.9 upstream stable release (LP: #1851550)
    - io_uring: fix up O_NONBLOCK handling for sockets
    - dm snapshot: introduce account_start_copy() and account_end_copy()
    - dm snapshot: rework COW throttling to fix deadlock
    - Btrfs: fix inode cache block reserve leak on failure to allocate data space
    - btrfs: qgroup: Always free PREALLOC META reserve in
      btrfs_delalloc_release_extents()
    - iio: adc: meson_saradc: Fix memory allocation order
    - iio: fix center temperature of bmc150-accel-core
    - libsubcmd: Make _FORTIFY_SOURCE defines dependent on the feature
    - perf tests: Avoid raising SEGV using an obvious NULL dereference
    - perf map: Fix overlapped map handling
    - perf script brstackinsn: Fix recovery from LBR/binary mismatch
    - perf jevents: Fix period for Intel fixed counters
    - perf tools: Propagate get_cpuid() error
    - perf annotate: Propagate perf_env__arch() error
    - perf annotate: Fix the signedness of failure returns
    - perf annotate: Propagate the symbol__annotate() error return
    - perf annotate: Fix arch specific ->init() failure errors
    - perf annotate: Return appropriate error code for allocation failures
    - perf annotate: Don't return -1 for error when doing BPF disassembly
    - staging: rtl8188eu: fix null dereference when kzalloc fails
    - RDMA/siw: Fix serialization issue in write_space()
    - RDMA/hfi1: Prevent memory leak in sdma_init
    - RDMA/iw_cxgb4: fix SRQ access from dump_qp()
    - RDMA/iwcm: Fix a lock inversion issue
    - HID: hyperv: Use in-place iterator API in the channel callback
    - kselftest: exclude failed TARGETS from runlist
    - selftests/kselftest/runner.sh: Add 45 second timeout per test
    - nfs: Fix nfsi->nrequests count error on nfs_inode_remove_request
    - arm64: cpufeature: Effectively expose FRINT capability to userspace
    - arm64: Fix incorrect irqflag restore for priority masking for compat
    - arm64: ftrace: Ensure synchronisation in PLT setup for Neoverse-N1 #1542419
    - tty: serial: owl: Fix the link time qualifier of 'owl_uart_exit()'
    - tty: serial: rda: Fix the link time qualifier of 'rda_uart_exit()'
    - serial/sifive: select SERIAL_EARLYCON
    - tty: n_hdlc: fix build on SPARC
    - misc: fastrpc: prevent memory leak in fastrpc_dma_buf_attach
    - RDMA/core: Fix an error handling path in 'res_get_common_doit()'
    - RDMA/cm: Fix memory leak in cm_add/remove_one
    - RDMA/nldev: Reshuffle the code to avoid need to rebind QP in error path
    - RDMA/mlx5: Do not allow rereg of a ODP MR
    - RDMA/mlx5: Order num_pending_prefetch properly with synchronize_srcu
    - RDMA/mlx5: Add missing synchronize_srcu() for MW cases
    - gpio: max77620: Use correct unit for debounce times
    - fs: cifs: mute -Wunused-const-variable message
    - arm64: vdso32: Fix broken compat vDSO build warnings
    - arm64: vdso32: Detect binutils support for dmb ishld
    - serial: mctrl_gpio: Check for NULL pointer
    - serial: 8250_...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers