[SRU] Set the default IO scheduler to CFQ in Kubuntu Trusty

Bug #1378789 reported by Rohan Garg
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
kubuntu-settings (Ubuntu)
Undecided
Unassigned
Trusty
Undecided
Unassigned
Utopic
Undecided
Unassigned

Bug Description

[Impact]
Using deadline causes performance regressions in Baloo. See bug 1310402 for an example.

[Test Case]
* Use Kubuntu trusty until it slows to a crawl
* Upgrade kubuntu-settings
* Reboot
* Make sure boot completes successfully
* Make sure IO scheduler is set to CFQ on boot in case you have a HDD : cat /sys/block/sd*/queue/scheduler
* use it and verify it no longer slow to a crawl

[Test Case for Unity]
 - installing an Ubuntu precise desktop with unity7 (which I believe was the
   last release using cfq by default) and the .0 kernel (3.2.0)
 - using the system for a bit to try to reproduce the original problem -
   application windows becoming grayed out by the WM under normal usage,
   indicating that the app is not responding
 - changing the scheduler to deadline and testing to see if the problem
   persists or resolves itself
 - upgrading to the trusty backport kernel (linux-image-generic-lts-trusty)
 - verifying that the desktop behaves correctly with the default deadline
   scheduler
 - changing the scheduler to cfq and testing whether the original problem
   recurs

[Regression Potential]
None that I can think of, deployment has been successful on Utopic

Rohan Garg (rohangarg)
Changed in kubuntu-settings (Ubuntu Utopic):
status: New → Fix Released
Revision history for this message
Harald Sitter (apachelogger) wrote :

(wearing both my kubuntu and kde dev hat)

Super important bug. It can make IO come to a crawl making desktop systems unusable on rationale media, which in turn makes users unhappy, users complain to upstream, user complain to us, users complain to users. This is not good. It makes people waste time on a perfectly solvable issue, it annoys both upstream and kubuntu developers.
All in all this gives a bad impression and must absolutely be resolved, in particular since 14.04 is a LTS release. Having people's systems come to a crawl every once in a while until the next LTS comes out is absolutely not acceptable.

Revision history for this message
Scott Kitterman (kitterman) wrote : Re: [Bug 1378789] Re: [SRU] Set the default IO scheduler to CFQ in Kubuntu Trusty

How do you guarantee this doesn't cause regressions elsewhere?

Revision history for this message
Rohan Garg (rohangarg) wrote :

How would I prove that? Is there something specific you'd like me to test? We've deployed this in 14.10 for a few months now and haven't seen any issues, before deploying the same change to 14.10 I asked the Ubuntu kernel team if we can switch to CFQ and they gave me the green light saying that it should be just a drop in replacement.

Oh and udev seems to get loaded fairly late in the boot process ( as far as I can understand it gets loaded after the virtual file systems event ), so booting should NOT be affected by this change at all.

Revision history for this message
Harald Sitter (apachelogger) wrote :

system administrators can choose a different scheduler if they so wish.

cfq is the kernel default hence probably better tested by everyone in the world than deadline.

also since a quick web search suggests that deadline only offers better performance in database/server scenarios I doubt what is set in kubuntu-settings will have any reasonable impact on anything other than baloo.

Revision history for this message
Scott Kitterman (kitterman) wrote :

[09:16:35] <ScottK> OK. How about this for a plan:
[09:17:31] <ScottK> Since it's changed in 14.10 and that's about to get large
scale testing, we wait until ~ a month after 14.10 release to see if we get
negative feedback and if not go ahead.
[09:17:47] <ScottK> In the scheme of 5 years of LTS, that time is a detail.
[09:20:40] <ScottK> Riddell: ^^^ does that seem like a reasonable approach?
[09:23:23] <Riddell> ScottK: that seems excessively conservative, we know
upstream linux recommends it, we know baloo recommends it, ubuntu kernel team
don't seem too sure why they switched, we don't have any reason to think it'll
cause problems
[09:24:38] <ScottK> OK. See if another SRU team is willing to move faster or
go to the TB then. I'll add this to the bug.

Revision history for this message
Rohan Garg (rohangarg) wrote :
Revision history for this message
Harald Sitter (apachelogger) wrote :

FTR, I am with Riddell, waiting until after relase seems overly cautious.

Revision history for this message
Rohan Garg (rohangarg) wrote :

Likewise, waiting until after 14.10 seems overly conservative.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in kubuntu-settings (Ubuntu Trusty):
status: New → Confirmed
Revision history for this message
Steve Langasek (vorlon) wrote :

On Fri, Oct 10, 2014 at 11:26:20AM +0100, Jonathan Riddell wrote:
> On Thu, Oct 09, 2014 at 09:02:04AM -0700, Steve Langasek wrote:
> > On Thu, Oct 09, 2014 at 05:39:33PM +0200, Rohan Garg wrote:

> Is there any further information on this? The upstream Baloo author
> would like to know why his work is causing unnecessary slowdowns for
> Kubuntu users and getting his work bad name.

> This is a change that has been done from upstream Linux and Ubuntu is
> the only distro to make such a change so I find it very hard to
> believe that going back to upstream default would cause a problem.
> These changes are in utopic where it solves the problem of slugging
> Kubuntu systems very nicely. So I'd ask the SRU team to consider
> letting this is again without delay.

I am not willing to personally sign off on this SRU, because I don't think
it provably passes the "no regressions" threshold. And I don't think that
the SRU team as a whole should let this into -proposed without the kind of
thorough regression test case that I outlined in my previous mail to
ubuntu-release (which will minimize the regressions, but still doesn't prove
that there aren't any). I also agree with what Scott wrote on the bug
report. This is not a recent behavior change in the Ubuntu kernel; it is
the sort of thing that we should take the time to gather as much data as
possible about, before pushing this change out in an SRU. Gathering data on
top of the 14.10 release is a useful way to do this.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Revision history for this message
Jonathan Riddell (jr) wrote :

Steve also asked for some testing with Ubuntu Unity:

 - installing an Ubuntu precise desktop with unity7 (which I believe was the
   last release using cfq by default) and the .0 kernel (3.2.0)
 - using the system for a bit to try to reproduce the original problem -
   application windows becoming grayed out by the WM under normal usage,
   indicating that the app is not responding
 - changing the scheduler to deadline and testing to see if the problem
   persists or resolves itself
 - upgrading to the trusty backport kernel (linux-image-generic-lts-trusty)
 - verifying that the desktop behaves correctly with the default deadline
   scheduler
 - changing the scheduler to cfq and testing whether the original problem
   recurs

although I feel this should be done by the Ubuntu Unity team, there's no reason why Kubuntu should have to spend resources on the fallout from their bugs

Jonathan Riddell (jr)
description: updated
Revision history for this message
Scott Kitterman (kitterman) wrote : Please test proposed package

Hello Rohan, or anyone else affected,

Accepted into utopic-proposed. The package will build now and be available in a few hours in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed
Revision history for this message
Mathew Hodson (mhodson) wrote :
Changed in kubuntu-settings (Ubuntu Trusty):
status: Confirmed → Fix Committed
Mathew Hodson (mhodson)
tags: added: kubuntu
Revision history for this message
Jonathan Riddell (jr) wrote :

I installed trusty on my machine which has a magnatic hard disk and a solid state hard disk. Looking in /sys/block/sd*/queue/scheduler I can see they are both using deadline.
After installing kubuntu-settings-desktop from trusty-proposed and rebooting I check /sys/block/sd*/queue/scheduler again and the magnetic hard disk is using cfq and the solid state one is still using deadline

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kubuntu-settings - 1:14.04ubuntu15.1

---------------
kubuntu-settings (1:14.04ubuntu15.1) trusty; urgency=medium

  * Add udev rule to switch to cfq on rotational media (LP: #1378789)
 -- Rohan Garg <email address hidden> Wed, 08 Oct 2014 13:51:33 +0200

Changed in kubuntu-settings (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for kubuntu-settings has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Adam Porter (alphapapa) wrote :

Thank you for making this change, finally.

I'd just like to point out that Baloo is far from the only thing that uses ionice. mlocate even uses it by default when it's available. I also use it extensively in backup scripts and other long-running background operations. I even use it for Dropbox (which does a lot of I/O on startup).

Frankly, I consider it a bug that deadline doesn't support ionice. Desktop systems commonly have background I/O operations running nowadays, and ionice is crucial for preventing such operations from harming interactive responsiveness. It's my understanding that deadline was only intended for servers and SSDs in the first place, so perhaps its use in other Ubuntu variants on HDDs should be revisited

Or we could use BFQ on desktops and kiss all these problems goodbye...if only the kernel team would build it... (I've tried applying the patches to a current Ubuntu kernel, but it's a real mess since they aren't kept in sync. If the kernel team would build it with the kernel--not necessarily set it as default, just build it--then people could use it, experiment with it, and gather data about it. Just watch the videos made by the BFQ devs on YouTube--they clearly show that it's worth it!)

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers