USB 3.0 performance drop between 3.2.0-21-generic and 3.2.0-22-generic

[Bzzz]

I noticed a significant worse performance of my USB 3.0 Expresscard when using 3.2.0-22-generic compared to the older 3.2.0-21-generic.

Controller is a
05:00.0 USB controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)

+ a Corsair Flash Voyager GT 3.0 32GB

I tried it four times each, and results were the same every time. Shutting down or just rebooting the machine to the other kernel version does not make any difference - it clearly has something to to with the kernel itself. Boot options where the same on both cases, I tried it with the USB storage attached at boot and also connecting it later.

Please note the attached screens of the benchmark feature of Disk Utility (palimpsest).

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: alsa-base 1.0.25+dfsg-0ubuntu1
ProcVersionSignature: Ubuntu 3.2.0-21.34-generic 3.2.13
Uname: Linux 3.2.0-21-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 2.0-0ubuntu4
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: bzzz 3493 F.... pulseaudio
Date: Sun Apr 8 04:34:35 2012
InstallationMedia: Kubuntu 11.04 "Natty Narwhal" - Release amd64 (20110426.3)
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_GB
 TERM=xterm
 PATH=(custom, user)
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: alsa-driver
Symptom: audio
Symptom_AlsaPlaybackTest: ALSA playback test through plughw:Intel successful
Symptom_Card: Built-in Audio - HDA Intel
Symptom_DevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: bzzz 3493 F.... pulseaudio
Symptom_Jack: Green Headphone Out, Right
Symptom_PulsePlaybackTest: PulseAudio playback test successful
Symptom_Type: No sound at all
Title: [6475VAT, Conexant CX20561 (Hermosa), Green Headphone Out, Right] No sound at all
UpgradeStatus: Upgraded to precise on 2012-02-04 (63 days ago)
dmi.bios.date: 08/02/2010
dmi.bios.vendor: LENOVO
dmi.bios.version: 7UET89WW (3.19 )
dmi.board.name: 6475VAT
dmi.board.vendor: LENOVO
dmi.board.version: Not Available
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvr7UET89WW(3.19):bd08/02/2010:svnLENOVO:pn6475VAT:pvrThinkPadT400:rvnLENOVO:rn6475VAT:rvrNotAvailable:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 6475VAT
dmi.product.version: ThinkPad T400
dmi.sys.vendor: LENOVO
modified.conffile..etc.modprobe.d.alsa.base.conf: [modified]
mtime.conffile..etc.modprobe.d.alsa.base.conf: 2012-02-06T00:36:12.817222
---
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 2.0-0ubuntu4
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: bzzz 3479 F.... pulseaudio
DistroRelease: Ubuntu 12.04
HibernationDevice: RESUME=UUID=ab2e99a3-850d-480f-99db-b0711b005ed2
InstallationMedia: Kubuntu 11.04 "Natty Narwhal" - Release amd64 (20110426.3)
MachineType: LENOVO 6475VAT
Package: linux 3.2.0.22.24
PackageArchitecture: amd64
ProcEnviron:
 LANGUAGE=en_GB
 TERM=xterm
 PATH=(custom, user)
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.2.0-22-generic root=/dev/mapper/lvmtit-titroot ro crashkernel=384M-2G:64M,2G-:128M pciehp_force=1 pciehp_poll_mode=1 pcie_aspm=force elevator=deadline
ProcVersionSignature: Ubuntu 3.2.0-22.35-generic 3.2.14
StagingDrivers: mei
Tags: precise staging
Uname: Linux 3.2.0-22-generic x86_64
UpgradeStatus: Upgraded to precise on 2012-02-04 (63 days ago)
UserGroups: adm admin audio cdrom debian-tor dialout lpadmin plugdev pulse sambashare
dmi.bios.date: 08/02/2010
dmi.bios.vendor: LENOVO
dmi.bios.version: 7UET89WW (3.19 )
dmi.board.name: 6475VAT
dmi.board.vendor: LENOVO
dmi.board.version: Not Available
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvr7UET89WW(3.19):bd08/02/2010:svnLENOVO:pn6475VAT:pvrThinkPadT400:rvnLENOVO:rn6475VAT:rvrNotAvailable:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 6475VAT
dmi.product.version: ThinkPad T400
dmi.sys.vendor: LENOVO

[Bzzz (da-bzzz)]

Really? alsa-driver for USB performance?...

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 976325

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[Bzzz (da-bzzz)]

apport information

[penalvch (penalvch)]

Bzzz, thank you for reporting this bug and helping make Ubuntu better. If you could also please test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results.

Thanks in advance.

[Bzzz (da-bzzz)]

3.2.14-030214-generic_3.2.14-030214.201204021356_amd64 wasn't able to boot

[Bzzz (da-bzzz)]

I also had a run on a different machine with Oneiric + 3.0.0-17-generic kernel, which was also capped at ~70 MB/s read and 15 MB/s write.

btw, any idea how to gain constant write rates? Using Windows, 40 MB/s average can be reached. Ubuntu is doing something stupid when it comes to the write characteristics of flash memory.

[penalvch (penalvch)]

Bzzz, thank you for testing the mainline kernel. The next step is to perform a kernel bisect in order to identify the offending commit. Could you please do so following https://wiki.ubuntu.com/Kernel/KernelBisection ?

[Bzzz (da-bzzz)]

Well, right now I'm bisecting that thing...haven't found a bad one yet, only positive results.

Current position is between sha1 cba898a327da48eb1f7dd6b86ff174e76dd883a4 and the Ubuntu-3.2.0-22.35 release, which means 16 revisions left.

As each build takes about an hour plus rebooting, testing and fixing occasional errors, I don't know if I can finish in time. My only USB 3.0 stick is already sold and has to be shipped tomorrow. However, the only clearly USB related commit left (sha1 c56daacff345b5d1f3b4fa7f0a65a886517c1cc3) would show up in the next kernel build. We'll see...

[Bzzz (da-bzzz)]

I have no idea why faulty commit +1 is reported, but I guess someone will figure that out....I thought it was the USB one that had bad influence.

For the range c56daacff345b5d1f3b4fa7f0a65a886517c1cc3cba898a327da48eb1f7dd6b86ff174e76dd883a4 I didn't do make clean every time, but I hope that had no side effects. Any other bisects were performed with previous clean.

3a50863f6706ece7719a68be0ae57957164a0f0c is the first bad commit
commit 3a50863f6706ece7719a68be0ae57957164a0f0c
Author: Brad Figg <email address hidden>
Date: Fri Mar 30 09:24:43 2012 -0700

    sched: Fix nohz load accounting

    BugLink: http://bugs.launchpad.net/bugs/838811

    Various people reported nohz load tracking still being wrecked, but Doug
    spotted the actual problem. We fold the nohz remainder in too soon,
    causing us to loose samples and under-account.

    So instead of playing catch-up up-front, always do a single load-fold
    with whatever state we encounter and only then fold the nohz remainder
    and play catch-up.

    Reported-by: Doug Smythies <email address hidden>
    Reported-by: LesÅ=82aw Kope=C4=87 <email address hidden>
    Reported-by: Aman Gupta <email address hidden>
    Signed-off-by: Peter Zijlstra <email address hidden>
    Link:
    http://<email address hidden>
    Signed-off-by: Ingo Molnar <email address hidden>

    (backport of upstream: c308b56b5398779cd3da0f62ab26b0453494c3d4)

    Signed-off-by: Brad Figg <email address hidden>
    Signed-off-by: Tim Gardner <email address hidden>

git bisect log
# bad: [f91db971b3244b74adc45a6f096adc2078d9db40] UBUNTU: Ubuntu-3.2.0-22.35
# good: [77056febf1221c992212af1297f4412ff93c9faf] UBUNTU: Ubuntu-3.2.0-21.34
git bisect start 'Ubuntu-3.2.0-22.35' 'Ubuntu-3.2.0-21.34'
# good: [02905906ddc049720ded93ded03bbc2652f4646a] Linux 3.2.13
git bisect good 02905906ddc049720ded93ded03bbc2652f4646a
# good: [6b89ccc6a088f78bea08d4e84bcd86aebd31bfae] UBUNTU: Start new release
git bisect good 6b89ccc6a088f78bea08d4e84bcd86aebd31bfae
# good: [787054f061a570dd75cb92547fda683dc1b0686f] net/hyperv: rx_bytes should account the ether header size
git bisect good 787054f061a570dd75cb92547fda683dc1b0686f
# good: [e2edc36fb78668700d7b6084253da9f455f255b6] KVM: expose latest Intel cpu new features (BMI1/BMI2/FMA/AVX2) to guest
git bisect good e2edc36fb78668700d7b6084253da9f455f255b6
# good: [da9ae770cee7fd7ef72d817415ae0af7f0082c79] toshiba_acpi: Refuse to load on machines with buggy INFO implementations
git bisect good da9ae770cee7fd7ef72d817415ae0af7f0082c79
# good: [cba898a327da48eb1f7dd6b86ff174e76dd883a4] ethtool: Null-terminate filename passed to ethtool_ops::flash_device
git bisect good cba898a327da48eb1f7dd6b86ff174e76dd883a4
# bad: [9cbb750707996baba71a3fa8fc214488b4bdabb4] tools: hv: Use hyperv.h to get the KVP definitions
git bisect bad 9cbb750707996baba71a3fa8fc214488b4bdabb4
# bad: [c56daacff345b5d1f3b4fa7f0a65a886517c1cc3] UBUNTU: [Config] Enable CONFIG_USBIP_CORE=m
git bisect bad c56daacff345b5d1f3b4fa7f0a65a886517c1cc3

(tried to shorten things up, but that didn't work...had to clear one as good to get to the previous commit i...

[penalvch (penalvch)]

Bzzz, thank you for bisecting the kernel and identifying the offending commit. Could you please test the newest mainline kernel and report the results http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.5-rc1-quantal/ ?

[Bzzz (da-bzzz)]

Unfortunately not...the stick is already shipped, and as my laptop got some replacement parts from IBM/Lenovo service (including a new HDD), right now I'm running a newly installed 12.04.

I might try a SATA-USB converter with a 3,5" HDD, but that would differ significantly from my previous setup. Do you think that would yield some useful information?

[penalvch (penalvch)]

Bzzz, if your hardware has changed, and you are having a problem in Ubuntu, please file a new report by executing the following via the Terminal and feel free to subscribe me to it:
ubuntu-bug linux

Thanks!

[Bzzz (da-bzzz)]

No USB3 hardware -> no problem atm ;)

[penalvch (penalvch)]

Bzzz, this bug is being closed as you no longer have the hardware. Please feel free to report and future bugs you may find.