[Jaunty] Slow transfer rate writing to automounted usb drives

[Diaa Sami]

This is not a new bug but this report is against one of the latest development releases and should help pinpoint the bug.

When writing to usb devices, a flash drive in my case, performance is around 50% of that of Windows using the same hardware, when mounting the device manually(as root), performance goes back to normal.

I've performed some non-scientific experiments to help find the problem and its characteristics, you can find the results below and the log files requested attached.

Copying a 147 MB file from the PC(NTFS partition) to a 4GB USB drive(FAT32 partition):

Windows, copied using the GUI, stopwatch timing: 15s
Windows, copied using the cmdline(via copy), automatic timing(via wtime.exe): 13.8s

Jaunty Alpha 4, mounted automatically, copied using nautilus, stopwatch timing: 31s
Jaunty Alpha 4, mounted automatically, copied using terminal(cp), automatic timing(via time): 25.8s
Jaunty Alpha 4, mounted automatically, copied using terminal(dd), automatic timing(via dd): 26.1s, reported speed at end (5.9 MB/s)

Jaunty Alpha 4, mounted manually, copied using nautilus, stopwatch timing: 15s
Jaunty Alpha 4, mounted manually, copied using terminal(cp), automatic timing(via time): 16s
Jaunty Alpha 4, mounted manually, copied using terminal(dd), automatic timing(via dd): 16.1s, reported speed at end (9.5 MB/s)

Copying a 147 MB file from a USB drive(FAT32 partition) to the PC(NTFS partition):

Windows, copied using the GUI, stopwatch timing: 6.6s
Windows, copied using the cmdline(via copy), automatic timing(via wtime.exe): 6.8s

Jaunty Alpha 4, mounted automatically, copied using nautilus, stopwatch timing: 8s
Jaunty Alpha 4, mounted automatically, copied using terminal(cp), automatic timing(via time): 2.6s
Jaunty Alpha 4, mounted automatically, copied using terminal(dd), automatic timing(via dd): 18.8s, reported speed at end (8.2 MB/s)

Jaunty Alpha 4, mounted manually, copied using nautilus, stopwatch timing: 6.5s
Jaunty Alpha 4, mounted manually, copied using terminal(cp), automatic timing(via time): 2.5s
Jaunty Alpha 4, mounted manually, copied using terminal(dd), automatic timing(via dd): 17.9s, reported speed at end (8.6 MB/s)

[Scott James Remnant (Canonical) (canonical-scott)]

This has nothing to do with util-linux and everything to do with the kernel

[Leann Ogasawara (leannogasawara)]

Thank Diaa Sami.

The dmesg output that you attached, did you capture that after you had mounted the device automatically and then copied the file from the PC (NTFS partition) to the $GB USB drive (FAT32 partition)? If not, care to attach that.

Additionally, a few other things I'd appreciate if you could test. . . Jaunty Alpha5 is currently available and has a slightly updated 2.6.28-8.27 kernel. Might not hurt to give this a try and verify the issue remains.

If the issue still remains even with this newer 2.6.28-8.27 Jaunty Alpha5 kernel, would you be willing to test a few of the most recent upstream vanilla kernel?? We've packaged these upstream kernels to help users like yourself to easily install and test. The benefit is that you could then confirm if a fix might exist upstream, if the bug also exists upstream, or if it might be an Ubuntu specific change we need to look at. Please refer to https://wiki.ubuntu.com/KernelMainlineBuilds for more information. If you could test the 2.6.28.7 upstream kernel (http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.28.7/) and then 2.6.29-rc7 upstream kernel (http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.29-rc7/). Basically 2.6.28.7 is the upstream version that the Ubuntu 2.6.28-8.27 kernel is based on. If the upstream 2.6.28.7 kernel works then we'll know it's an Ubuntu specific change we need to narrow down. Likewise when you test the upstream 2.6.29-rc7 and it works, then there's likely a fix we might be able to backport.

Definitely let us know your results. I really appreciate it. Thanks in advance.

[Diaa Sami (diaa.sami)]

The dmesg output I attached was after I auto-mounting and manual-mounting.

I'd be glad to help fix this bug, I'm downloading Jaunty alpha 5 and I'll try and get back to you.

[Diaa Sami (diaa.sami)]

I've tried with Jaunty alpha 5, once with its default kernel and then tried each of the additional 2 kernels that you specified and it's the same.
HDD -> Flash is always significantly slower(23-24s vs 15-16s in Windows)
Flash -> HDD is almost unaffected(5-6s vs 6s in Windows)

[Leann Ogasawara (leannogasawara)]

Hi Diaa Sami,

Having reproduced the issue with the upstream kernel, it might not hurt also opening an upstream bug report as well at bugzilla.kernel.org. Thanks.

[Diaa Sami (diaa.sami)]

I'd like to try again to make sure I didn't miss anything, Is there a distro that uses the vanilla kernel that I can try? if not, do you think that trying Debian may help?

[teambuilding (teambuilding)]

Leann - is this bug related to the extremely slow transfer of files from PC HDD to external USB HDD? Can copy up to about 50 Megs no problems with speed, but then it virtually stands still and takes hours (literally) to finish off the rest. If a large file (e.g. 700 megs) the file never completes. This was introduced for the first time with jaunty, wasn't a problem with 8.10.

[Simon Holm (odie-cs)]

Hi Diaa,

can you provide us with the mount options for the automatic and manual mount cases? I.e.
grep <device> /proc/mounts

It would also be nice with a dstat trace of the data written to disk for both cases, see https://help.ubuntu.com/community/DiskPerformance or Theodore's comments in https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/197762.

@teambuilding: if it literally takes hours it sounds like a completely different problem and you should open a new bug. See https://help.ubuntu.com/community/DiskPerformance (or Theodore's comments) for what to put into a good bug report.

[Matt Joiner (anacrolix)]

I get this same effect, slowing after about 200MB, on an internal NTFS drive.

[Simon Holm (odie-cs)]

tor, 11 06 2009 kl. 01:16 +0000, skrev Matt Joiner:
> I get this same effect, slowing after about 200MB, on an internal NTFS
> drive.

This is just what the "Nautilus shows high transfer rate at beginning of
transfer but then slows down" part of [1] describes right? Please read
that page and please note that this bug is specifically about lower
performance of automatically vs. manually mounted device and nothing
else.

Diaa: We'd still like to see the actual mount options for the automatic
and manual cases, i.e.

grep <device> /proc/mounts

And we'd also like to see the dstat traces, see [1].

[1] https://help.ubuntu.com/community/DiskPerformance

[Diaa Sami (diaa.sami)]

Sorry for the delay, I'll redo the tests and provide the required info soon.

[Feistybird (bryanjen-tw)]

I found a fix that suggested that you add the option 'pci=routeirq' to the end of your kernel line in /boot/grub/menu.lst .

http://ubuntuforums.org/showpost.php?p=7847684&postcount=236

[Mander (arwintcher)]

I have this problem as well. I tried adding the pci-routeirq and it made no difference that I can see. Nautilus can take up to a minute to open my home folder, as well as directories on an NTFS partition or a vfat partition on an external hard drive.

It's intermittent for me, though. Any suggestions as to how to figure out what makes it lag sometimes, and not others?

[Diaa Sami (diaa.sami)]

well, unfortunately - for this bug - I changed my hardware(Intel P35+Core2Duo to AMD Phenom) and I can't reproduce the bug on my new hardware but I tried on my laptop and I think it happens, a certain file is copied on Windows in 23 seconds but on Kubuntu is copied in 27s when auto mounted and 19s when manually mounted.

Are you still interested in dstat output and/or mount options?

I'll provide them quickly this time.

[Przemek K. (azrael)]

Similar bug: bug #197762

[Przemek K. (azrael)]

Does this bug affect only USB flash drives or does it affect USB hard disks too?