[NO MORE COMMENTS] vague problems with USB file transfers

[Amar Boudjerida]

NOTICE: please do not add more comments to this bug! If you have a problem with USB transfers, please file your own report as these problems are typically hardware-dependent. Thanks!

== ORIGINAL DESCRIPTION ==

When transferring large files (800meg) or even medium-size files (200meg), the transfer rate decreases constantly. It starts at around 4meg/sec and goes down to even less than 800kb/s. When I cancel the download and start it over again, the transfer rate comes back up to 4meg/sec.
I am using Nautilus on Ubuntu 8.04 (hardy heron)

Bug Still Active up to release 9.10

[Sebastien Bacher (seb128)]

Thank you for your bug. Could you also use cp and gvfs-copy from a command line and compare the speed?

[Jeff Fortin Tam (kiddo)]

personnally, I could not reproduce this using a small ipod (1g) transferring a 380 mb file, nor with a conventional usb hard drive (transferring 1.4 gb in 3 files), using nautilus+gvfs.

[Jeff Fortin Tam (kiddo)]

I remote-controlled Amar's computer and did the requested testing. My benchmark was transferring two 350 mb avi files onto a 1 gib USB 2.0 key (this key is not the only one affected).

I did the testing using plain nautilus drag and drop, cp, and gvfs-copy.

nautilus = 15 mins
time cp: 3m21.337s
time gvfs-copy: 11m39.413s

We can assume the difference between gvfs' copy operation and the one done by cp in a terminal to be significant. Observing nautilus' behavior, it started at 7-9 mb/s, then gradually dropped speed over time and stabilized at 950 kb/s.

[Sebastien Bacher (seb128)]

unconfirming, copying a 650M video on an usb key takes the same time using cp and gvfs-cp on my hardy installation. Could you attach your messages, syslog and .xsession-errors log after copying?

[Jeff Fortin Tam (kiddo)]

I screencasted the problem (over a VNC session, so please bear with the ugliness :), and as you can see, a file that should have taken 30-40 seconds (if it kept transferring at its initial speed) took 2 mins 42.

[Jeff Fortin Tam (kiddo)]

note that the *original* report is not entirely accurate anymore. The speed is not in kb/s now, but it is around 4mb/s while it began at 7-9. So, the speed is not as excruciatingly slow as before, but a noticeable drop in performance can be seen nonetheless.

[Sebastien Bacher (seb128)]

still not confirming this behaviour on different installation and nobody else opened bug about similar issues, that seems to be specific to your configuration and would require debugging by somebody having the issue, you might want to open a bug on bugzilla about that, upstream might have better ideas on the topic

[Ivo Yordanov (lalanon)]

I have the same problem when copying files larger than 50-100 MB to any mass storage device on the USB interface. The transfer goes normally until the first ~100 MB are transfered and then rapidly the sped decreases to ~800 KB/s (on my 2GB MemoryStick inserted in my Sony PSP it starts at ~7 MB/s). The computer is Toshiba Satellite M45-S331 running Ubuntu 8.04. I believe there are other people experiencing the same problem, but since it's not a crash and the system does not offer to report a bug automatically, few regular users will report it at all.

[yngwie (yngwie)]

I can confirm I have the exact same problem with Ubuntu 8.04. Many users complain about the same issue, from what I could see on internet. Whatever the USB device I use (USB stick or external hard-disk plugged as USB storage), all my USB 2.0 devices (and recognized as such in syslog or messages files) run extremely slow (actually, it's basically USB 1.0 speed) after some 10 seconds. I both use a desktop PC and a Laptop, all of them include USB 2.0 interfaces: copying large files (say >600 MB) with Nautilus or cp lead to the same transfer rate (around 1 Mb/s or less) after a short while on those two PC. When I dual-boot with Win2K or WinXP with the exact same hardware (my laptop and my deskptop PC), the copy runs at the speed of light (compared to Ubuntu).

My hardware is :

Laptop: Fujitsu Amilo D7850 (Intel P4 2.8, 512 MB)
Desktop : assembled PC with MSI K7N2, AMD Athlon Barton 3000+, 1 GB RAM + Belkin USB 2.0 PCI adapter

I just installed Mandriva 2008.1 to give a try and to check if it makes any difference: it makes NO difference. So I suspect a kernel issue.

Did anyone facing the same issue try it with another distro ?

[Léo Studer (leo-studer)]

I confirm this issue with a fersh new Hardy install, usb file transfer has always been fine with the previous Hardy distros, so it must probably be a gvfs related issue.

[Léo Studer (leo-studer)]

I meant with the previous ubuntu distros, sorry about that

[Mac (mbobrov1973)]

Same problem here. Ubuntu 8.04. Toshiba Tecra M3. 8GB PQI USB drive.

[Evirsama (evirsama)]

Same problem with Ubuntu 8.04 on a AMD 64.

[jnygaard (jens-olav-nygaard)]

Same problem here. Really, really, really annoying. Actually worse than annoying: critical.

Ubuntu 8.04 installed the last couple of days. x86_64:

Linux xxx 2.6.24-16-generic #1 SMP Thu Apr 10 12:47:45 UTC 2008 x86_64 GNU/Linux

J.O.

[jnygaard (jens-olav-nygaard)]

I think this has to do with problems with ehci-hcd, maybe in connection with particular kernels and or motherboards and or chipsets... not sure.

When I remove this module, the drive works properly, maybe on 1.1-speeds, but at least it works...

J.O.

[sgytis (svgytis)]

This bug is critical - it prevents normal usage of USB HDDs. I tried copying a not so large number of files (~500, total size ~500MB) FROM attached USB HDD (250GB) to computer. After fast copy of initial 40-80MB copying crawls at 0.7MB/s. The bug is not computer dependent - the same happens on Shuttle Pentium4 with SATA disks and Thinkpad T40. The same slowness was observed when copying from USB HDD partitions with fat16, ext3 and NTFS file systems. It is even worse, after some time of copying and stopping it the computer is unable to mount the disk without rebooting. The speed of copying TO USB HDD is much faster however. Both computers have fresh Hardy installed.

[Sebastien Bacher (seb128)]

discussing the settings is not really constructive, that issue is a speed one that only some people get, it's not critical by any mean, it might be annoying but adding random comments about the bug settings here will not make it work automagically, the desktop team is small and has thousand of open bugs so everything can't be worked immediatly, especially if the issue is hardware dependent and an upstream one

[JonB (jonbennison)]

I have the same problem and it is extremely frustrating. Compaq Presario M2350ea laptop.

[zmago (zmago-fluks)]

For me also same problem... i have Acer Travelmate 2300 ...

USB is not useful for large files :(

[RamonBucher (thatvetguy)]

same here with eeepc 701 and ubuntu 8.04...

[Yannick (splitsch)]

Hello !
I have the same problem here.
The files are transfer at a 2Mio/sec. speed. Sometimes, though, it works at the "full" speed of 12Mio/sec.

Here is the output of lsmod | grep usb

[code]
usb_storage 73664 1
libusual 19108 1 usb_storage
lmpcm_usb 7168 0
usbhid 31872 0
hid 38784 1 usbhid
scsi_mod 151436 5 usb_storage,sg,sr_mod,sd_mod,libata
usbcore 146028 7 ehci_hcd,usb_storage,libusual,lmpcm_usb,usbhid,uhci_hcd
[/code]

Is there any way to fix this?

Thanks

[Pascal (spam-trash-ouvaton)]

same problem with Toshiba M60

lsmod|grep usb :
usbnet 20232 2 rndis_host,cdc_ether
usb_storage 73664 2
libusual 19108 1 usb_storage
dvb_usb_nova_t_usb2 6788 0
dvb_usb_dibusb_common 10756 1 dvb_usb_nova_t_usb2
dib3000mc 13960 2 dvb_usb_dibusb_common
dvb_usb 19852 2 dvb_usb_nova_t_usb2,dvb_usb_dibusb_common
dvb_core 81404 1 dvb_usb
i2c_core 24832 5 dvb_pll,mt2060,dib3000mc,dibx000_common,dvb_usb
usbhid 31872 0
hid 38784 1 usbhid
scsi_mod 151436 6 usb_storage,sbp2,sg,sr_mod,sd_mod,libata
mii 6400 3 usbnet,8139too,8139cp
usbcore 146028 11 rndis_host,cdc_ether,usbnet,usb_storage,libusual,dvb_usb_nova_t_usb2,dvb_usb,usbhid,ehci_hcd,uhci_hcd

[Sareen Shah (sareen-eng)]

Confirming here as well, Toshiba A75

not sure what info to provide, here's "lsmod | grep usb" as the above posters did

usb_storage 73664 1
libusual 19108 1 usb_storage
scsi_mod 151436 4 sg,sd_mod,usb_storage,libata
usbcore 146028 6 usb_storage,libusual,wacom,ehci_hcd,ohci_hcd

[Tim Kersten (timkersten)]

I can confirm this happens when using dd'ing a 1G image directly to a usb device.

(warning! this will erase all data on the usb key)
i.e. from terminal 1:

dd if=/dev/zero bs=1000 count=1000000 of=/dev/sdb # where /dev/sdb is your usb key

from terminal 2:
while true; do killall -SIGUSR1 dd; sleep 1; done # this will make dd print status messages ever second

The output is that the transfer speed starts at about 16MB/s and drops to a few KB/s after some minutes.

When running this same command on a fedora box, the transfer speed started at about 5.5MB/s and finished at 4.9MB/s.

[Pedro Villavicencio (pedro)]

thanks for the comment but when a bug is set as Triaged and with an upstream task it means that there's enough info which means no need to re confirm it everytime you want to subscribe to the bug, that will only create a lot of bugmail noise, thanks you all.

[Sebastien Bacher (seb128)]

the new comment is useful in the sense that it seems to indicate that the issue is not a gvfs one

[Neil Burlock (malone)]

I'm going to add a bit more that might be useful because this bug is driving me crazy.

It affects both my systems, one a laptop, the other a brand new desktop I purchased two weeks ago and is happening on both USB and network file transfers (via either Samba or UFS to my NAS box). I've also been experiencing occasional degraded GUI performance if I let a very large file copy (three GB or more) continue. My PC would gradually become un-responsive to the point that mouse clicks and key presses took seconds to be recognised and windows were being redrawn in slow motion. All the while, system monitor showed that both CPU's were nearly idle. When I closed the file copy window, performance returned to normal and there were no dmesg errors.

For anyone experiencing the problem, there's a kind of workaround that some people have had luck with (myself included). Try copying one file at a time and then pausing between each one to let the cache empty out, (since writing appears continue long after the copy dialog has closed). On USB devices in particular, try unmounting and re-mounting between files to make sure the file you are working on is really finished. Obviously, this is only going to work if you are copying a few large files around. If you have lots of small files to move, then zip them up before copying them and you might get lucky.

[Neil Burlock (malone)]

I seem to have happened upon a "solution".

I added pci=routeirq to the grub boot options and I got my USB and network performance back, and the GUI is actually *usable* while copying files too!!!!

I haven't run extensive tests, but I've been able to copy simultaneously over the network and to two USB devices at the same time with the transfer speed remaining constant, near the speed that I've been expecting (for so long), and I've done that repeatedly. That simply was impossible previously.

Over time I checked for this problem on three computers, and found it on all of them. Coincidentally, the three affected computers are all dual core AMD systems no more than 18 months old running various nVidia chipsets from various manufacturers.

This problem needs to be investigated because I can't be so unlucky as to have seen this bug so often and yet have the bug be one that "only some people get". Many more users out there must be experiencing this on a daily basis and thinking that "Ubuntu can't copy files properly", not realising that there's something wrong, and getting more and more frustrated, as I have been.

[Sebastien Bacher (seb128)]

the new comment seems to confirm the bug is not a gvfs one, could be nvidia specific, my machines are intel ones and I don't get the bug

[Evirsama (evirsama)]

Agree with Sebastien. Have a nvidia one with the bug. ubuntu release 7.10.

[Tim Kersten (timkersten)]

I had this on an amd/ati machine, so I don't think it's nvidia specific.

On Thu, Jul 24, 2008 at 12:41 PM, Evirsama <email address hidden> wrote:

> Agree with Sebastien. Have a nvidia one with the bug. ubuntu release
> 7.10.
>
> --
> file transfer on USB disk slows down with gvfs
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>

[giovani (wejick-gmail)]

On Thu, Jul 24, 2008 at 6:55 PM, Tim Kersten <email address hidden> wrote:
> I had this on an amd/ati machine, so I don't think it's nvidia specific.
>
> On Thu, Jul 24, 2008 at 12:41 PM, Evirsama <email address hidden> wrote:
>
>> Agree with Sebastien. Have a nvidia one with the bug. ubuntu release
>> 7.10.
>>
>> --
>> file transfer on USB disk slows down with gvfs
>> https://bugs.launchpad.net/bugs/197762
>> You received this bug notification because you are a direct subscriber
>> of the bug.
>>
>
> --
> file transfer on USB disk slows down with gvfs
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>

i had this on my p4 with SiS 661fx and my Pentium dual core e2xx with
intel 945g. So that clear all X-files about nVDIA specific.

--
ininih GiantiXBraiN kok otak nya gak besar besar
wejick.wordpress.com

[Gabe A (gabextreme)]

I have a Dell Inspiron E1405. This bug is present both when writing to my SD card and to my USB SD card reader. Different cards yield the same results, and it works fine/full-speed on Windows.

For me, this bus is very pronounced. When writing any file over about 100MB, the speed suddenly drops off to under 100KB/sec, where it remains for the rest of the transfer (starting from normal speeds of 15MB/sec).

pci=routeirq does NOT work for me (neither the built-in nor USB card-reader behave correctly after reboot/kernel refresh).

This bug is critical. I've also replicated it on my old Toshiba laptop exactly. Ubuntu Hardy, latest.

[Neil Burlock (malone)]

My pci=routeirq helped, but didn't turn out to be everything I hoped for. I can get away with copying a few GB of files now before the slowdown sets in, but it's still happening. The pattern is slightly different with this tweak however, since it seems to be copying fine for a certain number of files, then it'll start on the next file and the slowdown will start.

If this issue isn't related to gvfs, then shouldn't it be assigned elsewhere? It'd be nice if we could narrow down exactly which package was causing this. My vote at the moment is the kernel because of the improvement I've experienced from changing the boot options, but I'm hardly qualified.

[Tim Kersten (timkersten)]

I agree that this should be assigned to the kernel, as I can't see what
other package could affect write speeds using dd. "dd" is low level, has
nothing to do with gfs. My guess is it's the kernel, or whatever sits
between dd and the kernel. (Is there even something there?)

On Fri, Aug 1, 2008 at 6:36 PM, Neil Burlock <email address hidden> wrote:

> My pci=routeirq helped, but didn't turn out to be everything I hoped
> for. I can get away with copying a few GB of files now before the
> slowdown sets in, but it's still happening. The pattern is slightly
> different with this tweak however, since it seems to be copying fine for
> a certain number of files, then it'll start on the next file and the
> slowdown will start.
>
> If this issue isn't related to gvfs, then shouldn't it be assigned
> elsewhere? It'd be nice if we could narrow down exactly which package
> was causing this. My vote at the moment is the kernel because of the
> improvement I've experienced from changing the boot options, but I'm
> hardly qualified.
>
> --
> file transfer on USB disk slows down with gvfs
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in gvfs: New
> Status in "gvfs" source package in Ubuntu: Triaged
>
> Bug description:
> When transferring large files (800meg) or even medium-size files (200meg),
> the transfer rate decreases constantly. It starts at around 4meg/sec and
> goes down to even less than 800kb/s. When I cancel the download and start it
> over again, the transfer rate comes back up to 4meg/sec.
> I am using Nautilus on Ubuntu 8.04 (hardy heron)
>

[Dale Clarke (dale-clardale-deactivatedaccount)]

This bug is also affecting Firewire hard drives, I am down to 18.5kb/sec and can take a very long time. Tried the solution above but alas nothing.

I am ATI X1550 graphics along with Intel Core duo...

I would class this as critical as it means I can not backup the system to an external disk which makes Ubuntu unusable for me!

[Bugsy (carlo-suomi24-deactivatedaccount)]

I can confirm that this affects also Intel G965 and P35 chipsets whit Lacie USB2 hard-drives. Almost allways the speed is 3-6 MB/s but sometimes it works in full speed 30 MB/s.

I've attached a screenshot where I first transferred one file to the usb2-drive with average speed over 30 MB/s and after 15 minutes when I tried to transfer different file to the drive the transfer speed was only about 3 MB/s.

I think that this is a lot more worse bug than our developers think because making backups is getting too slow.

Also the pci=routeirq did not help for me and this affects both nautilus and cp transfer from command line.

[Murz (murznn)]

Simon Holm Thøgersen, I can try to do the reports, can you give me some step-by-step instructions and examples how I can do it better?
I can try to install new kernel and do some tests, if it will not too hard and long process.
At now I try deadline scheduler and see the problem again after copying about 1.5gb to the flash. At start when copying first gb the speed was about 5-6 mb/sec, but after 1.5gb slows down to 0.2-1.5 mb.
I have attach part of output "iostat -m 1" command at ~1.6-1.7 gb copied and speed is slow down in deadline scheduler.

P.S. I have found an easy way yo check current scheduler and switch it without reboot at http://www.cyberciti.biz/faq/linux-change-io-scheduler-for-harddisk/:
View Current Disk scheduler
# cat /sys/block/{DEVICE-NAME}/queue/scheduler
# cat /sys/block/sda/queue/scheduler
Sample output:
noop anticipatory deadline [cfq]

Set I/O Scheduler For A Hard Disk
To set a specific scheduler, simply type the command as follows:
# echo {SCHEDULER-NAME} > /sys/block/{DEVICE-NAME}/queue/scheduler

[u356 (pavelkok)]

Murz, thanks! Change queue to noop solve my problem without reboot.
Now I have stable 8-9 Mbyte/sec with SD class 6 card.
This trouble affected my desktop (64 bit) and notebook (32 bit) Ubuntu 9.10 installs.

[u356 (pavelkok)]

Update
After 10 Gbyte transfer to sd card speed is down again. Scheduler fix is not enought for me.

May be it would help for someone - http://ubuntuforums.org/showpost.php?p=8653721&postcount=29
"yes its fixed, but not in the 2.6.31 kernel branch. I experienced the same problem you described when trying to copy files from my internal hard disk to either a usb external hard disk or a usb thumbdrive. the speed would start at about 30 MB/s then quickly drop down to 5 MB/s and sometimes get down to even 1.5 MB/s. I thought it was pretty funny that i could download files from the Internet at 2.6 MB/s but couldnt even transfer them via usb that fast. It also spiked my CPU and caused my wifi to hang and disconnect as well.

Last night I installed the Linux 2.6.33RC3 kernel from:
http://kernel.ubuntu.com/~kernel-ppa...e/v2.6.33-rc3/

I ran some tests and found that my USB file transfers ran at full speed and did not drop like before. I should point out that I am running the 64bit kernel so i dont know if this issue exists with the 32bit kernel or not."

[Marc MAURICE (dooblem)]

I did all the following tests, and always the same problem : speed decrease to 2-3 MB/s.

Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.31-17-generic + Nautilus
Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.31-17-generic + rsync
Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.33-rc6 + Nautilus
Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.33-rc6 + rsync

Thinkpad X31 + Archlinux + Linux 2.6.32 + Dolphin
Thinkpad X31 + Archlinux + Linux 2.6.32 + rsync

[tonyo (toverfield)]

I have this problem too. HP Pavilion (Core 2 Quad, 3 GB RAM, 2.6.31-17-generic x86_64)

I was unable to use dd to write a memstick image to a Sandisk USB stick. It was crawling to a virtual stop before reaching 40% complete. I hope my observations will help to solve this problem.

My first observation was that I could use dstat output to anticipate the slow down and then press "Ctrl-Z" to suspend the dd so that the write buffers could drain. Carefully nursing the transfer along resulted in a successful and quick copy.

My other observation was that I could work around this problem using the "oflag=direct" option to dd, which presumably bypasses the lazy write buffering.

# dd if=sandisku3.dd of=/dev/sdh bs=256k oflag=direct
7839+1 records in
7839+1 records out
2055021056 bytes (2.1 GB) copied, 217.913 s, 9.4 MB/s

I think these observations suggest that the extremely high write backlog (over 700MB "Dirty" in /proc/meminfo) causes a performance fall-over when available memory resources become exhausted.

[Mitch Towner (kermiac)]

Setting this task back to invalid.

@ Fail2Ban: Please do not change the status of tasks without commenting to advise why you have done this.
Thanks in advance!

[gdneil (nsy999)]

Hello.... Just commenting on the same problem - which for me also appeared with external USB hard drives.... Transfer rate slowing down with large files.

Tried all of the suggestions...

-Starting with pci=routeirq helped a little maybe?
-Turning off image preview in Nautilus might have helped a little?
-Reformatting externals (flash and HDs) to ntfs instead of fat helped the most
-Recompiling the latest ntfs-3g helped a little more

And then in frustration, I installed Debian(Squeeze) on an extra partition and timed transfers of big files of different types (fat32, ntfs, ext3, ext4) and found that Debian was much much much faster in all cases. Unfortunately though, I seem to have lost the notes with my actual timings.... Not sure if this helps, but hope it might... the problem (for me at least) seems to be coming from the Ubuntu side...

I've used Ubuntu since 8.10 and it has always been slower at transferring files than Windows, but 9.10 was significantly worse. Kind of useable with the mods above, but nowhere near debian. Tried putting some of the relevant debian packages into my ubuntu installation, but no luck (and not enough knowledge/experience in these things I admit) but maybe a possible direction? For now though, finding myself more often booting Debian.

All the best,
-Neil

Toshiba Libretto U100
1gb RAM, lots of external HDs

[kasrawis (kasrawis)]

same problem here :( seems to be bugs in kernal, slow copying :(

[RishiRamraj (thereisnocowlevel)]

gdneil, kasrawis:

Have you tried setting elevator=noop as per post #233?

all:
I suspect that there might be no singular cause for this problem. If I find the time, I'll try to weed through all of the bug reports (including duplicates) and identify the suspected causes, platforms affected and proposed workarounds.

[Digital5700 (edward-edwardh)]

echo 10 > dirty_writeback_centisecs
echo 10 > dirty_expire_centisecs

Now it should basically write to the stick continously instead of every 3000 centisecs (30 secs).

I learned when I found that transferring terabytes through my gigabit results in 30s long pauses, then 80mb/s for a few seconds and then another pause.

The same principle should be applicable to usb sticks, although it won't explain why usb harddrives are much faster...

[Pi (sub2008-deactivatedaccount-deactivatedaccount)]

I believe this is the same bug, I asked in the ubuntuforums.org and someone told me to post my discoveries here:

Whenever I plug-in my USB 8gb pendrive to copy a large file, it takes eons.

Using Ubuntu 9.10 for AMD64.

At first it copies a file at high speeds, because it's not actually writing to the USB but to the cache; at 500-700mb, the write cache is full and flushes, *while* still copying the rest of the file. Since the system is trying to access the USB in two different places at the same time, the transfer speed bunks down, often to 200-300kb/s. The final process, instead of taking 2 minutes, takes one hour. This continues until 1.4gb, when the cache finishes the flush, the copy can continue streamlined and the last 200-300mb are copied again at quicker speeds.

I've made a test, copying a 1.6gb file to the USB and waiting for the cache to get full. Once it happens, I copy another file of the same size to the USB at the same time, when it starts copying I cancel the previous copy - cancelling it could take 1-2 minutes, but once it's cancelled, the second copy goes amazingly quick since the pendrive doesn't have any access troubles anymore. However, that means for the trick to work that I can't copy four 1.6 files to the 8gb pendrive, but only three. The fourth one would need an hour unless I start copying it just before the first three finish so the write cache is still not in use or something like that.

IMHO the problem is using the cache in certain USB drives. For example the same problem happens with my usb-connected HDD; but due to its faster access time, the slowdown is less noticeable and disappears quickly. The low access time in usb pendrives makes the slowdown due to the cache to last for too long to be bearable.

I hope this clarifies the situation; sorry if this has been mentioned before but I didn't have time to check such a long bug report before adding my own experience.

Is there simply a method to disable cache (or lower it to a minimum) to an already mounted device?

[Ulrich Lukas (ulrich-lukas)]

Pi, this is consistent with my observations.

[Olaf Geibig (olafgeibig)]

So it worked for me. I added these two parameters to my kopt line in /boot/grub/menu.lst as described in post #233

elevator=noop pci=routeirq

It's still not a speed king but it doesn't take ages now.

[Angelos Vlassopoulos (avlass)]

How can I do these changes on Karmic's Grub2? There is no menu.lst anymore

[mlx (myxal-mxl)]

Grub now uses binary configuration format, which has to be generated
from a text file. The text file is /etc/default/grub. Edit it, then
run update-grub.

[kwstas tsob (grandamra)]

hi guys,

i thought i could post here something i found out..

i had similar issues but also when i was trying to copy a file(around 50-700mb or larger) from my local hdd to another folder of the same hdd!!!
so what i found out was that this was happening only in files that i had downloaded with "transmission"!
i tried copying with files i downloaded with "firefox" or "qbittorrent" and until now i have no problem!

hope it helps some of you

[Theodore Ts'o (tytso)]

>i had similar issues but also when i was trying to copy a file(around 50-700mb or larger) from my
>local hdd to another folder of the same hdd!!!
>so what i found out was that this was happening only in files that i had downloaded with "transmission"!

What file system were you using on your hdd, and what version of Ubuntu are you using?

This isn't surprising, because bittorrent downloads tend to write the file in random order (depending on what is made available from the bt seeders and the bt peers) and in multiple threads. Using ext4 and a relatively modern kernel, it won't be too bad, because ext4 has better allocation algorithms than ext3. It's best though, if the bt client uses fallocate(), which is I'll bet what qbittorrent is doing.

So this is a useful bit of information, but I suspect some users have different causes for their problems, which is one of the reasons why most kernel developers have refused to use Launchpad. Too many ubuntu users are glomming onto a bug report, and they post different performance problems all in a single bug report, without bothering to give full information about their hardware, software versions, what file system they are using, etc. And then on top of that Launchpad is slow as a dog when there it gets too long.

Which is why this bug is marked invalid and why I beg people, if they have performance problems, to open new bugs and to give full details about their hardware configuration, what version of Ubuntu they are running, the exact specifics of the USB device, etc. Otherwise this can be a great place for people to vent, but most people who aren't paid to pay attention to dysfunctional bug forums will simply say, "Beam me up Scotty, there's no intelligent life here".

[allisterbrizan (allisterbrizan)]

Please don't brush this aside as "poor performance". This is an issue in long disk activities. I notice this bug in both disk copies and dd.
I use dd quite frequently to clone disk images onto identical computers and very regularly have to copy these 75.4GB disk images. While i have not done a forensic investigation this is what i have noticed after facing this issue MANY times:

1. It definitely affects Ububtu 9.10, Ububtu 9.04, Ububtu 8.10, Ububtu 9.10UNR, Ububtu 9.10UNR. (It seems to be a bigger problem UNR distros)

2. In Ubuntu 9.04 i get this slowdown in approximately 25% of the dd clones i do (2 out of 7 computers this weekend). Canceling and restarting the computer don't fix this issue.
Note: For dd clones i boot off a USB stick and invoke dd in a terminal window. The drive images copy from a file on a NTFS drive to /dev/sda or vice-versa.

3. Using the computer makes this bug less likely to happen. (I truly believe this has something to do with efficient power consumption or reducing disk writes.)

4. The read/write rate decreases over time in a predictable manner. A dd clone which works normally copies the 75.4GB file in 3054s give or take 10. If the bug occurs it takes approx. 23200s.

This is a real problem. I will do some proper testing and analysis based on suggestion.

[Theodore Ts'o (tytso)]

@allisterbrizan,

Please, do the investigation, and then open a new bug!!!

Your issue with dd clones is completely different from kwstas tsob's problem in #256 which had to do with source file fragmentation because of bittorrent.

You see? Both are performance problems, and both are complete different!!

And discussing them in the same bug report in Launchpad is completely unproductive!!!

So the issue is not writing off performance problems; just this specific Launchpad bug, which is clearly degenerated beyond any hope of help. (It doesn't help that launchpad is very slow when there are over 80 comments in the bug report.)

If you want to have any hope of anyone paying attention, OPEN A NEW BUG.

[kwstas tsob (grandamra)]

Theodore Ts'o

as i said in my first post i only posted this so i could help others understand better the problem. i had already posted to the forum of transmission about it...

i'm on karmic 9.10 with ext3.

[kwstas tsob (grandamra)]

it seems that i solved the problem by changing the preallocation number of transmission's conf file from 1 to 2.

[Zordid (zordid-gmx)]

This bug is the most annoying, most critical and nerve-wracking one in the whole Ubuntu universe!
This makes me hate my beloved Ubuntu cause I am not able to use ANY USB-connected memory device! I can also constitute that when copying large files (>100MB) to any USB device (memory stick, hard drive, does not matter...) the transfer starts "normal" with a couple of MB/sec, everything's normal - but then, after 100 MB have been transfered, the speed drops dramatically down to a few KB/sec - when copying 2 GB files, I sometimes get the remaining time up to over 1 hour!!

What I can see then, is that the load of the system rises to 6 or even 8 - only because of that file transfer going on. When the copy finally is done, the system behaves normal again and the load is where it belongs, 0.something or so.

This problem has been there for quite some time now, at least the last 1,5 to 2 years! I am not sure though when it first hit me...

I am using the 64bit Ubuntu 9.10 currently on a Dell Latitude D830.

Is there ANYBODY out there who knows how to fix this?? I don't want to have to use Windows just because file copying is so damn slow! Also, I am always kind of in a trouble when I want to copy files for friends: when they see how long it takes I cannot find a single positive argument for them to switch to Ubuntu anymore!! :-(

[zvaral (z-varallyay)]

This issue seems to be quite old and still exist in Ubuntu Lucid Lynx 10.04 beta.

(Bug #541937 is a duplicate of this bug report but I reported it because this problem occurred in a newer Ubuntu release mentioned here.)

[cheffe76 (p-letna)]

The priority should be changed to high. This is a BIG issue, someone capable should fix it.

Bug description:
Ubuntu 9.10 -> Slow USB transfer speed, while copying (starts with high speed for about 150m, then speed drops to <1 MB/s.

Tried it with several USB sticks. Did not try the terminal yet.

Cheers

[Timm (tbaumeister)]

Similar problems occur when copying large files from a virtualbox guest to the ubuntu host (see Bug #557796) While this may be entirely unrelated, I am posting this because both problems occur on my system and have similar symptoms.

[adri58 (adri58)]

El 08/04/2010 5:58, Timm escribió:
> Similar problems occur when copying large files from a virtualbox guest
> to the ubuntu host (see Bug #557796) While this may be entirely
> unrelated, I am posting this because both problems occur on my system
> and have similar symptoms.
>
>
But that problem is from virtalbox, i think, not linux

[David Malizia (david-malizia)]

Same Problem.

I use VMWARE player and it can take control of USB devices. If I insert any pendrive Windows 2003 through VMWARE recognizes it as a USB 1.1 device. If I try the same trick booting my PC from windows the pendrive is recognized as USB 2.0 both by the host windows XP and the vmware windows 2003.

$uname -a
Linux ubuntu-best 2.6.31-20-generic-pae #58-Ubuntu SMP Fri Mar 12 06:25:51 UTC 2010 i686 GNU/Linux

$lsusb

Bus 006 Device 002: ID 046d:c521 Logitech, Inc. MX620 Laser Cordless Mouse
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 004: ID 0781:5406 SanDisk Corp. Cruzer Micro 1/2/4GB Flash Drive
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 002: ID 04f2:b064 Chicony Electronics Co., Ltd
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 002: ID 08ff:1600 AuthenTec, Inc. AES1600
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

$ lsusb -vv -s 002:004

Bus 002 Device 004: ID 0781:5406 SanDisk Corp. Cruzer Micro 1/2/4GB Flash Drive
Device Descriptor:
  bLength 18
  bDescriptorType 1
  bcdUSB 2.00
  bDeviceClass 0 (Defined at Interface level)
  bDeviceSubClass 0
  bDeviceProtocol 0
  bMaxPacketSize0 64
  idVendor 0x0781 SanDisk Corp.
  idProduct 0x5406 Cruzer Micro 1/2/4GB Flash Drive
  bcdDevice 2.00
  iManufacturer 1
  iProduct 2
  iSerial 3
  bNumConfigurations 1
  Configuration Descriptor:
    bLength 9
    bDescriptorType 2
    wTotalLength 32
    bNumInterfaces 1
    bConfigurationValue 1
    iConfiguration 0
    bmAttributes 0x80
      (Bus Powered)
    MaxPower 200mA
    Interface Descriptor:
      bLength 9
      bDescriptorType 4
      bInterfaceNumber 0
      bAlternateSetting 0
      bNumEndpoints 2
      bInterfaceClass 8 Mass Storage
      bInterfaceSubClass 6 SCSI
      bInterfaceProtocol 80 Bulk (Zip)
      iInterface 0
      Endpoint Descriptor:
        bLength 7
        bDescriptorType 5
        bEndpointAddress 0x81 EP 1 IN
        bmAttributes 2
          Transfer Type Bulk
          Synch Type None
          Usage Type Data
        wMaxPacketSize 0x0200 1x 512 bytes
        bInterval 0
      Endpoint Descriptor:
        bLength 7
        bDescriptorType 5
        bEndpointAddress 0x02 EP 2 OUT
        bmAttributes 2
          Transfer Type Bulk
          Synch Type None
          Usage Type Data
        wMaxPacketSize 0x0200 1x 512 bytes
        bInterval 0
can't get de...

[mlx (myxal-mxl)]

#267: That's a vmWare problem. See here> http://communities.vmware.com/thread/118175

@Canonical: Is it a good idea to have people with completely different issues dogpile on this evergreen bug? Marking any new bug as a duplicate of this one instead of telling them to get more info, referring them to https://help.ubuntu.com/community/DiskPerformance probably isn't helping, either :-\

[Theodore Ts'o (tytso)]

Comment which I added to bug #541937:

No, no, no!!!!!

Please, can we not tell people to dogpile onto bug #197762?

That bug is so dogpiled as to be useless.

We need to have separate bugs for separate performance problems, instead of assuming that all reported performance problems with USB drives should be directed to a bug that multiple people have been trying to shut down as "there's no intelligent life here, Scotty", which was opened over two years ago, and for which there are at least three separate and distinct problems, and probably more, all horribly mixed together.

Can someone official in Ubuntu step up and say whether or not this horrible practice should be continued?

I'm seriously thinking about writing a blog entry, "Why Launchpad is terminally broken, and why all upstream devs should ignore it as a waste of their time", and pointing to #197762, and misguided attempts to get people to dogpile onto this misbegotten bug, as a exhibit number #1 as to why kernel developers are ignoring Launchpad and why it is more of a menance than a help to upstream developers. If canonical isn't providing upstream development assistance, and they're not providing enough talented people to do bug triage, and they are claiming that the value they are bringing to upstream projects is lots of users and lots of bug reports, this is a prime example of why This Isn't Working For Us.

[Jeremy Foshee (jeremyfoshee)]

Ted,
     I couldn't agree more. I'm not sure I fall into what you would call "someone official in Ubuntu", but I am the Kernel Bug Triager. My stance on these is to have reporters, with any deviation from the reported bug they think is affecting them, file a new bug while subscribing to the bug they find similar. In this fashion, they are able to test fixes against their reported issue for the other reported bugs. The difficulty is in the push to consolidate that seems to be afflicting some bug tragers. This is difficult to address, as I am sure you can imagine.

For what it is worth, I appreciate that you take the time to look at these issues and comment on them. Given that the nature of this bug ticket has changed so much since it was reported, and given the fact that no one here seems to be willing to continue work on their respective bugs. I am marking this particular bug as Invalid. Those of you affected by this whose bugs are marked as duplicates will need to remove the duplicate designation. To the reporter, Amar Boudjerida, please open a new bug for your issue and we will begin again.

To those of you who did not file a bug in favor of marking this bug 'me too', please file a bug with your collected data and a clear description of the issue in both the bug title and the description fields so that we may work them as appropriate.

Ted, I'd like for you and I to chat a bit about the best way to go about the ubuntu kernel bug filing process if you are open to it. I think between us we can get this to a better place.

Thanks!

~JFo

[Shawn Reist (icabaud)]

I have been following this post for some time.

I have installed Debian Lenny and started using that as a primary system
and the Ubuntu system I have I tinker with.

Call me a rebel or such.. but I have noticed differences on how the
systems' (Ubuntu and Debian) handle file transfers.

I am not a super geek and can't devote a lot of time to tracking down
the differences.

One thing I have noticed is that Ubuntu does not write some of the
configuration files for USB devices. . .or that some of the utilities
required to read the USB information can't retrieve anything because the
required files do not exist. there are increasing wait periods or data
written/read from a USB device, something in the wait states. . .
someone with more technical familiarity needs to look at these.

As far as I can see there are several issues, but without some help or
guidance I can't give more information. Most people from what I see are
quoting the same fixes over and over; it is or has become a circular
enigma. Specific data has to be collected from a range of systems in a
specific order and collated/sorted through.

Shawn.

[Theodore Ts'o (tytso)]

On Tue, Apr 20, 2010 at 09:06:13PM -0000, Jeremy Foshee wrote:
> Ted,
> I couldn't agree more. I'm not sure I fall into what you would
> call "someone official in Ubuntu", but I am the Kernel Bug
> Triager. My stance on these is to have reporters, with any deviation
> from the reported bug they think is affecting them, file a new bug
> while subscribing to the bug they find similar. In this fashion, they
> are able to test fixes against their reported issue for the other
> reported bugs. The difficulty is in the push to consolidate that
> seems to be afflicting some bug tragers. This is difficult to
> address, as I am sure you can imagine.

So let me suggest some technological fixes first --- not that they
will completely solve the problem, but Launchpad desperately needs them.

The first is I think we need to have a way for someone to put a
statement at the very top of the Launchpad bug. Either right before
or right after the bug description. This would be a place for someone
to put a message such as "for people who think that they have this
problem, please see this wiki page first, which will describe how to
do more bug-specific diagnosis --- i.e., how to collect various bits
of diagnostic data and then to suggest which of several subsidary bugs
that they can check the "this bug affects me too" box.

One of the major problem is that Launchpad is !@#@! slow once it has
more than several hundred comments, and if people try to put helpful
information in the two hundred and seventieth comment, most users
never bother to read that far. So you really need a way to put
information to users who are first getting redirected to the bug that
they will actually *see*.

The other thing that you really need is a way for a bug to be closed.
A bug which is closed is one which can't be used as a duplicate, and
for which no one but an admin is allowed to post further comments, or
change the state of the bug, or add some other package as being
affected by a bug. It's basically a way of saying, "This bug is a EPA
Superfund toxic waste site, and please open a new bug if you think you
have a similar problem". Combined with the first feature, where
someone can place a message explaining why a bug is closed, and what
people should do, would be a big, big help.

Another thing that might be useful (and which would also require some
number of triagers to have privilege bits; you can't give it out to
everyone), is some way to mark certain comments as being irrelevant.
If the bug is really about USB1 vs. USB2 detection, the last thing you
need is for someone to say, "me too! I have a HDD performance problem
when using XFS". When we have a vast number of non-technical users,
we ****desperately**** need a way to moderate out the crap comments.
Maybe for political reasons they are displayed by default, but if an
upstream developer is going to viewing the bug, they really will want
a way to hit a button and get rid of the irrelevant comments. (Maybe
there would be a moderation reason where the triager can say,
"unrelated problem; please open a new bug report with full details of
your hardware and software configuration" before marking a comment
with the crap...

[David Tombs (dgtombs)]

OK Ted, I have taken your advice and changed the title and description. Hope this will solve some problems.

[Martin Pool (mbp)]

Hi Ted,

> Launchpad is slow

Sadly often true. I'm told this may be the major focus for later in 2010.

> Put a message at the top

The usual thing is to change the description. There is a different bug asking for ways for packagers to display more messages later. Unfortunately some users won't read them before talking.

> Close/lock bugs

https://bugs.edge.launchpad.net/malone/+bug/73122

> Filtered view

Doing this as an opt-in mode for upstreams would be interesting. There are some other things like code reviews where we do at least highlight comments by well-known users.

[Pavol Klačanský (pavolzetor-deactivatedaccount)]

same problem, with external USB hard drive, speed falls from 20 MB/s to 1 MB/s, I need back up 100GB, its ridiculous

[Charlton (charlton-grant)]

Linux home 2.6.32-25-generic #44-Ubuntu SMP Fri Sep 17 20:05:27 UTC 2010 x86_64 GNU/Linux

AMD 64 quad processor same problem
USB thumb drives formatted to fat, vfat, ntfs, ext3.
Various sizes 1gb - 16gb
various manufacturers

initial fast copy +320mb (or so it says) then slows to a crawl if it finishes at all. via command line and nautilus.
When I do boot into windows on the same machine no problems whatsoever other than the fact I am in windows. :(
I have had this problem since 9.04 maybe before but I am using ubuntu for everything now.

[raimondohc (raimondohc)]

Same problem here: transfer starts well (about 15-20 Mb) but immediately (in less than 20 seconds falls to 7-5 Mb) slows down upt to 1,5 Mb or less. That problem doesn't appear in Windows XP SP3.
Here's my specs:
Ubuntu 10.04.1 LTS 64 bits
Kernel version: Linux home 2.6.32-25-generic
System up-to-date
CPU: Intel Pentium E2160
MB: Nvidia MCP73
GPU: GeForce 7300 LE
HDD: Samsung SATA 500 Gb

[Nathanel Titane (nathanel.titane)]

running up-to-date ubuntu natty amd64 and reporting (AGAIN!) to state that it took over 45 minutes to get 3.2 Gb in my usb flash drive.

The mainline kernel "fix" does not seem to affect my system nor does it even show any sort of improvement eversince i've been on ubuntu (hardy heron amd64)

The transfer rate was at about 1.1 Mb/sec throughout the whole thing and I'm using a main port (USB 2.0) to transfer my stuff in...

SYSTEM REPORTNOT ATTACHED.. OOPS ERROR

UNAME -A--------------------------------------------------------------------------------------------------------------------------------------------------

Linux Mercury 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

HWINFO-----------------------------------------------------------------------------------------------------------------------------------------------------

============ start debug info ============
libhd version 16.0u (x86-64)
using /var/lib/hardware
kernel version is 2.6
----- /proc/cmdline -----
  BOOT_IMAGE=/boot/vmlinuz-2.6.38-8-generic root=UUID=7ec66c7c-c592-4c97-a211-1993619209b0 ro quiet splash vga=795 vt.handoff=7
----- /proc/cmdline end -----
debug = 0xff7ffff7
probe = 0x00138fc4aa17fcf9fffe (+memory +pci +isapnp +net +floppy +misc +misc.serial +misc.par +misc.floppy +serial +cpu +bios +monitor +mouse +scsi +usb -usb.mods +modem +modem.usb +parallel +parallel.lp +parallel.zip -isa -isa.isdn +isdn +kbd +prom +sbus +int +braille +braille.alva +braille.fhp +braille.ht -ignx11 +sys -bios.vbe -isapnp.old -isapnp.new -isapnp.mod +braille.baum -manual +fb +pppoe -scan +pcmcia -fork -parallel.imm +s390 -cpuemu -sysfs -s390disks +udev +block +block.cdrom +block.part +edd +edd.mod -bios.ddc -bios.fb -bios.mode +input +block.mods +bios.vesa -cpuemu.debug -scsi.noserial +wlan -bios.crc -hal -bios.vram -bios.acpi -bios.ddc.ports -modules.pata -net.eeprom -x86emu -max -lxrc)
shm: attached segment 10059815 at 0x7f396443d000
>> hal.1: read hal data
  hal: connected to: :1.38
  hal: empty device list
>> floppy.1: get nvram
>> floppy.2: klog info
>> bios.1: cmdline
>> bios.1.1: apm
>> bios.2: ram
  bios: 0 disks
>> bios.2: rom
>> bios.3: smp
----- BIOS data 0x00400 - 0x004ff -----
  400 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  430 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  440 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  450 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  460 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  470 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  480 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  490 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  4a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  4b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  4c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "...............