[NO MORE COMMENTS] vague problems with USB file transfers

[Amar Boudjerida]

NOTICE: please do not add more comments to this bug! If you have a problem with USB transfers, please file your own report as these problems are typically hardware-dependent. Thanks!

== ORIGINAL DESCRIPTION ==

When transferring large files (800meg) or even medium-size files (200meg), the transfer rate decreases constantly. It starts at around 4meg/sec and goes down to even less than 800kb/s. When I cancel the download and start it over again, the transfer rate comes back up to 4meg/sec.
I am using Nautilus on Ubuntu 8.04 (hardy heron)

Bug Still Active up to release 9.10

[Sebastien Bacher (seb128)]

Thank you for your bug. Could you also use cp and gvfs-copy from a command line and compare the speed?

[Jeff Fortin Tam (kiddo)]

personnally, I could not reproduce this using a small ipod (1g) transferring a 380 mb file, nor with a conventional usb hard drive (transferring 1.4 gb in 3 files), using nautilus+gvfs.

[Jeff Fortin Tam (kiddo)]

I remote-controlled Amar's computer and did the requested testing. My benchmark was transferring two 350 mb avi files onto a 1 gib USB 2.0 key (this key is not the only one affected).

I did the testing using plain nautilus drag and drop, cp, and gvfs-copy.

nautilus = 15 mins
time cp: 3m21.337s
time gvfs-copy: 11m39.413s

We can assume the difference between gvfs' copy operation and the one done by cp in a terminal to be significant. Observing nautilus' behavior, it started at 7-9 mb/s, then gradually dropped speed over time and stabilized at 950 kb/s.

[Sebastien Bacher (seb128)]

unconfirming, copying a 650M video on an usb key takes the same time using cp and gvfs-cp on my hardy installation. Could you attach your messages, syslog and .xsession-errors log after copying?

[Jeff Fortin Tam (kiddo)]

I screencasted the problem (over a VNC session, so please bear with the ugliness :), and as you can see, a file that should have taken 30-40 seconds (if it kept transferring at its initial speed) took 2 mins 42.

[Jeff Fortin Tam (kiddo)]

note that the *original* report is not entirely accurate anymore. The speed is not in kb/s now, but it is around 4mb/s while it began at 7-9. So, the speed is not as excruciatingly slow as before, but a noticeable drop in performance can be seen nonetheless.

[Sebastien Bacher (seb128)]

still not confirming this behaviour on different installation and nobody else opened bug about similar issues, that seems to be specific to your configuration and would require debugging by somebody having the issue, you might want to open a bug on bugzilla about that, upstream might have better ideas on the topic

[Ivo Yordanov (lalanon)]

I have the same problem when copying files larger than 50-100 MB to any mass storage device on the USB interface. The transfer goes normally until the first ~100 MB are transfered and then rapidly the sped decreases to ~800 KB/s (on my 2GB MemoryStick inserted in my Sony PSP it starts at ~7 MB/s). The computer is Toshiba Satellite M45-S331 running Ubuntu 8.04. I believe there are other people experiencing the same problem, but since it's not a crash and the system does not offer to report a bug automatically, few regular users will report it at all.

[yngwie (yngwie)]

I can confirm I have the exact same problem with Ubuntu 8.04. Many users complain about the same issue, from what I could see on internet. Whatever the USB device I use (USB stick or external hard-disk plugged as USB storage), all my USB 2.0 devices (and recognized as such in syslog or messages files) run extremely slow (actually, it's basically USB 1.0 speed) after some 10 seconds. I both use a desktop PC and a Laptop, all of them include USB 2.0 interfaces: copying large files (say >600 MB) with Nautilus or cp lead to the same transfer rate (around 1 Mb/s or less) after a short while on those two PC. When I dual-boot with Win2K or WinXP with the exact same hardware (my laptop and my deskptop PC), the copy runs at the speed of light (compared to Ubuntu).

My hardware is :

Laptop: Fujitsu Amilo D7850 (Intel P4 2.8, 512 MB)
Desktop : assembled PC with MSI K7N2, AMD Athlon Barton 3000+, 1 GB RAM + Belkin USB 2.0 PCI adapter

I just installed Mandriva 2008.1 to give a try and to check if it makes any difference: it makes NO difference. So I suspect a kernel issue.

Did anyone facing the same issue try it with another distro ?

[Léo Studer (leo-studer)]

I confirm this issue with a fersh new Hardy install, usb file transfer has always been fine with the previous Hardy distros, so it must probably be a gvfs related issue.

[Léo Studer (leo-studer)]

I meant with the previous ubuntu distros, sorry about that

[Mac (mbobrov1973)]

Same problem here. Ubuntu 8.04. Toshiba Tecra M3. 8GB PQI USB drive.

[Evirsama (evirsama)]

Same problem with Ubuntu 8.04 on a AMD 64.

[jnygaard (jens-olav-nygaard)]

Same problem here. Really, really, really annoying. Actually worse than annoying: critical.

Ubuntu 8.04 installed the last couple of days. x86_64:

Linux xxx 2.6.24-16-generic #1 SMP Thu Apr 10 12:47:45 UTC 2008 x86_64 GNU/Linux

J.O.

[jnygaard (jens-olav-nygaard)]

I think this has to do with problems with ehci-hcd, maybe in connection with particular kernels and or motherboards and or chipsets... not sure.

When I remove this module, the drive works properly, maybe on 1.1-speeds, but at least it works...

J.O.

[sgytis (svgytis)]

This bug is critical - it prevents normal usage of USB HDDs. I tried copying a not so large number of files (~500, total size ~500MB) FROM attached USB HDD (250GB) to computer. After fast copy of initial 40-80MB copying crawls at 0.7MB/s. The bug is not computer dependent - the same happens on Shuttle Pentium4 with SATA disks and Thinkpad T40. The same slowness was observed when copying from USB HDD partitions with fat16, ext3 and NTFS file systems. It is even worse, after some time of copying and stopping it the computer is unable to mount the disk without rebooting. The speed of copying TO USB HDD is much faster however. Both computers have fresh Hardy installed.

[Sebastien Bacher (seb128)]

discussing the settings is not really constructive, that issue is a speed one that only some people get, it's not critical by any mean, it might be annoying but adding random comments about the bug settings here will not make it work automagically, the desktop team is small and has thousand of open bugs so everything can't be worked immediatly, especially if the issue is hardware dependent and an upstream one

[JonB (jonbennison)]

I have the same problem and it is extremely frustrating. Compaq Presario M2350ea laptop.

[zmago (zmago-fluks)]

For me also same problem... i have Acer Travelmate 2300 ...

USB is not useful for large files :(

[RamonBucher (thatvetguy)]

same here with eeepc 701 and ubuntu 8.04...

[Yannick (splitsch)]

Hello !
I have the same problem here.
The files are transfer at a 2Mio/sec. speed. Sometimes, though, it works at the "full" speed of 12Mio/sec.

Here is the output of lsmod | grep usb

[code]
usb_storage 73664 1
libusual 19108 1 usb_storage
lmpcm_usb 7168 0
usbhid 31872 0
hid 38784 1 usbhid
scsi_mod 151436 5 usb_storage,sg,sr_mod,sd_mod,libata
usbcore 146028 7 ehci_hcd,usb_storage,libusual,lmpcm_usb,usbhid,uhci_hcd
[/code]

Is there any way to fix this?

Thanks

[Pascal (spam-trash-ouvaton)]

same problem with Toshiba M60

lsmod|grep usb :
usbnet 20232 2 rndis_host,cdc_ether
usb_storage 73664 2
libusual 19108 1 usb_storage
dvb_usb_nova_t_usb2 6788 0
dvb_usb_dibusb_common 10756 1 dvb_usb_nova_t_usb2
dib3000mc 13960 2 dvb_usb_dibusb_common
dvb_usb 19852 2 dvb_usb_nova_t_usb2,dvb_usb_dibusb_common
dvb_core 81404 1 dvb_usb
i2c_core 24832 5 dvb_pll,mt2060,dib3000mc,dibx000_common,dvb_usb
usbhid 31872 0
hid 38784 1 usbhid
scsi_mod 151436 6 usb_storage,sbp2,sg,sr_mod,sd_mod,libata
mii 6400 3 usbnet,8139too,8139cp
usbcore 146028 11 rndis_host,cdc_ether,usbnet,usb_storage,libusual,dvb_usb_nova_t_usb2,dvb_usb,usbhid,ehci_hcd,uhci_hcd

[Sareen Shah (sareen-eng)]

Confirming here as well, Toshiba A75

not sure what info to provide, here's "lsmod | grep usb" as the above posters did

usb_storage 73664 1
libusual 19108 1 usb_storage
scsi_mod 151436 4 sg,sd_mod,usb_storage,libata
usbcore 146028 6 usb_storage,libusual,wacom,ehci_hcd,ohci_hcd

[Tim Kersten (timkersten)]

I can confirm this happens when using dd'ing a 1G image directly to a usb device.

(warning! this will erase all data on the usb key)
i.e. from terminal 1:

dd if=/dev/zero bs=1000 count=1000000 of=/dev/sdb # where /dev/sdb is your usb key

from terminal 2:
while true; do killall -SIGUSR1 dd; sleep 1; done # this will make dd print status messages ever second

The output is that the transfer speed starts at about 16MB/s and drops to a few KB/s after some minutes.

When running this same command on a fedora box, the transfer speed started at about 5.5MB/s and finished at 4.9MB/s.

[Pedro Villavicencio (pedro)]

thanks for the comment but when a bug is set as Triaged and with an upstream task it means that there's enough info which means no need to re confirm it everytime you want to subscribe to the bug, that will only create a lot of bugmail noise, thanks you all.

[Sebastien Bacher (seb128)]

the new comment is useful in the sense that it seems to indicate that the issue is not a gvfs one

[Neil Burlock (malone)]

I'm going to add a bit more that might be useful because this bug is driving me crazy.

It affects both my systems, one a laptop, the other a brand new desktop I purchased two weeks ago and is happening on both USB and network file transfers (via either Samba or UFS to my NAS box). I've also been experiencing occasional degraded GUI performance if I let a very large file copy (three GB or more) continue. My PC would gradually become un-responsive to the point that mouse clicks and key presses took seconds to be recognised and windows were being redrawn in slow motion. All the while, system monitor showed that both CPU's were nearly idle. When I closed the file copy window, performance returned to normal and there were no dmesg errors.

For anyone experiencing the problem, there's a kind of workaround that some people have had luck with (myself included). Try copying one file at a time and then pausing between each one to let the cache empty out, (since writing appears continue long after the copy dialog has closed). On USB devices in particular, try unmounting and re-mounting between files to make sure the file you are working on is really finished. Obviously, this is only going to work if you are copying a few large files around. If you have lots of small files to move, then zip them up before copying them and you might get lucky.

[Neil Burlock (malone)]

I seem to have happened upon a "solution".

I added pci=routeirq to the grub boot options and I got my USB and network performance back, and the GUI is actually *usable* while copying files too!!!!

I haven't run extensive tests, but I've been able to copy simultaneously over the network and to two USB devices at the same time with the transfer speed remaining constant, near the speed that I've been expecting (for so long), and I've done that repeatedly. That simply was impossible previously.

Over time I checked for this problem on three computers, and found it on all of them. Coincidentally, the three affected computers are all dual core AMD systems no more than 18 months old running various nVidia chipsets from various manufacturers.

This problem needs to be investigated because I can't be so unlucky as to have seen this bug so often and yet have the bug be one that "only some people get". Many more users out there must be experiencing this on a daily basis and thinking that "Ubuntu can't copy files properly", not realising that there's something wrong, and getting more and more frustrated, as I have been.

[Sebastien Bacher (seb128)]

the new comment seems to confirm the bug is not a gvfs one, could be nvidia specific, my machines are intel ones and I don't get the bug

[Evirsama (evirsama)]

Agree with Sebastien. Have a nvidia one with the bug. ubuntu release 7.10.

[Tim Kersten (timkersten)]

I had this on an amd/ati machine, so I don't think it's nvidia specific.

On Thu, Jul 24, 2008 at 12:41 PM, Evirsama <email address hidden> wrote:

> Agree with Sebastien. Have a nvidia one with the bug. ubuntu release
> 7.10.
>
> --
> file transfer on USB disk slows down with gvfs
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>

[giovani (wejick-gmail)]

On Thu, Jul 24, 2008 at 6:55 PM, Tim Kersten <email address hidden> wrote:
> I had this on an amd/ati machine, so I don't think it's nvidia specific.
>
> On Thu, Jul 24, 2008 at 12:41 PM, Evirsama <email address hidden> wrote:
>
>> Agree with Sebastien. Have a nvidia one with the bug. ubuntu release
>> 7.10.
>>
>> --
>> file transfer on USB disk slows down with gvfs
>> https://bugs.launchpad.net/bugs/197762
>> You received this bug notification because you are a direct subscriber
>> of the bug.
>>
>
> --
> file transfer on USB disk slows down with gvfs
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>

i had this on my p4 with SiS 661fx and my Pentium dual core e2xx with
intel 945g. So that clear all X-files about nVDIA specific.

--
ininih GiantiXBraiN kok otak nya gak besar besar
wejick.wordpress.com

[Gabe A (gabextreme)]

I have a Dell Inspiron E1405. This bug is present both when writing to my SD card and to my USB SD card reader. Different cards yield the same results, and it works fine/full-speed on Windows.

For me, this bus is very pronounced. When writing any file over about 100MB, the speed suddenly drops off to under 100KB/sec, where it remains for the rest of the transfer (starting from normal speeds of 15MB/sec).

pci=routeirq does NOT work for me (neither the built-in nor USB card-reader behave correctly after reboot/kernel refresh).

This bug is critical. I've also replicated it on my old Toshiba laptop exactly. Ubuntu Hardy, latest.

[Neil Burlock (malone)]

My pci=routeirq helped, but didn't turn out to be everything I hoped for. I can get away with copying a few GB of files now before the slowdown sets in, but it's still happening. The pattern is slightly different with this tweak however, since it seems to be copying fine for a certain number of files, then it'll start on the next file and the slowdown will start.

If this issue isn't related to gvfs, then shouldn't it be assigned elsewhere? It'd be nice if we could narrow down exactly which package was causing this. My vote at the moment is the kernel because of the improvement I've experienced from changing the boot options, but I'm hardly qualified.

[Tim Kersten (timkersten)]

I agree that this should be assigned to the kernel, as I can't see what
other package could affect write speeds using dd. "dd" is low level, has
nothing to do with gfs. My guess is it's the kernel, or whatever sits
between dd and the kernel. (Is there even something there?)

On Fri, Aug 1, 2008 at 6:36 PM, Neil Burlock <email address hidden> wrote:

> My pci=routeirq helped, but didn't turn out to be everything I hoped
> for. I can get away with copying a few GB of files now before the
> slowdown sets in, but it's still happening. The pattern is slightly
> different with this tweak however, since it seems to be copying fine for
> a certain number of files, then it'll start on the next file and the
> slowdown will start.
>
> If this issue isn't related to gvfs, then shouldn't it be assigned
> elsewhere? It'd be nice if we could narrow down exactly which package
> was causing this. My vote at the moment is the kernel because of the
> improvement I've experienced from changing the boot options, but I'm
> hardly qualified.
>
> --
> file transfer on USB disk slows down with gvfs
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in gvfs: New
> Status in "gvfs" source package in Ubuntu: Triaged
>
> Bug description:
> When transferring large files (800meg) or even medium-size files (200meg),
> the transfer rate decreases constantly. It starts at around 4meg/sec and
> goes down to even less than 800kb/s. When I cancel the download and start it
> over again, the transfer rate comes back up to 4meg/sec.
> I am using Nautilus on Ubuntu 8.04 (hardy heron)
>

[Dale Clarke (dale-clardale-deactivatedaccount)]

This bug is also affecting Firewire hard drives, I am down to 18.5kb/sec and can take a very long time. Tried the solution above but alas nothing.

I am ATI X1550 graphics along with Intel Core duo...

I would class this as critical as it means I can not backup the system to an external disk which makes Ubuntu unusable for me!

[Bugsy (carlo-suomi24-deactivatedaccount)]

I can confirm that this affects also Intel G965 and P35 chipsets whit Lacie USB2 hard-drives. Almost allways the speed is 3-6 MB/s but sometimes it works in full speed 30 MB/s.

I've attached a screenshot where I first transferred one file to the usb2-drive with average speed over 30 MB/s and after 15 minutes when I tried to transfer different file to the drive the transfer speed was only about 3 MB/s.

I think that this is a lot more worse bug than our developers think because making backups is getting too slow.

Also the pci=routeirq did not help for me and this affects both nautilus and cp transfer from command line.

[Savvas Radevic (medigeek)]

Can you try the following:
1) connect your usb device (with your card if it's a card reader)
2) Transfer your files, let it finish, even if it takes ages
3) When you're done, right click on the Desktop icon or the sidebar icon of nautilus and unmount it properly.

post back the output of this command:
dmesg | tail

[Dale Clarke (dale-clardale-deactivatedaccount)]

Update on my problem with Firewire Hard Drive.

I gparted the drive to Ext3 and changed the access from root to User and so far I am hitting 20mb/s and at the moment no stops or access problems. Fingers Crossed.

This seems to make it a NTFS problem with my drive setup.

[Neil Burlock (malone)]

Savvas Radević, there's nothing to post, nothing is added to dmesg other than the usual USB messages when plugging it in. This is the reason why it's attached to gvfs even though nobody is sure what is causing it.

I tried formatting my laptop yet again and stuck on the new Hardy .1 release. It's been more stable, maintaining higher speeds without the boot option. Maybe it's a configuration issue during the install? I'm going to try formatting my desktop machine again to see if I can get any improvement, but first I'll need to do a full backup and that's going to take a while... I'll report back when I get enough spare time to have a go .

[Thomas Mashos (tgm4883)]

I believe I also have this bug on my Inspiron E1505 running Hardy AMD64. This machine has intel graphics and wireless. This slowdown happens to me whether i'm copying to either of my USB drives or over the wired or wireless network.

The wired network didn't seem to slow down until about 600MB if that helps.

[Alex Kon (alkon)]

I confirm this bug on a Shuttle XPC copying to an external hard drive via USB.

Does this problem occur only when copying to a non-linux partition?

[Neil Burlock (malone)]

I never got around to reinstalling. Rather than waste any more time on this, I've moved on to the 8.10 beta. File copying is improved on my system, but it's still not working reliably.

I've been copying some data (about 30GB for each test) from my local USB drive to the my network server and I'm still experiencing a gradual slowdown over time, however it's taking a lot longer to slow down than it did under Hardy, and instead of locking the whole computer up for the duration, only file access is being periodically blocked. Any applications that were open prior to when the copy commenced remain usable.

Overall, the experience is still highly erratic, sometimes a copy works at full speed, sometimes it stalls. It's a lot better than it was though.

[Rhanh-BKK (boythanh)]

Hi.

This problem is also present on my machine, Asus K8N4-E SE, AMD Sempron 2800+. Copying 4 GB of data to an 8 GB USB stick takes around 45 minutes, with Windows (on the same computer) it takes 6 minutes. The data transfer starts very fast until about 75 MB's are completed, then it appears to pause for quite some time (LED in the USB stick does not flash) and then continue for a few MB, pause again and so forth.

This problem was present in kernel 2.6.24-16-generic and is still present in 2.4.24-21-generic. And it's driving me NUTS.

Ubuntu Hardy 8.04.1

Regards

Thanh

[chris larsen (chris-daeda)]

Sorry to add more spam to this list but I at least want to log the issue with my sw/hw configuration.

The way I experienced it was first my guest vm locking my machine almost entirely (i'd get mouse movement after about 3 minutes for about a second then have to wait another 3 minutes) while performing an install on a ubuntu 8.04 ISO image i have on an external USB drive. I decided to copy the image to an SD card in my media slot. I got down to about 700Kb/s then nautilus stopped reporting progress altogether for 5 minutes at a time and then would come back with about 5 mb copied.

Ubuntu Hardy 8.04
Kernel: 2.6.24-21-generic

Hardware:
Intel® Core™2 Duo processor Low Voltage L7400 (4MB of L2 cache, 1.5 GHz, 667MHz FSB)
Intel® 945GM Express chipset
Intel® Graphics Media Accelerator 950

[klulukasz (klulukasz)]

i confirm bug cpu:core duo 1.66ghz

[didwah (peter-dee)]

Unfortunately I also have the same issues and experiences as mentioned in this bug report.

My HW specs as follows:
peter@desktop:~$ lspci
00:00.0 Host bridge: Intel Corporation 82975X Memory Controller Hub (rev c0)
00:01.0 PCI bridge: Intel Corporation 82975X PCI Express Root Port (rev c0)
00:1b.0 Audio device: Intel Corporation 82801G (ICH7 Family) High Definition Audio Controller (rev 01)
00:1c.0 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 1 (rev 01)
00:1c.3 PCI bridge: Intel Corporation 82801G (ICH7 Family) PCI Express Port 4 (rev 01)
00:1c.5 PCI bridge: Intel Corporation 82801GR/GH/GHM (ICH7 Family) PCI Express Port 6 (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #1 (rev 01)
00:1d.1 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #2 (rev 01)
00:1d.2 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #3 (rev 01)
00:1d.3 USB Controller: Intel Corporation 82801G (ICH7 Family) USB UHCI Controller #4 (rev 01)
00:1d.7 USB Controller: Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller (rev 01)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01)
00:1f.2 SATA controller: Intel Corporation 82801GR/GH (ICH7 Family) SATA AHCI Controller (rev 01)
00:1f.3 SMBus: Intel Corporation 82801G (ICH7 Family) SMBus Controller (rev 01)
01:03.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link)
02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE614x SATA II PCI-E controller (rev 01)
03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E Gigabit Ethernet Controller (rev 20)
05:00.0 VGA compatible controller: nVidia Corporation G72 [GeForce 7300 LE] (rev a1)

Intel Dual core Pentium D CPU 2.80GHZ, 8 GB of DDR 667 RAM

Ubuntu 8.04.1 Linux desktop 2.6.24-21-generic #1 SMP Tue Oct 21 23:09:30 UTC 2008 x86_64 GNU/Linux

Hope this information helps.

Cheers

[Vinicius Seixas (vipseixas)]

I want to confirm the problem and add some info that I did not see here:

1 - Problem occurs not only with usb/ehci-hcd, using a internal card reader (sdhci-pci) results in same slowdown
2 - It is not dependent on file-system: same speed with ext3 or vfat
3 - The problem is random, sometimes it is very slow (< 1MB/s) and sometimes just slow (~2 MB/s)
4 - Using pci=routeirq AND noapic seems to increase speed a for smaller files (10MB/s with a 300MB file and 1MB/s with a 700MB file)
    4.1) I tried both boot configs separated and nothing changed
    4.2) I did not repeat tests extensively to be sure this is a consistent or random behavior
5 - No relevant dmesg messages loading modules and during or after transfer

I don't know how those data transfers are ran by the kernel, but it seems to be a very low-level problem.

My system:

$ uname -a
Linux xxx 2.6.27-10-generic #1 SMP Fri Nov 21 19:19:18 UTC 2008 x86_64 GNU/Linux

The hardware is a ATI SB600 for USB and Ricoh R5C822 for card readers.

I can supply any other info you need and if there is a way to debug this, just give me some guidance and I can do this job...

[goto (gotolaunchpad)]

I have the same problem with a P4 2.4, 1 GB RAM, tested on all the onboard (Gigabyte 8PE667 Ultra) USB ports (all of them are 2.0).

Tested loading and unloading ehci-hcd.

Perhaps this happens due some kind of software uptate. I didn't had this problem a month ago.

I have Ubuntu 8.10 and with the previous version I had no problem with USB drives.

Sorry about my bad english.

[goto (gotolaunchpad)]

This problem is too much for me. I use my PC to handle big amounts of data (such as HD video).
I'm going to try another linux distribution. I love Ubuntu but I can't work with 1 MBps transfers.

[Pedro Villavicencio (pedro)]

Please don't change the status from Triaged back to Confirmed, to read about what the status means please have a look to https://wiki.ubuntu.com/Bugs/Status ; thanks in advance.

[Ulrich Lukas (ulrich-lukas)]

I have the issue, too.

Adding the boot option "pci=routeirq" greatly improves the write performance (twice as fast).

Kubuntu Intrepid, AMD64, latest apt-get-upgrades, linux-image-2.6.27-10-generic.

Hardware: Asrock mainboard with NVidia nForce 630a chipset. AMD 64-Bit-Dual-Core-CPU.

I'm attaching the output of lspci -vv and lsusb -v.

HTH. Feel free to ask if you need further info.

[goto (gotolaunchpad)]

> Adding the boot option "pci=routeirq" greatly improves the write performance (twice as fast).
Doesn't this make it look like a kernel issue? Maybe add "linux" to the affected projects?

[Ulrich Lukas (ulrich-lukas)]

I checked again, and I have to take back my previous statement. "pci=routeirq" does not seem to speed up file transfers for my system. (The KDE4 file transfer progress window seems to provide a not very conclusive average speed value)

The reason I posted in this bug report is that I believe this actually is a kernel issue.

I did several measurements copying big files (1.4 GiB Video) with Dolphin, Konqueror, on the command line using "dd" and using cp like follows: "time (cp 1.4GiB-testfile.mkv /media/disk; sync)".
The speed varies between 3MiB/sec and 8 MiB/s.

On another PC with OpenSuse installed, I actually get much higher average transfer rates, between 12 MiB/s and 15 MiB/s.

On the same PC booting MS Windows, I also get around 14 MiB/s.

[goto (gotolaunchpad)]

I've just reinstalled ubuntu 8.10 and i get 24MB/s USB transfers. But without applying the latest software upgrades!!! There are some packets that crashes the USB performance (perhaps the kernel updates?).

My current version:
Linux version 2.6.27-7-generic (buildd@rothera) (gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu10) ) #1 SMP Fri Oct 24 06:42:44 UTC 2008

With my previous installation I didn't notice any change with pci=routeirq

Tested with Fedora 10 Live CD (KDE) and I also get around 20 MB/s

[goto (gotolaunchpad)]

Would be great if we know with which update the problem comes...

[Jimmy Merrild Krag (beruic)]

Have just installed Intrepid and applied the latest software upgrades. With kernel 2.6.27-9 I get bad transfer rates (round 1000 kB/s), but booting 2.6.27-7 I get normal rates (> 20 MB/s).

I think we can conclude it's a kernel issue. Don't you?

[goto (gotolaunchpad)]

Yes, I think so, too. Any idea what to do now?

[Anton Veretenenko (anton.veretenenko)]

Tested transfer rates from partition to partition on 2.6.27-7-generic kernel.
GUI copy starts at 10 Mb/s and then goes down to 6-7 Mb/s and stay.
dd from console is a bit faster, starts at 40+ Mb/s and slowly goes down to 13 Mb/s at the end with 14 Gb file.
The first Gb of data transfers pretty fast with dd.

$ dd if=test.file of=/media/disk/test.file bs=8M
1716+1 records in
1716+1 records out
14396870774 bytes (14 GB) copied, 1087.35 s, 13.2 MB/s
$ uname -r
2.6.27-7-generic

The same as on 2.6.27-11-generic kernel.

[ferriol (ostres)]

I tried with 2.6.27-7-generic and speed remains slow

[Plankware (steve-plankware)]

I've got the same problem. Recently bought a WD 2TB (Mirror edition) USB - copies between 800kb and 4mb (depending on its mood). On XP - it flies

[Plankware (steve-plankware)]

Still a problem on latest

$ uname -a
Linux Sparta 2.6.24-23-generic

[Joey-Elijah Sneddon (joey-elijah)]

Also getting this.

File transfer to a USB Cruzer 2gb and an unbranded SD card reader with SDHC class 4 4GB card: -

First half or so of file transfer is at acceptable speed but rate soon falls to well below acceptable (barely 1MB/s) and if left to continue the GUI can become a bit laggy.

It'd be quicker to burn a 600Mb file to a CD in brasero than transfer via nautilus.

[nicolas314 (nicolas314-deactivatedaccount)]

I just checked on the linux-usb mailing-lists, there is absolutely nothing on that topic. E-mails sent directly to the USB guys come back with a mention like "don't even bother sending us e-mails, go to a newbie web site or contact your distribution bug team".
Dont acte. What are the next steps?

[KennethAar (kenneth-virkelighet)]

This seems to be a duplicate of bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/119730/comments/82 And has nothing to do with usb transferrates but rather sata/harddisk transferrates. Setting "noatime,nodiratime" to the mount options in /etc/fstab solves the problem for me atleast.

[Neil Burlock (malone)]

"noatime, nodirtime" hasn't improved the situation on my PC much, if at all. Copying speed is still all over the place, one time it'll copy fine, the next it stalls every couple of megabytes, blocking file i/o to any other device connected to the system and stalling the gui.

I've noticed that it affects devices differently. The following are listed in order of most likely stall to least likely: network file transfers, my collection of USB flash drives, 1" USB HDD, 3.5" USB HDD. Maybe it's device speed related?

For those trying to escalate the problem, I hit a brick wall last year trying to do the same. I was basically told to give it up because it's going to take a kernel developer who has been afflicted by the bug, and who notices it, to fix it. Well, that, and I don't know the secret handshake that gets things done...

[KennethAar (kenneth-virkelighet)]

I just have to ask: Did you reboot, after setting noatime and nodiratime? a
And did you check if you have a "Performance" HDD option in your BIOS settings and set it to "Bypass"'?

[Neil Burlock (malone)]

Yes, I rebooted and no, I don't have a performance option in the bios.

[Neil Burlock (malone)]

I just noticed that vmware is loading way faster now, down from about 3 minutes to 20 seconds to open a 768MB RAM virtual machine. That's a major improvement.

Copying is still broken though.

[Ulrich Lukas (ulrich-lukas)]

nicolas314 wrote:
> I just checked on the linux-usb mailing-lists, there is absolutely
nothing on that topic.
> [...] What are the next steps?

Well - how about opening a thread then ;-)

Can you do this?

[nicolas314 (nicolas314-deactivatedaccount)]

I'd rather let Ubuntu developers deal with the Linux-USB guys. I do not have enough knowledge or time to track this bug with the (presumed) authors.

[Vinicius Seixas (vipseixas)]

> This seems to be a duplicate of bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/119730/comments/82 And has nothing to do
> with usb transferrates but rather sata/harddisk transferrates. Setting "noatime,nodiratime" to the mount options in /etc/fstab solves the
> problem for me atleast.

noatime & nodiratime will not make any noticeable difference for the transfer of just 1 big file, and a lot of people here reported that symptom... Besides that the problem arises with non-ext volumes too.

I don't know if is just coincidence or luck, but I had this problem with my 64 bits system, I reinstalled Ubuntu and replaced with the 32 bits version and the problem is gone... Now I can achieve stable 10+ MB/s with my pen drive and 20+ MB/s with my usb sata disk.

Just in case it is useful:

$ uname -a
Linux vs-note 2.6.27-10-generic #1 SMP Fri Nov 21 12:00:22 UTC 2008 i686 GNU/Linux

[Neil Burlock (malone)]

I took a chance and reinstalled my system with 32 bit 8.10, as suggested. I've done some quick testing and my USB devices are operating much faster, and the speed remains stable. Additionally, the whole system is *far* more responsive. Previously, I couldn't even use Firefox because it was too slow on certain web pages like Slashdot, with problems like scrolling down an article taking up to a minute due to incredibly sluggish screen redrawing. Previously doing simply things would also max out a single CPU core, however now the CPU is barely being hit unless I do something that is actually CPU intensive.

It's early days and I still need to install everything I use day to day, and do some more testing, but it would certainly seem that this is a bug in the 64 bit kernel that is affecting the whole system, but presenting most obviously as a file copying problem. Maybe it's multi-tasking related?

I'll report back when I've done some more testing.

[chris larsen (chris-daeda)]

I posted above and would like to say that I unmounted the automounted drive and mounted it manually using the mount command from /dev/sdb1 to /mnt/usbdrive and it was blazing (ok comparatively.. 13-18 MB/s and consistent. Beautiful sight. Now I want to create a workaround script to do that automatically and fire up nautilus using that mount point.

I need to test this on my non-laptop computer next and see if it's the same problem as perhaps it's hardware specific?? Currently I am using a tablet pc. Maybe others have laptops as well?

Key Specs:
Motion Computing LE1700 Tablet
Intel® 945GM Express chipset
Intel® Core2 Duo processor Low Voltage L7400 (4MB of L2 cache, 1.5 GHz, 667MHz FSB)
2.6.24-23-generic, Ubuntu 8.04, 32-bit

[Neil Burlock (malone)]

I've finished reinstalling and testing 32bit 8.10.

The problem seems to be greatly diminished. Once I had all my software installed (system updates, Thunderbird, VMWare, restricted extras, nfs driver, nVidia driver) Firefox was showing some symptoms of running slower on Slashdot, but nowhere near as bad as it had been under 64bit - I assume that it's Java related from the restricted extras.

File transfer over the network has improved. I'm finding that it's still hit and miss when it comes to speed, however I have yet to see it gradually slowing down while copying. A file may start slowly (the lowest I've seen in my tests was about 4MB/s, up from 1.8MB/s previously) but over the duration it will remain stable or increase slightly. The first few files (each about 700MB in size) that are copied to my server after mounting the device are typically getting 9MB/s to 11MB/s in bursts.

chris larsen: I have previously found a link between re-mounting a device and resetting it's copying speed, so I'm not surprised you are seeing an improvement there.

[Digital5700 (edward-edwardh)]

Same problem here.

Corsair Voyager GT 16gb (among the fastest USB sticks around)
Linux hitler 2.6.28.2 #3 SMP Sat Jan 31 14:04:46 CET 2009 x86_64 GNU/Linux
4gb memory
Ubuntu 8.10
pci=routeirq has no effect.

# time mount-dd-umount.sh
20mb takes 7 seconds = 2.8mb/s
30mb takes 6 = 5mb/s
100 takes 14 = 7.4mb/s
300 takes 180 = 1.6mb/s
500 takes 241 = 2.0mb/s

[Diaa Sami (diaa.sami)]

I'd like to add my own benchmark on Kubuntu Intrepid, hoping it'd help
I did it on 26/1/09 with a fully upgraded intrepid and I'm running the 32-it server kernel and KDE 4.2 btw.

Copying a 144 MB file to 3 usb devices

My USB1 card reader(pretty old)
automounted: 100 seconds(around 1.5 MB/s)
manually mounted: 96.7 seconds(around 1.6 MB/s)

A slightly newer mp3 player
automounted: 60 seconds(around 2.5 MB/s)
manually mounted: 54 seconds(around 2.8 MB/s)

A newer sandisk 4G flash drive
automounted: 23.7 seconds(around 6.4 MB/s)
manually mounted: 17.9 seconds(around 8.4 MB/s)

Fortunately the difference isn't very big(10-30%)... but there shouldn't be any.

[Digital5700 (edward-edwardh)]

Bus 001 Device 043: ID 152d:2338 JMicron Technology Corp. / JMicron USA Technology Corp. JM20337 Hi-Speed USB to SATA & PATA Combo Bridge
20mb/s write.

Bus 001 Device 040: ID 1b1c:1a90 (Corsair Voyager GT)
1.5mb/s

Differences in attached lsusb -v (pata vs voyager) seem to be...
iConfiguration 4 USB Mass Storage
iConfiguration 0
iInterface 6 MSC Bulk-Only Transfer
iInterface 0

Maybe the voyager is so slow because something isn't registering it a proper mass-storage device?

[Uphaar Agrawalla (uphaar)]

Perhaps an extra piece of info which may be useful - I've recently bought a Western Digital Passport portable hard drive. The USB transfer rates with this drive are good (7-10MB/s).
However the USB flash drive performs as described by others above. The speed decreases after some 70-80 MB is copied and comes down to ~400KB/s.
Running an up-to-date Intrepid.

[kulight (kulight)]

i get the same problem in jaunty 9.04 alpha fully updated

[goto (gotolaunchpad)]

This bug was reported one year ago. It seems that nobody cares about it.
It's very frustating to have a 500GB hard disk with 1MBps (or less) file transfers. The solution? Change the computer or install windows. I've tested FreeBSD, many linux flavours and OpenSolaris with the same results (do they share the USB code?). I thought that the USB was broken so I bought a new one, with the same results.
I'm using a 2.4 GHz computer and it's I/O is slower than a 286. Many users have the same problem with many different configurations and machines. I think that importance should be high. I wold be happy to report logs or anything that would be helpful but nobody cares....

Sorry for that text in my poor english but i'm VERY frustrated and waiting that a 2 Gig file copies in 45 minutes.... crazy.

[goto (gotolaunchpad)]

Another note. After connecting the drive it takes some time to be recongniced and if I do a dmesg I get hundreds of lines like these:

[ 145.184349] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 145.376446] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 145.564288] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 145.752260] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 145.996033] usb 4-3: new high speed USB device using ehci_hcd and address 73
[ 146.052317] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 146.240291] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 146.428259] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 146.616235] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 146.804325] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 146.992309] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 147.240040] usb 4-3: new high speed USB device using ehci_hcd and address 79
[ 147.296241] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 147.484354] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 147.676699] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 147.868289] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 148.064261] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 148.260353] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 148.448325] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 148.700044] usb 4-3: new high speed USB device using ehci_hcd and address 86
[ 148.756253] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 148.944366] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 149.136337] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 149.328307] hub 4-0:1.0: unable to enumerate USB device on port 3
[...]
[ 149.902708] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 150.116296] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 150.304264] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 150.492228] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 150.680322] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 150.868291] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 151.060275] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 151.248367] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 151.440361] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 151.628311] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 151.816276] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 152.004372] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 152.248034] usb 4-3: new high speed USB device using ehci_hcd and address 104
[ 152.304303] hub 4-0:1.0: unable to enumerate USB device on port 3
[ 152.623512] hub 4-0:1.0: unable to enumerate USB device on port 3

It starts flooding that message and finally it gets mounted but painfully slow.

[chris larsen (chris-daeda)]

goto, I posted a reply earlier about mounting the drive through UUID or /dev/sdxx manually instead of letting fuse/gvfs handle it. Have you tried that? my speeds increased tremendously after that. You said you thought "the USB" was broken so you replaced it. That is very vague, what did you replace? My guess is that it is a motherboard/chipset issue. I would try a different system board if manually mounting using the mount command does not work.

[goto (gotolaunchpad)]

Thanks for your response Chris,

I edited the fstab through UUID and /dev/sd* but with the same result. The only difference is that if I reboot, the system wont load because it displays the message "hub 4-0:1.0: unable to enumerate USB device on port 3" forever. The only way to boot is to disconnect the usb drive and reboot again.

I've tested many mount options and tried synchronuos and async.

On the previous post (https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/197762/comments/86) when i said i bought a new USB i meant a new USB Drive.

The normal copy behaviour on my drive is: First 68 MB are copied really fast. Then it gets stalled (around 5 seconds) and then continues coping at 1mbps or less, with some random intermitent stalled periods.

Perhaps there is some kind of problem when detecting the usb drive (with old USB disks (40GB) i have no problems and under windows the USB gets 20mbps, so i don't think it's a hardware issue). I'm not a hardware expert so I don't know where to start. Or perhaps its a communication issue with new external HD USB controlers.

I've downloaded the latest kernel source files to have a look into ehci module and usb drivers. The USB related source files related are unchanged since 1999, or 2003 the newer ones.

More notes:
On source files: linux-2.6.9/drivers/ehci-*.c (unchanged since 2001 or 2003) there are LOTS of "FIXME" notes from the developer.

I will try to compile the module by myself and do some random patches here and there to see what happens (yes, it sounds stupid).

Final note: "goto" is a BugMeNot account. I'm unable to get registered here, tested with different mail accounts and i don't receive the confirmation mail.

[jamesnmandy (jamesnmandy)]

I also have this issue. Just installed Ubuntu 8.10 Interpid Ibex fresh as a standalone OS on the same exact machine which Windows XP, Vista and 7 were all running on. On the afore mentioned operating systems USB transfer speed was normal 2.0 speed, taking maybe a handful of minutes to transfer a 6 to 7 Gb file from local HDD to USB 2.0 HDD or the other way around. With absolutely no other changes other than changing to the latest stable 8.10 Ubuntu release speeds are now gimped to USB 1.0 speed at < or = 1 megabyte per second. The same exact 7Gb file now takes approximately 2 hours roughly to transfer. Same hardware, same cable, same USB port, only change was Windows to Ubuntu 8.10. Please fix asap. I use USB transfers for large files on a daily basis and this is seriously ruining what has otherwise been a very positive experience (finally) with Linux after a few years of trying several distros over and over. Thanks.

[jamesnmandy (jamesnmandy)]

"i get the same problem in jaunty 9.04 alpha fully updated"
-kulight

that is very discouraging news, it seems incomprehensible that something used by nearly everyone every day would remain gimped after so long, they can stop with all the "new pretty features" they are boasting about in Jaunty and fix core system issues first, i like the looks of Intrepid Ibex just fine, but if my USB support remains neutered it wont matter what Ubuntu looks like, i will onyl be seeing screenshots of it randomly on the internet, it wont be on my PC this way for long....which sucks because other than this i like it so well

[Leann Ogasawara (leannogasawara)]

Hi Everone,

I'd like to try to gather a bit more debugging information for the kernel team to review. I realize that even though all of you may be experiencing the same symptoms reported here, it really may be hardware specific. It's helpful to the kernel team if bug reports target one specific issue against a specific set of hardware. We can easily mark bugs as duplicates later on if necessary. With that being said, this bug has grown quite long and I think it would be more beneficial to the kernel team if we can break this into separate more concise bug reports.

First, for those of you who specifically mentioned this is a regression after updating to 2.6.27-9 from 2.6.27-7, if one of you could please open a new bug report and then anyone else who can confirm this regression subscribe to that new bug that would be great.

Next, I'd like to get everyone testing against the latest kernel available. The latest pre-release of Jaunty 9.04 (currently Alpha4) contains a 2.6.28 based kernel. If would be great if everyone could test and confirm this issue with the latest kernel - http://cdimage.ubuntu.com/releases/jaunty/ . If you could then attach the following information, making sure to attach each file separately:

* cat /proc/version_signature > version.log
* dmesg > dmesg.log
* sudo lspci -vvnn > lspci-vvnn.log

It would also be best to try to wait and capture your dmesg output after you're reproduced this bug just in case any additional error messages may get logged. I realize some of you mentioned you saw no additional error messages, but please still attach your dmesg output just the same so we can compare. We can then focus on grouping those with the same hardware or similar dmesg errors into separate bug reports.

We really appreciate your cooperation with testing and gathering information. Please let us know your results. Thanks in advance.

[jamesnmandy (jamesnmandy)]

thank you Leann, i will have to look into updating to 9.04 Alpha4 and providing the information provided, as i have time

[kulight (kulight)]

if you do as Leann Ogasawara suggested wich seems like a good idea
please post a link of the new bug here so we will know and not open new ones

ill start the jaunty bug:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/334914

[Diaa Sami (diaa.sami)]

well, I want to help, I did some experiments with Jaunty alpha4 and I get slowdowns when writing only, should I start a new bug or use the one already opened?

[Digital5700 (edward-edwardh)]

Leann Ogasawara: Here are your requested log files.

Hold this helps somewhat.

[kulight (kulight)]

" well, I want to help, I did some experiments with Jaunty alpha4 and I get slowdowns when writing only, should I start a new bug or use the one already opened? "

i think its safe to assume its the same bug as im experiencing

[Ram Kumar (ramkumail)]

I confirm this on amd64 machine with ubuntu 8.10 with latest updates installed.
It is really disappointing to see a bug this critical has not been fixed for this long a time. My disappointment apart, if somebody an find a work around for this, it will be greatly useful.

[curtis.lysher (curtis-lysher)]

I have been searching for two days for why I am having this same problem. I have two systems, both running intrepid from the same install disc. The system with the problem is a Vostro 1000 laptop running Intrepid with 2.6.27-11. The other system is an Acer Aspire5641 desktop running the same kernel. The usb drives were both formatted with gparted on the laptop and owned by my local user account. The Acer desktop has no problems whatsoever, getting around 35MBps transfers on any usb drive. Those same drives start at about 4MBps on the laptop then drop to 600kBps. I noticed this problem because my 120gb external maxtor is what I use for movies, and my movies started skipping and then the system would become highly unstable. My laptop is almost never online as my desktop has most of the storage, so I just transfer what I want with my usb hdd. However, I do plug it in weekly and grab all updates through the update manager. The laptop played 720p movies fine, and now cannot even playback a 320x240 video without locking up. When this happens my internal hdd goes crazy, and the system slows so badly that my cursor barely moves. It is very obvious thanks to google that this is not a hardware specific issue, many people are having issues with all sorts of hardware devices, including usb controllers and different drives.

[Diaa Sami (diaa.sami)]

> I confirm this on amd64 machine with ubuntu 8.10 with latest updates installed.
> It is really disappointing to see a bug this critical has not been fixed for this long
> a time. My disappointment apart, if somebody an find a work around for this, it
> will be greatly useful.

Mounting drives manually fixes this problem for me, it's tedious I know but fixes the problem.

To mount a drive manually, identify its name by executing
sudo sfdisk -ls -uM
which lists all drives connected, their sizes and types, the execute a command similar to the following
sudo mount <device_name> <empty_folder>
where <device_name> is the device name you identified using the previous command and <empty_folder> is the path of an empty folder where the drive will be mounted, if it's NTFS, I think you'll need to use 'mount.ntfs-3g' instead of 'mount'

Example:
$ sudo sfdisk -ls -uM
/dev/sda: 244198584

Disk /dev/sda: 30401 cylinders, 255 heads, 63 sectors/track
Units = mebibytes of 1048576 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start End MiB #blocks Id System
/dev/sda1 * 0+ 12291- 12292- 12586896 7 HPFS/NTFS
/dev/sda2 17602+ 120001- 102399- 104856255 7 HPFS/NTFS
/dev/sda3 120001+ 238472- 118472- 121314847+ f W95 Ext'd (LBA)
/dev/sda4 0 - 0 0 0 Empty
/dev/sda5 120001+ 140482- 20482- 20972826 83 Linux
/dev/sda6 140482+ 191681- 51200- 52428096 83 Linux
/dev/sda7 237445+ 238472- 1028- 1052226 82 Linux swap / Solaris
/dev/sda8 191972+ 227812- 35841- 36700461 7 HPFS/NTFS
/dev/sdb: 625131864

so I pick /dev/sda6 and create a folder /mnt/tmp then execute
sudo mount /dev/sda6 /mnt/tmp

note: Before removing the device you need to execute
sudo umount <device_name>
or
sudo umount <folder_name>

You can find the official longer guide at https://help.ubuntu.com/community/Mount/USB

[L-Train (so1dieroffortune)]

This bug it not hardware specific. I got 4 different PCs here that have been purchased over the last 5 years. 2 Dells, 1 HP, and 1 Gateway. Some have single processors, some have dual core, some are intel some are AMD, some have nvidia some have ati, some have integrated intel video cards.

All 4 experience this bug! The transfer speed on my newest dual core intel desktop (Dell that came with Ubuntu pre-installed) drops down to 200 Kb/S pretty quickly. This is after the first 60 MB is copied in 3 seconds! and then the speed slowly drops to 200 KB, and it takes hours to copy files that are several gigs in size! Rebooting my PC sometimes (but rarely) helps. If I'm lucky, after the reboot, for the first 2 GB the speed would be 15-20 MB/S, but then it would drop down to 200 KB/s. This is if I'm lucky. If I'm not lucky the fast speed lasts only for the first 50-100 MB, and then its the wait game all over again. 90% of the time - I'm not lucky.

I am a software developer, and a math guy. Statistics and experience tells me that this bug is experienced by majority of people out there (What are the odds that I have 4 different machines purchased over the years with different hardware from different manufacturers and ALL of them experience the same "rare" bug?). The bug is probably severely underreported because most people don't copy huge files over the USB 2.0. I didn't have a need to do this until now when i decided to start backing up my video/music/high-res pictures. If I only used my USB drive to copy 50 MB or smaller files every now and then - i would've never noticed the performance issue.

[jamesnmandy (jamesnmandy)]

i have to say if it is hardware specific, it specifically affects all sorts of hardware......seeing as it is a software bug

[Leann Ogasawara (leannogasawara)]

While I understand everyone's frustration here, please try to remain focused on providing the requested information - see https://bugs.edge.launchpad.net/ubuntu/+bug/197762/comments/92 .

"Next, I'd like to get everyone testing against the latest kernel available. The latest pre-release of Jaunty 9.04 (currently Alpha4) contains a 2.6.28 based kernel. If would be great if everyone could test and confirm this issue with the latest kernel - http://cdimage.ubuntu.com/releases/jaunty/ . If you could then attach the following information, making sure to attach each file separately:

* cat /proc/version_signature > version.log
* dmesg > dmesg.log
* sudo lspci -vvnn > lspci-vvnn.log"

It probably wouldn't hurt to also attach the output of `sudu lsusb -v > lsusb-v.log`.

@Digital5700 - could you attach each file separately rather than zipped up?

@kulight - I'll follow up with you at your new bug. Thanks.

[netframe (tmcginty)]

Reviewing jarlath's files, very similar to mine on 2.6.28 kernel, Intel proc, Asus P5P800 MB with USB 2.0 connected external SATA drive. Same diminishing transfer rate for 1.4GB test transfer. Coming from 8.10 2.6.27-11.27-generic kernel. What were we to expect in this .28 build that could be different (other than a flat speedy transfer rate)?

[Sebastien Bacher (seb128)]

the issue seems to be a linux one rather than a GNOME bug

[Jeff Fortin Tam (kiddo)]

I doubt that. How come a command-line copy would go much faster than using gvfs then? At least that's how the original bug was reported.

Top that with the hundred comments in various directions saying that $FOO kernel version/option doesn't make a difference, and that this is replicable on dozens of different hardware configurations, it seems pretty dangerous to me to throw away the gnome thesis. Any KDE users affected?

[Sebastien Bacher (seb128)]

reading the bug users have similar issues using the command line to do copies

[moron (slave-codegrunt)]

Jean-François, I am a KDE user and I am definitely affected by this issue.

[Jeff Fortin Tam (kiddo)]

Alright, then it eases my mind. I had kinda lost track of this issue
over time with the >100 comments, I guess. Thanks!

[Ulrich Lukas (ulrich-lukas)]

I did a bunch of tests. First of all, I'm attaching the output of:

cat /proc/version_signature
dmesg
sudo lspci -vvnn
sudo lsusb -vv

I used Ubuntu Jaunty kernel "2.6.28-9-generic", except for the log of "lsusb", which was acquired when the system was running under 2.6.29-rc7, vanilla, from kernel.org.

The system is the latest Kubuntu Jaunty alpha with updates as of 10 March 2009; architecture: AMD64.
I did the tests running KDE 4.2.1-0ubuntu5 and used the automatic "pop-up" feature to mount a 16-GiB USB pendrive specified for a minimum throughput of approx. 14 MiB/sec.

These are my observations:

Copying large files (i.e. approx. 8 GiB) from high-performance harddisk to the USB drive using
_Microsoft Windows XP SP2_, the above-all average transfer speed is 13...14 MiB/sec. This is always reproducible.

Using my Ubuntu system:
When _copying large files using KDE_, after the first 3GiB or so, the transfer rate drops from the initial 14 MiB/sec to an average of about 5.5 MiB/sec. The indicated speed on the progress window fluctuates between a few kiB/sec and full speed.

Using the _command line_ and timing the following commands using a shellscript yields /slightly higher/ average transfer speeds, approx. 7 MiB/sec.

Ubuntu kernel 2.6.28-9-generic and a custom 2.6.29-rc7: No difference.

Conclusion: Using Ubuntu, the transfer speed is always less than half the speed than under MS Windows XP.

I did at least 20 tests with different file sizes, file systems on the pendrive (VFAT, EXT2, fuse NTFS3G NTFS)

[Ulrich Lukas (ulrich-lukas)]

This is an additional time invoking "dmesg" a few minutes later; the difference (one line at the bottom) does not always appear and I'm not sure if this is correlated to the speed problem.

[Ulrich Lukas (ulrich-lukas)]

This method I used for obtaining average transfer speeds on the command line.

The USB pendrive was still mounted via KDE4's "pop-up" mounting feature:

On the command line:
$ dd if=/dev/urandom of=/tmp/7-GiB-testfile bs=7M count=1024
$ sync
$ time ./test.sh

The shell script "test.sh" used:
$ #!/bin/sh
$ cp /tmp/7-GiB-testfile /media/disk/
$ sync

[Night64 (roberto-berlim)]

Using Linux version 2.6.28-9-generic (buildd@crested) (gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu2) ) #31-Ubuntu SMP Wed Mar 11 15:43:49 UTC 2009 I have the same results as Ulrich Lukas above.

[Bogdan Gribincea (bogdan-gribincea)]

I tried the vanilla 2.6.29 kernel from http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.29/ and the issue still exists. I manually mounted the USB drive.

I'm using Jaunty Beta 1.

[Sebastien Bacher (seb128)]

the issue is rather a linux one

[Ulrich Lukas (ulrich-lukas)]

Has this issue been reported upstream (i.e. LKML)?
I couldn't find a thread on gmane, but maybe I'm just too blind.

[BukvA (bukva)]

i've had the same here: MSI motherboard, AMD cpu with Nvidia chipset, but this post was helpful http://ubuntuforums.org/showpost.php?p=6992097&postcount=4

[BukvA (bukva)]

oops, i meant that turning off legacy support was helpful for me, sorry, but that's true - i'm not able to boot from usb stick though.

[Pavol Klačanský (pavolzetor-deactivatedaccount)]

Same problem here, I have Thinkpad T500, starting speed was about 15 MB/s and at the end only about 3 MB/s, size of transfered file is 3,2 GB

dist: jaunty 64 bit

[Kurt De Smet (nihilus-dommel)]

I had the same problem, needed to backup 200+ GB to a USB HD due to a hardware upgrade. Then I remembered the board had a strange bios reset recently (hence the upgrade) so I checked and set the bios SATA configuration back from IDE to AHCI (My mobo has 3 possible settings IDE, RAID and AHCI). Now it's supper silent (no more swapping to disk) and a steady transfer rate (+/- 27 MB/sec).

This is the solution that helped me, and hopefully some others.

[Ulrich Lukas (ulrich-lukas)]

Hi Kurt,

albeit the AHCI-mode enables the use of Native Command Queueing (NCQ)
for some mainboards, I somehow doubt that this has to do with the USB
issue. Especially because of the "swapping" you reported which IMO has
nothing to do with the USB speed issues reported before.

[Kurt De Smet (nihilus-dommel)]

Hi Lucas,

I noticed everything goes fine until the memory gets full, then starts the swapping and the slowdown. It's like the memory is used as buffer, when that runs out it uses the virtual memory on the HD. No problem with small sizes that fit the memory but when you select 100 GB at once ... then the HD starts writing to itself fairly rapidly. When AHCI was enabled the memory gets used to but strangely enough it does not use the virtual memory (NCQ?). As you stated the NCQ may prevents the HD from playing with itself. It may not be the main problem (that's why I didn't file a bug) but I'm fairly sure some people have IDE selected or only have IDE in their BIOS. I only thought it was worth mentioning how I solved my problem because I read very similar stories to mine here. It could be good to know if they are using a PATA or SATA drive as source. Once you filter those out you can narrow down to the real problem.

[psypher (psypher246)]

I wil have to disagree with about ram. busy trying to clone my root partition with dd to a usb hard drive. booted with jaunty 32bit live cd and started the process with dd if=/dev/sda3 of=/media/disk/image.img conv=sync,noerror bs=64K. 1st 10GB flew by at 25-30MB/s then started slowing down to 10MB/s till about 25GB and then crawls at 3MB/s. stopped it around 53GB after about 5 hours and tried turning off USB legacy emulation in the BIOS as suggested by some other posts. started the process again and the EXACT same thing happened. ram usage is not a factor at all:

                          total used free shared buffers cached
Mem: 3095648 1243452 1852196 0 763720 264116
-/+ buffers/cache: 215616 2880032
Swap: 1638588 0 1638588

rebooted to switch the sata hard drive to ahci. could not boot ubuntu at all so thats not a workaround. had the same issue on my desktop. not sure how you are supposed to get any ubuntu booted in ahci mode.

also tried manually mounting the usb drive and also made no difference. in fact i have tried this when i was still running hardy and it made no difference then. so as far as I am concerned there is no workaround for this problem at all. started the clone last night at 11:30 and is still running now at 9:30 and is only on 63GB. shocking.

This issue has been plaguing us since hardy and is frankly embarrassing for OSS and Ubuntu.
This is not a medium bug IMO but should be marked as CRITICAL. Need to do emergency clone of a hard drive you cannot expect someone to wait 12 hours for something that should take 1 or 2 This bug was logged a year ago. what can be done to fix it finally???

[psypher (psypher246)]

backup is now at 64GB and has slowed down to 500KB/s and I will have to stop it.

Usb is now officially a completely useless backup tool on linux.

just stopped it, dd claims 65GB has been copied avg 1.5MB/s

like to mention that this has always been a very intermittent issue. sometimes works fine mostly does not. doing this dd image has been the most reliable way to reproduce the problem.

[Pinguin (riebel)]

Hello,

this bug isn't ubuntu specific I think.

I can reproduce this problem with Debian 3.1/4.0/5.0 and linux 2.6.29 too.
Can you try it with Ubuntu 6.06 again? There doesn't affects me this problem...

[Neil Burlock (malone)]

No, it's not Ubuntu specific and Canonical didn't cause it.

Right now I'm copying between two SATA drives, connected to my PC via eSATA. So far a lousy 600MB has been copied and I'm already down to 5MB/s and falling.

Here is what I know about the problem:

It affects ALL hardware.

It affects ALL file copying connections to varying degrees: USB, network, SATA/eSATA.

In some instances, it also affects multitasking, all but locking the computer up despite little to no CPU use in System Monitor.

There are no error logs, messages or anything else that occur when the problem is happening.

In my experience, if you test for the problem and nothing seems to be wrong, then that's because you aren't copying enough data, you don't know how fast your hardware should be copying files, or you are lucky enough to only be subtly affected that time. Try copying more multi-gigabytes of data, several times and see if you can't get wildly different durations for the same data size. As I state above, all hardware I have tested is affected to varying degrees, some hardware shows minor performance degradation over time, some hardware is all but useless for file copying.

If you ask me, it's either the scheduler (GUI lockups with no CPU usage makes me think this) or whatever performs the actual file copy process. It's definitely not a USB problem because it affects network copying too.

[Theodore Ts'o (tytso)]

For people who are seeing this --- try booting the kernel with the "mem=" command line option and see if this makes a difference. For example if you have 4 gigs of memory, try using "mem=1G"; if you have a gig of memory, try using "mem=512M". Does this change how big the file needs to be before you start seeing the slowdown?

[Theodore Ts'o (tytso)]

Something else to try; I would suggest using "dstat" with the -D option (i.e, dstat -D sda,sdb) or iostat 1 (although I find dstat to be a bit more user friendly) to measure I/O transfer rates. I would recommend using "dd if=/dev/zero of=/media/disk/test.img bs=32k" to measure the transfer rates, and I would *not* recommend using a flash based device because they can show a lot of variance all by themselves. So please, use some kind of HDD.

Speaking personally, I've been doing a rather huge amounts of copying of files back and forth between disks, using ext4, in the 30-40 gigabyte range, and I've not noticed anything like this. This could be filesystem related, or related to the page writeback algorithms, and so something else to try would be to try dd'ing to a raw disk, and see if you see the slowdown. For me, using a 2.6.30-rc5 and the ext4 filesystem, I've done many, *many* copies of data using "rsync -axH" to copy over entire filesystems (for doing things like testing fsck speedups between ext4 and ext3) and I've noticed a problem. I normally have a window open running "dstat -D sda,sdb", so I can get dynamic output like this once a second:

----total-cpu-usage---- --dsk/sdb-----dsk/sdc-- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ: read writ| recv send| in out | int csw
 19 26 42 11 0 2| 0 16M: 15M 0 | 0 0 | 0 0 |2081 9176
 20 24 41 13 0 2| 0 16M: 15M 0 | 0 0 | 0 0 |2092 9287
 18 24 42 14 0 1| 0 12M: 14M 0 | 0 0 | 0 0 |2003 8987

Note that filesystem activity can make a huge difference. When copying files around, there's enough seeks going on that in practice I can only read or write 16 megabytes/second. If I do a dd if=/dev/zero of=/scratch/zero.img bs=32k", I get this:

----total-cpu-usage---- --dsk/sdb-- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
  3 10 57 28 0 1| 0 57M| 0 0 | 0 0 | 746 1268
  4 27 43 25 0 2|4096B 58M| 0 0 | 0 0 |1065 951
  4 21 44 30 0 1| 0 60M| 0 0 | 0 0 | 701 396
  5 23 50 20 0 2| 0 61M| 0 0 | 0 0 | 718 288

Here's what I get if I do "dd if=/dev/zero of=/dev/closure/scratch bs=32k"

----total-cpu-usage---- --dsk/sdb-- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
 11 13 47 27 0 1| 0 61M| 0 0 | 0 0 | 740 1327
  3 15 49 32 0 1| 0 61M| 0 0 | 0 0 | 522 249
  2 10 50 37 0 1| 0 61M| 0 0 | 0 0 | 476 246
  6 14 49 30 0 1| 0 61M| 0 0 | 0 0 | 544 260
  3 12 49 35 0 1| 0 61M| 60B 60B| 0 0 | 558 260

And here's what I get if I format /dev/closure/scratch as ext3 instead of ext4, and then do "dd if=/dev/zero of=/scratch/zero.img bs=32k":

usr sys idl wai hiq siq| read writ| recv send| in out | int csw
  5 19 17 57 0 1|4096B 47M| 0 0 | 0 0 | 637 503
  6 15 0 77 0 1| 0 52M| 0 0 | 0 ...

[Neil Burlock (malone)]

Theodore, thanks for the info.

I don't have a lot of time this morning before work, but I was able to run one test which illustrates the problem. In the attached dstat output, I copied 3.4GB between two eSATA HDD. It started off fast (as usual), but by the time the copy finished the dialog was showing about 5.5MB/s.

I'll look into it more later today.

[Neil Burlock (malone)]

Forgot to mention, that dstat is for the destination drive only. I'm also not seeing as much of a slowdown if I copy from the source to the internal drive, then to the destination drive.

[Neil Burlock (malone)]

I did have a long list of tests to attach here, but using dstat over the last couple of hours it appears that one of my drives that is connected via eSATA might be faulty. Until last week when I purchased a new 640GB I was using it (WD 500GB ext4) as my desktop drive and prior to that as my server drive.

I got the following from a transfer test of the 500GB WD drive:

2906193920 bytes (2.9 GB) copied, 62.6089 s, 46.4 MB/s

Which seems OK. However in my tests I've found that the eSATA slowdown is occurring only when reading from this particular drive, not when writing to it. I've tested several other WD drives, none of which are showing any slowdowns in reading or writing over eSATA. For now, eSATA is off the hook as being involved in this problem and I will dispose of the 500GB drive once I've got the rest of my data off it.

I'll begin testing USB and network transfers tomorrow. Potentially, this drive could have been involved in a lot of the performance problems since much of my copying is from flash to desktop to server and back in the reverse order. I'll need to do more testing.

In the meantime, I can't thank you enough Theodore Ts'o for explaining that performance test and dstat. I wish someone had suggested it a year ago...

I've attached a copy of the dstat from reading the 500GB drive for reference.

[Ulrich Lukas (ulrich-lukas)]

System running Ubuntu Jaunty AMD64, vanilla 2.6.29.1. KDE4.2, two terminal windows, no other graphical applications.

I created two test files of identical size with random data and copied to the USB pendrive, once on a freshly rebooted system, and once after I first experienced a slowdown while copying some other files to the pendrive and deleting them again; doing a "sync" after each copy.
A second terminal window ran dstat -D sda,sdb both times.

( Preparation:
dd if=/dev/urandom of=/var/tmp/850-MiB-testfile.A bs=1M count=850
dd if=/dev/urandom of=/var/tmp/850-MiB-testfile.B bs=1M count=850
reboot )

First run after reboot:
(/dev/sdb1 mounted on /media/pendrive)
cp /var/tmp/850-MiB-testfile.A /media/pendrive

Second run after the first slowdowns were observed during copying of other files and deleting them again:
cp /var/tmp/850-MiB-testfile.B /media/pendrive

The output results of "dstat -D sda,sdb" are attached.

First run, approx. 16 MiB/sec constant: dstat-D.OK.850-MiB-random-data.txt

Second run with slowdown: dstat-D.slow.850-MiB-random-data.txt

I'm sorry, I couldn't figure out what causes the slowdown after a while.

I'm doing some more tests tomorrow, with other kernel versions
and with a borrowed USB harddisk.

[Theodore Ts'o (tytso)]

Ulrich --- if you have time to try to do more tests, may I gently suggest that you start a new Launchpad bug? Drop a pointer from in this launchpad bug number to the new one, but let's do all of the experimentation on a new launchpad bug. There are a couple of reasons for this:

(1) Launchpad has horrific scalability problems, and it's painfully slow to find new entries on this Launchpad entry, since it has so many comments already.

(2) It's not clear everyone on this bug is sufferring from the same bug. They may have similar symptoms, but that doesn't mean it is the same bug. It's actually **highly** confusing to have many people glomming on with a "me, too!!" comment, giving a very tiny amount of information, but not actually giving a full description of what they are seeing. In fact, the tiny amounts of data may be horribly misleading, because they may be seeing a different bug.

(3) As a result, many kernel developers on LKML don't spend a lot of time on Launchpad bugs; the bug reporting is ___so___ ___bad___ that it's just too frustrating and time-consuming for them. And unfortunately, Canonical has a fairly limited kernel team, and it takes them a lot of time try to synthesize a good bug report from a lousy one. In fact, someone recently posted this bug report on LKML, and what it mostly got was a vent from someone complaining how useless most Ubuntu bugs were. (Well, it also got me interested enough to actually take a look, but I've gotten inured to how lousy Ubuntu bug reporting infrastructure is for large bugs, as well as mostly uninformative bug reports.)

So if we do have one or more people who have the time and energy to do some real bug reporting (which takes real work), may suggest that each person open a separate bug, and each bug, give (a) full details about what kernel version, including whether it is a Ubuntu kernel or a mainline kernel, (b) full details about your hardware version, including which disks are hooked up which way (i.e., eSATA, USB, SATA, PATA, etc.), and (c) exhaustive details about how you ran your test case, whether or not is repeatable, etc. Also note that it is an instance of Launchpad bug #197762, and that anybody else who wants to glom on with a "me, too" is kindly asked to put that remark in the #197762 cesspit^H^H^H^H^H^H^H^Htop-level bug, and not in the new bug report, which we can hopefully keep specific to your situation.

I would again suggest that you try doing multiple tests:

1) Copying one file from one disk to another
2) Copying from /dev/zero to file on the disk
3) Copying from /dev/zero to the raw disk device (yes, this will blow away your filesystem; hopefully you have scratch devices)

The ideal bug reporter will have multiple devices that can be attached via different hardware interfaces (i.e., eSATA, USB, maybe a spare SATA port), so we can try to rule out hardware attachment problems.

If someone is willing to do all of that, I'm more than willing to try to drill down as far as I can; although if it turns out to be a USB problem, I'll need to call in help from a USB developer ---- but with a well constructed bug report, that shouldn't be a problem; I should be ...

[Robert Fekete (fekete77-robert)]

Hi, I am experiencing a similar problem on a fresh jaunty install when copying files to an USB hdd. I do not even need to
transfer large files, transferring around a few mp3-s kills the transfer rate in a few seconds.

Theodore-- I will try to get some time and start collecting the info you requested in your previous comment.

[Simon Holm (odie-cs)]

So, I believe I am the one Theodore refered to as venting how the useless Ubuntu bug reports are. While they really are, I also really want to help.

So, following Theodore's suggestion I've started https://help.ubuntu.com/community/DiskPerformance on the Ubuntu Wiki. It is pretty basic right now, mostly repeating what Theodore already wrote here. Let's follow this from here (or rather in separate bug reports as Leann and Theodore have already advocated) and expand it according to common patterns of the reports.

[MVDHR (movedhere)]

I can reproduce on my Ubuntu 9.04 64 bit so far.

[Simon Holm (odie-cs)]

MVDHR,

and why should we care? This bug has been open for a year and 50 people have commented here, so news would be that the problem actually went away. You give no information that helps further debug the issue, please read https://help.ubuntu.com/community/DiskPerformance and diagnose your problem according to the tips given there before posting anything further. And please, please, please open a new bug once you've followed through with that page since it actively hurts when people hi-jacks other peoples bugs for what might be a completely different issue.

[Night64 (roberto-berlim)]

Testing with a Kingston DataTraveler on 2.6.27-14-generic x86_64 Ubuntu 9.04.

copying from /dev/zero

roberto@roberto-desktop:~$ dd if=/dev/zero of=/media/KINGSTON16/test-file bs=512
1038321+0 registros entrando
1038321+0 registros saindo
531620352 byte (532 MB) copiados, 18,2775 s, 29,1 MB/s

A few minutes later

3275058+0 registros entrando
3275058+0 registros saindo
1676829696 byte (1,7 GB) copiados, 265,338 s, 6,3 MB/s

Later, speed consistently droping.

3856803+0 registros entrando
3856803+0 registros saindo
1974683136 byte (2,0 GB) copiados, 402,468 s, 4,9 MB/s

5799921+0 registros entrando
5799921+0 registros saindo
2969559552 byte (3,0 GB) copiados, 1029,97 s, 2,9 MB/s

So far, I had the same results from everybody. But yesterday a friend used his iPod to copy 3.9 GiBi of data from my hard drive... And was blazing fast! I didn't have the time at the moment to do tests with his hardware, but later I noticed that the iPod had two different lines in dmesg:
[81601.496620] usbcore: registered new interface driver usb-storage
[81601.496716] USB Mass Storage support registered.

[Theodore Ts'o (tytso)]

Night64:

No, you're using usb-storage device driver for both the pendrive and the ipod. The messages that you quoted as coming from the ipod just means that the usb-storage module had gotten unloaded, and when it was loaded as a module, it prints those lines to the kernel log. The fact that things were fast going to the ipod and variable when you write to the your USB flash device tends to indicate that the problem is specific to the flash device; it might just be that the flash device is sometimes slower because writing to flash can be slower sometimes, and it might be exacerbated by the FAT filesystem, which is never known as a speed demon.

Note that this might not be true for other users on this bug report --- which is why I've suggested everyone file separate bug reports. People using Ubuntu 8.10 might possibly have been using the Low Performance USB storage driver, which Canonical inexplicably configured into older Ubuntu kernels. Anyone using that will definitely see massive performance problems --- and this has been known for ages. (Another reason why many kernel developers aren't happy trying to clear up Ubuntu-specific bugs....) In any case, this kernel configuration bug doesn't seem to be a problem for Ubuntu 9.04; libusual.ko doesn't appear to be built any more, which is a Very Good Thing. Some users from a year ago who might have been using the low performance USB driver might have a very different problem than yours --- which is why it's better for each user to open separate bug reports.

Remember that most of the people with the experience and knowledge to help are volunteers. If Ubuntu users continue to make life difficult, kernel developers will simply stop volunteering to wade through the cesspit of Ubuntu launchpad bugs....

[Simon Holm (odie-cs)]

Night64,

flash devices are slow and the only reason the transfer appears to be fast at the beginning is that data is written to OS memory and then later to disk. Use dstat to monitor disk performance as explained on https://help.ubuntu.com/community/DiskPerformance, And open new bug reports please (though not in this case since your device is just slow, unless you can demonstrate better performance using Windows everything is as expected).

[Night64 (roberto-berlim)]

Thanks for the info, Ts'o. But your comment about libusual caught my eye. In
Ubuntu 9.04 this module still shows up, right?

[81601.434729] usbcore: registered new interface driver libusual
----------------------------
Linux: Because rebooting is for adding new hardware.
Sent from Brasilia, DF, Brazil

On Wed, May 20, 2009 at 5:32 PM, Theodore Ts'o <email address hidden> wrote:

> Night64:
>
> No, you're using usb-storage device driver for both the pendrive and the
> ipod. The messages that you quoted as coming from the ipod just means
> that the usb-storage module had gotten unloaded, and when it was loaded
> as a module, it prints those lines to the kernel log. The fact that
> things were fast going to the ipod and variable when you write to the
> your USB flash device tends to indicate that the problem is specific to
> the flash device; it might just be that the flash device is sometimes
> slower because writing to flash can be slower sometimes, and it might
> be exacerbated by the FAT filesystem, which is never known as a speed
> demon.
>
> Note that this might not be true for other users on this bug report ---
> which is why I've suggested everyone file separate bug reports. People
> using Ubuntu 8.10 might possibly have been using the Low Performance USB
> storage driver, which Canonical inexplicably configured into older
> Ubuntu kernels. Anyone using that will definitely see massive
> performance problems --- and this has been known for ages. (Another
> reason why many kernel developers aren't happy trying to clear up
> Ubuntu-specific bugs....) In any case, this kernel configuration bug
> doesn't seem to be a problem for Ubuntu 9.04; libusual.ko doesn't appear
> to be built any more, which is a Very Good Thing. Some users from a
> year ago who might have been using the low performance USB driver might
> have a very different problem than yours --- which is why it's better
> for each user to open separate bug reports.
>
> Remember that most of the people with the experience and knowledge to
> help are volunteers. If Ubuntu users continue to make life difficult,
> kernel developers will simply stop volunteering to wade through the
> cesspit of Ubuntu launchpad bugs....
>
> --
> file transfers on USB disk are very slow
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in gvfs: Invalid
> Status in “gvfs” source package in Ubuntu: Invalid
> Status in “linux” source package in Ubuntu: Confirmed
>
> Bug description:
> When transferring large files (800meg) or even medium-size files (200meg),
> the transfer rate decreases constantly. It starts at around 4meg/sec and
> goes down to even less than 800kb/s. When I cancel the download and start it
> over again, the transfer rate comes back up to 4meg/sec.
> I am using Nautilus on Ubuntu 8.04 (hardy heron)
>

[Theodore Ts'o (tytso)]

On Wed, May 20, 2009 at 08:48:34PM -0000, Night64 wrote:
> Thanks for the info, Ts'o. But your comment about libusual caught my eye. In
> Ubuntu 9.04 this module still shows up, right?
>
> [81601.434729] usbcore: registered new interface driver libusual

Oops, sorry, that's what I get for not checking the driver names more
closely. libusual is mostly harmless; it's the mapping function that
arbitrates between ub and usb-storage drivers. The USB low
performance device driver is ub.ko, normally found in,
kernel/drivers/block/ub.ko:

% find /lib/modules -name ub.ko
/lib/modules/2.6.27-9-generic/kernel/drivers/block/ub.ko
/lib/modules/2.6.28-11-generic/kernel/drivers/block/ub.ko

You really, really, *really* want to avoid using the low performance
USB driver at all costs. Fortunately at least with modern kernels
libusual defaults to trying to prefer the usb-storage driver. I'm not
sure of the time frame, but a while ago, some distributions were
defaulting to using the ub driver, and *boy* was that a disaster.

The ub driver is useful in embedded devices where you might not want
to have the full USB stack (udev, hal, etc.), but I'm not sure why it
anyone would want it to use it on a normal system....

Simon's observation is quite right, some flash devices are just slow.
Note that "1x" flash devices will only write at 150k/s. A "16x"
device will write at 2.4 MB/s. And this is assuming the filesystem
doesn't get in the way. This is the speed rating used by Compact
Flash or Secure Digital (CF or SD) cards. USB sticks generally don't
have speed ratings at all, and depending on what you've purchased,
they could be quite slow indeed. Note that with SD cards, sometimes
the high speeds advertised (i.e., 150x) may not apply at all unless
you have special hardware --- or sometimes the manufacturers are just
lying. But given that there do exist "1x" flash devices that only
write at 150k/s should be a warning that some flash devices Are Just
Slow. Hence my request that people who are really interested in
debugging this should use USB hard drives, and not USB flash devices
--- there's no way for us on the other side to know whether you are
using a high quality or low-quality flash device, and so the speed
problem could just be normal operation.

In general, sometimes small writes will seem to go fast, because they
get cached in memory. But just because the write operation seems to
have completed doesn't mean that it's really done, or that it's safe
to remove the flash device. So when you see an LED on a USB stick
flashing, that's a sign that it's still writing out. If you write a
very large file, eventually there's no more memory available, and so
now the slow write speed becomes visible to the user.

So this is why I'm becoming more and more convinced that we should
just nuke this bug from orbit and start over. Naive users who don't
understand this, can often see normal system performance, and report a
"me too!", and this just gums up the works.

There ***may*** be a real bug hiding in here somewhere, by some users.
Or it culd be a configuration bug; or a hardware bug. But there are
too many users who are saying, "I'm seeing this too!", when...

[epv (epvubuntu)]

This isn't normal behavior. Obviously it behaves fine while all writes are going to buffer cache, but once it tries to write it out to the device, wait times on the partition shoot up to 30sec or more, and all processes trying to touch the usb disk block practically forever. Sysstat shows it as 100% busy continuously. The reports of "Fast at first, then slowing down" are of course because of the buffer cache making it look like it's initially "fast" but access to the underlying device is always very slow, whether you are writing to it with gvfs, cp, dd, buffer, or whatever.

The usb storage device *is* being claimed by ehci, not uhci, which I initially suspected was the problem. (btw the decision to compile the uhci-hcd and ehci-hcd drivers rather than leave them as modules, I think, is a bad one, since it eliminates the possibility of removing uhci-hcd to force ehci-hcd to claim a driver, as is sometimes necessary)

In addition, it's only writes that are slow, and they're vastly slower than could be explained by a crappy USB device- in my case around 100 - 200 kbytes/sec on a device that under a Hardy install writes at several mbytes/sec.

In my case this is happening on an lenovo X61 (intel) 2.6.28-11-generic.

[jamesnmandy (jamesnmandy)]

There's no way all these people are seeing this behaviour because they all
have vastly different yet commonly "crappy" hardware. The same exact
hardware works great under the "other" OS. It's 100% a *nix issue and it's
100% in the software. That's painfully obvious. I wish this would get
fixed really soon, I recently went back to the "other" OS and it itself has
many stability issues that Ubuntu does not have, so I promptly switched back
to 9.04. But this USB "bug" needs to be straightened out because it greatly
affects the usability of the OS in today's USB age.

On Mon, Jun 8, 2009 at 4:14 PM, epv <email address hidden> wrote:

> This isn't normal behavior. Obviously it behaves fine while all writes
> are going to buffer cache, but once it tries to write it out to the
> device, wait times on the partition shoot up to 30sec or more, and all
> processes trying to touch the usb disk block practically forever.
> Sysstat shows it as 100% busy continuously. The reports of "Fast at
> first, then slowing down" are of course because of the buffer cache
> making it look like it's initially "fast" but access to the underlying
> device is always very slow, whether you are writing to it with gvfs, cp,
> dd, buffer, or whatever.
>
> The usb storage device *is* being claimed by ehci, not uhci, which I
> initially suspected was the problem. (btw the decision to compile the
> uhci-hcd and ehci-hcd drivers rather than leave them as modules, I
> think, is a bad one, since it eliminates the possibility of removing
> uhci-hcd to force ehci-hcd to claim a driver, as is sometimes necessary)
>
> In addition, it's only writes that are slow, and they're vastly slower
> than could be explained by a crappy USB device- in my case around 100 -
> 200 kbytes/sec on a device that under a Hardy install writes at several
> mbytes/sec.
>
> In my case this is happening on an lenovo X61 (intel) 2.6.28-11-generic.
>
> --
> file transfers on USB disk are very slow
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in gvfs: Invalid
> Status in “gvfs” source package in Ubuntu: Invalid
> Status in “linux” source package in Ubuntu: Confirmed
>
> Bug description:
> When transferring large files (800meg) or even medium-size files (200meg),
> the transfer rate decreases constantly. It starts at around 4meg/sec and
> goes down to even less than 800kb/s. When I cancel the download and start it
> over again, the transfer rate comes back up to 4meg/sec.
> I am using Nautilus on Ubuntu 8.04 (hardy heron)
>

[jamesnmandy (jamesnmandy)]

Not sure if this has been mentioned or not, but I noticed today that
performance is much much better when copying directories to directories vs.
copying a file to a directory. For example, I had a directory on my
desktop, with three subdirectories, each with two files in them. When I
right-click copy and then right-click paste on to my external USB HDD to a
particular directory the file transfer speed stays at 15MBps +/- instead of
the usual starting at 15MBps and tapering off to 5MBps or less and
eventually just stopping altogether. The total size of the main directory
being copied was just over 36GB. Each of the three subdirectories contained
a 10-12GB file and a smaller 60kb file each.

I just thought it was odd that the file and directory structure affected the
file transfer performance and wanted to share.

On Mon, Jun 8, 2009 at 9:54 PM, jamesnmandy <email address hidden> wrote:

> There's no way all these people are seeing this behaviour because they all
> have vastly different yet commonly "crappy" hardware. The same exact
> hardware works great under the "other" OS. It's 100% a *nix issue and it's
> 100% in the software. That's painfully obvious. I wish this would get
> fixed really soon, I recently went back to the "other" OS and it itself has
> many stability issues that Ubuntu does not have, so I promptly switched back
> to 9.04. But this USB "bug" needs to be straightened out because it greatly
> affects the usability of the OS in today's USB age.
>
>
> On Mon, Jun 8, 2009 at 4:14 PM, epv <email address hidden> wrote:
>
>> This isn't normal behavior. Obviously it behaves fine while all writes
>> are going to buffer cache, but once it tries to write it out to the
>> device, wait times on the partition shoot up to 30sec or more, and all
>> processes trying to touch the usb disk block practically forever.
>> Sysstat shows it as 100% busy continuously. The reports of "Fast at
>> first, then slowing down" are of course because of the buffer cache
>> making it look like it's initially "fast" but access to the underlying
>> device is always very slow, whether you are writing to it with gvfs, cp,
>> dd, buffer, or whatever.
>>
>> The usb storage device *is* being claimed by ehci, not uhci, which I
>> initially suspected was the problem. (btw the decision to compile the
>> uhci-hcd and ehci-hcd drivers rather than leave them as modules, I
>> think, is a bad one, since it eliminates the possibility of removing
>> uhci-hcd to force ehci-hcd to claim a driver, as is sometimes necessary)
>>
>> In addition, it's only writes that are slow, and they're vastly slower
>> than could be explained by a crappy USB device- in my case around 100 -
>> 200 kbytes/sec on a device that under a Hardy install writes at several
>> mbytes/sec.
>>
>> In my case this is happening on an lenovo X61 (intel) 2.6.28-11-generic.
>>
>> --
>> file transfers on USB disk are very slow
>> https://bugs.launchpad.net/bugs/197762
>> You received this bug notification because you are a direct subscriber
>> of the bug.
>>
>> Status in gvfs: Invalid
>> Status in “gvfs” source package in Ubuntu: Invalid
>> Status in “linux” source package in Ubuntu: Confirmed
>>
>> Bug d...

[Simon Holm (odie-cs)]

man, 08 06 2009 kl. 20:14 +0000, skrev epv:
> This isn't normal behavior. Obviously it behaves fine while all writes
> are going to buffer cache, but once it tries to write it out to the
> device, wait times on the partition shoot up to 30sec or more, and all
> processes trying to touch the usb disk block practically forever.

If other processes get completely starved due to the writeout then there
is probably something in the VM, file system or I/O scheduler that needs
to be fixed. You would have to make a convincing argument, though, like
providing a very simple usecase. It should literally be something as
simple as the following

(cp bigfile /media/disk/ &) ; sleep 10 ; time echo foo > /media/disk/bar

If the time is really in seconds as you seem to suggest it is
interesting I'd say. On the other hand, if what you describe is due the
low performance of the underlying device then it is expected behaviour
(the low performance of the device might be due to a bug though).

> Sysstat shows it as 100% busy continuously.

'It' being the underlying device? That is what you would expect.

> The reports of "Fast at
> first, then slowing down" are of course because of the buffer cache
> making it look like it's initially "fast" but access to the underlying
> device is always very slow, whether you are writing to it with gvfs, cp,
> dd, buffer, or whatever.

Yes, and?

> In addition, it's only writes that are slow, and they're vastly slower
> than could be explained by a crappy USB device- in my case around 100 -
> 200 kbytes/sec on a device that under a Hardy install writes at several
> mbytes/sec.

If Hardy was fine then you should open a new bug report with the proper
details (once again, see [1] or Theodore's comments).

[1] https://help.ubuntu.com/community/DiskPerformance

[Simon Holm (odie-cs)]

> On Mon, Jun 8, 2009 at 9:54 PM, jamesnmandy <email address hidden>
> wrote:
>
> > There's no way all these people are seeing this behaviour because they all
> > have vastly different yet commonly "crappy" hardware. The same exact
> > hardware works great under the "other" OS. It's 100% a *nix issue and it's
> > 100% in the software. That's painfully obvious.

Stop thinking that Nautilus is a good indicator of speed already damnit!
It is possible that you and other users really get buggily slow
performance, but I assure that there is just as many in this thread that
experience no other bugs than themselves interpretting Nautilus wrong.

And since you didn't notice, there is nothing to solve as there haven't
been supplied sufficient information to diagnose a bug.

If you've got hardware that performs significantly better with Windows
or different kernels then file a proper bug.

> > I wish this would get fixed really soon, I recently went back to the "other"
> > OS and it itself has many stability issues that Ubuntu does not
> > have, so I promptly switched back to 9.04.

And I wish all you people got a clue as how to make a useful bug report
really soon, but it seems like I'll have to go back to dreaming.

> Not sure if this has been mentioned or not, but I noticed today that
> performance is much much better when copying directories to directories vs.
> copying a file to a directory. For example, I had a directory on my
> desktop, with three subdirectories, each with two files in them. When I
> right-click copy and then right-click paste on to my external USB HDD to a
> particular directory the file transfer speed stays at 15MBps +/- instead of
> the usual starting at 15MBps and tapering off to 5MBps or less and
> eventually just stopping altogether. The total size of the main directory
> being copied was just over 36GB. Each of the three subdirectories contained
> a 10-12GB file and a smaller 60kb file each.

Are these dstat numbers? Anyway, file a proper bug if you want progress.

[jamesnmandy (jamesnmandy)]

Hiding behind a "proper bug request" gets us nowhere. Additionally, I do
not have the time to do the research to figure out how to even begin to
"file a proper bug report". I'm one of those....you know, former Windows
users, who do everything with a GUI and if there's not a double click or
next->next->finish wizard to do it, then it's not going to happen with me
and my machine. I am what you would call the average user, average joe, who
just wants it to work like it's supposed to. It's not working like it is
supposed to and that's painfully obvious. I can tell you my same exact
hardware works multitudes better in terms of USB performance running the
other OS. Not sure what a bug report has to do with that, it's a fact. I
suppose you want average joe to show you how the software is messed up. I'm
game for that as long as you make error logging and reporting an automated
thing. Then you would not have to rely on people like me who obviously
don't know as much as yourself to find and implement bug fixes.

Thanks for your support.

2009/6/9 Simon Holm Thøgersen <email address hidden>

> man, 08 06 2009 kl. 20:14 +0000, skrev epv:
> > This isn't normal behavior. Obviously it behaves fine while all writes
> > are going to buffer cache, but once it tries to write it out to the
> > device, wait times on the partition shoot up to 30sec or more, and all
> > processes trying to touch the usb disk block practically forever.
>
> If other processes get completely starved due to the writeout then there
> is probably something in the VM, file system or I/O scheduler that needs
> to be fixed. You would have to make a convincing argument, though, like
> providing a very simple usecase. It should literally be something as
> simple as the following
>
> (cp bigfile /media/disk/ &) ; sleep 10 ; time echo foo > /media/disk/bar
>
> If the time is really in seconds as you seem to suggest it is
> interesting I'd say. On the other hand, if what you describe is due the
> low performance of the underlying device then it is expected behaviour
> (the low performance of the device might be due to a bug though).
>
> > Sysstat shows it as 100% busy continuously.
>
> 'It' being the underlying device? That is what you would expect.
>
> > The reports of "Fast at
> > first, then slowing down" are of course because of the buffer cache
> > making it look like it's initially "fast" but access to the underlying
> > device is always very slow, whether you are writing to it with gvfs, cp,
> > dd, buffer, or whatever.
>
> Yes, and?
>
> > In addition, it's only writes that are slow, and they're vastly slower
> > than could be explained by a crappy USB device- in my case around 100 -
> > 200 kbytes/sec on a device that under a Hardy install writes at several
> > mbytes/sec.
>
> If Hardy was fine then you should open a new bug report with the proper
> details (once again, see [1] or Theodore's comments).
>
> [1] https://help.ubuntu.com/community/DiskPerformance
>
> --
> file transfers on USB disk are very slow
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in gvfs: Invalid
> Status in “gvfs” so...

[Simon Holm (odie-cs)]

ons, 10 06 2009 kl. 22:38 +0000, skrev jamesnmandy:
> Hiding behind a "proper bug request" gets us nowhere.

And you posting useless information in this bug does?

> It's not working like it is supposed to and that's painfully obvious.

And it should be painfully obvious by now that the issue cannot be fixed
if nobody can provide sufficient information to solve the problem.
Capisce?

I really don't want to be rude, but how do you expect anyone to fix a
problem this vaguely described? It is not like we can reproduce it with
our own hardware, on the contrary it works as expected. It is not
exactly easy to tell that the BIOS should be upgraded to fix the problem
like in [1].

So send us your hardware, provide root ssh access to the hardware with
or give us a proper report. It is not like it should be too difficult to
follow the troubleshooting steps of [2] or simply shut up.

> I can tell you my same exact hardware works multitudes better in terms of
> USB performance running the other OS. Not sure what a bug report has to do
> with that, it's a fact.

It is a good first step to know the hardware doesn't suck, then we can
move on to the next in [2].

And I sympathize with you and others not being tech savy and just want
your OS to work, but unless you actually pay for support or someone to
look at your problem you should respect that there is only so much you
can expect of people using their spare time to support you.

[Simon Holm (odie-cs)]

[1] https://bugs.launchpad.net/ubuntu/+bug/334914/comments/30
[2] https://help.ubuntu.com/community/DiskPerformance

[Ulrich Lukas (ulrich-lukas)]

Who wants my Hardware?

Since I can't use my USB drive (I cancelled the last attempt to copy
6GiB after more than one hour and less than 10%):

Whoever wants to do further testing or bugfixing:

You get my 16-GiB USB pen drive for for free! I can send it to you
worldwide in a cushioned envelope.

Just write me an e-mail with your name and adress. The subject line must
contain the word "bug" (because of my spam filter).

[Angelos Vlassopoulos (avlass)]

I ran some tests and found out that my USB flash disk is significantly slower when it is formatted as FAT32, compared to ext3.

I have created a new bug, which can be found here:

https://bugs.launchpad.net/ubuntu/+bug/392089

[epv (epvubuntu)]

> (cp bigfile /media/disk/ &) ; sleep 10 ; time echo foo > /media/disk/bar

> If the time is really in seconds as you seem to suggest it is
> interesting I'd say. On the other hand, if what you describe is due the
> low performance of the underlying device then it is expected behaviour
> (the low performance of the device might be due to a bug though).

# (cp /dev/zero /media/disk/bigfile &); sleep 10; time echo foo > /media/disk/bar

real 0m5.647s
user 0m0.000s
sys 0m0.004s

# time echo foo > /media/disk/foo2

real 0m3.534s
user 0m0.000s
sys 0m0.000s
# time echo foo > /media/disk/foo3

real 0m4.589s
user 0m0.000s
sys 0m0.000s
# time echo foo > /media/disk/foo3

real 0m16.913s
user 0m0.000s
sys 0m0.000s

Imagine that! I wasn't lying.

>> The reports of "Fast at
>> first, then slowing down" are of course because of the buffer cache
>> making it look like it's initially "fast" but access to the underlying
>> device is always very slow, whether you are writing to it with gvfs, cp,
>> dd, buffer, or whatever.
>
>Yes, and?

"and" so the arguments of "it's fast at first, then gets slow" from naive users are
spurious, which I'm sure you realized.

>> In addition, it's only writes that are slow, and they're vastly slower
>> than could be explained by a crappy USB device- in my case around 100 -
>> 200 kbytes/sec on a device that under a Hardy install writes at several
>> mbytes/sec.
>
> If Hardy was fine then you should open a new bug report with the proper
> details (once again, see [1] or Theodore's comments).

There has been a lot of unhelpful or misguided information posted to this bug,
but be careful not to discount the good with the bad. It's fairly clear you either
don't believe there's a problem or aren't interested in it, which is fine, but it
might be both more helpful and more civil to just say so, rather than being rude
to people who are trying to help with whatever means they have available.

eric

[jamesnmandy (jamesnmandy)]

This is completely not flamebait, promise, I have been doing nothing but
bragging on the latest build of Ubuntu, but I loaded Windows 7 back on this
same exact machine, only difference was the OS, and suddenly I get transfer
speeds of 25Mb/s or higher consistently even on extremely large transfers.
On Ubuntu 9.04 it would never get above 15Mb/s and would quickly drop to
nearly half of that and eventually stop altogether if the transfer was a
very large one that had to be sustained. My point is, for all the nay
sayers who say it's people's hardware....it's not, unless it's that Ubuntu
is not playing nice with certain hardware configurations. This is obviously
the case, but it is not a hardware issue at all when a simple change of the
OS completely fixes the issue.

On Tue, Jul 14, 2009 at 6:58 PM, epv <email address hidden> wrote:

> > (cp bigfile /media/disk/ &) ; sleep 10 ; time echo foo >
> /media/disk/bar
>
> > If the time is really in seconds as you seem to suggest it is
> > interesting I'd say. On the other hand, if what you describe is due the
> > low performance of the underlying device then it is expected behaviour
> > (the low performance of the device might be due to a bug though).
>
> # (cp /dev/zero /media/disk/bigfile &); sleep 10; time echo foo >
> /media/disk/bar
>
> real 0m5.647s
> user 0m0.000s
> sys 0m0.004s
>
> # time echo foo > /media/disk/foo2
>
> real 0m3.534s
> user 0m0.000s
> sys 0m0.000s
> # time echo foo > /media/disk/foo3
>
> real 0m4.589s
> user 0m0.000s
> sys 0m0.000s
> # time echo foo > /media/disk/foo3
>
> real 0m16.913s
> user 0m0.000s
> sys 0m0.000s
>
> Imagine that! I wasn't lying.
>
>
> >> The reports of "Fast at
> >> first, then slowing down" are of course because of the buffer cache
> >> making it look like it's initially "fast" but access to the underlying
> >> device is always very slow, whether you are writing to it with gvfs, cp,
> >> dd, buffer, or whatever.
> >
> >Yes, and?
>
> "and" so the arguments of "it's fast at first, then gets slow" from naive
> users are
> spurious, which I'm sure you realized.
>
> >> In addition, it's only writes that are slow, and they're vastly slower
> >> than could be explained by a crappy USB device- in my case around 100 -
> >> 200 kbytes/sec on a device that under a Hardy install writes at several
> >> mbytes/sec.
> >
> > If Hardy was fine then you should open a new bug report with the proper
> > details (once again, see [1] or Theodore's comments).
>
> There has been a lot of unhelpful or misguided information posted to this
> bug,
> but be careful not to discount the good with the bad. It's fairly clear you
> either
> don't believe there's a problem or aren't interested in it, which is fine,
> but it
> might be both more helpful and more civil to just say so, rather than being
> rude
> to people who are trying to help with whatever means they have available.
>
> eric
>
>
> ** Attachment added: "iostat output during "cp /dev/zero
> /media/disk/bigfile"
> http://launchpadlibrarian.net/29027766/iostat-output
>
> --
> file transfers on USB disk are very slow
> https://bugs.launchpad.net/bugs/197762
> You received this bug noti...

[Theodore Ts'o (tytso)]

Eric,

The problem is that your problem may be very different from other peoples' problem. First of all, in your test, you are continually dirtying the page cache with "cp /dev/zero /mnt/bigfile". This brings in filesystem effects, and VM effects, and so it's hard to tell what is actually going on. Without a lot of instrumentation, and knowing exactly the state of memory, and when the filesystem might be doing a commit, it's hard to say exactly what might be going on. You also haven't given us the kernel bootup messages (dmesg output after the system boots), as well as the type of device that you are using, the lsusb output, etc., etc., etc. You haven't told us how much memory there is in the system.

And certainly the test which you did "(cp /dev/zero /media/disk/bigfile &); sleep 10 ; time echo foo > /media/disk/test" is not one that we can replicate on Windows, so the assertion that something is wrong is also not something which is immediately obvious given the very limited amount of information you have given. Maybe it's *somewhere* in this bug report, bug this is why it is really not useful to mingle multiple users' problems in one bug report.

If you want to open a separate bug report, give lots of details, and be willing to work with people who can suggest specific, repeatable experiments that examine one aspect of the system at a time, then we can try to see if there's something unusual. If you just want to whine about how horrible Ubuntu or Linux is, then keep on doing what you're doing here. It's not going to lead to anything useful though. But if you are really interested in being constructive, we need your help in giving us something useful in terms of enough information so we can find the root cause of a problem, and then try to solve it.

[Theodore Ts'o (tytso)]

jamesnmandy,

For someone who 'does not have the time to file a proper bug report', you seem to have plenty of time to write notes complaining about the bug. Seriously --- if you aren't willing to help determine the problem --- and you like GUI solutions --- maybe you should just go back to Windows. You may find that you have even less responsiveness to your bug reports, unless you pay $$$ to Microsoft, but hey, if it makes you happier, you should switch.

The problem could be with Nautilus (the gui file manager); or it could be with the device driver; or it could be with the ext3 filesystem issue where some process was triggering fsync calls. If you aren't willing to figure out why it's no fast enough for you, and you aren't willing to lift your little finger to help us figure out why; then please, don't complain --- you're clearly unhappy, and perhaps moving to Vista will make you happy. And at the end of the day, it's better for you to be happy than to have you bitter and complaining --- really, trust me, in the end you'll be happier to stop kvetching about Linux. It's simply not productive for anyone, least of all yourself.

On the other hand, if you are willing to learn a few things, and spend a bit of time, then maybe using Linux will be more rewarding for you --- and again, in the long time you will probably be happier.

[epv (epvubuntu)]

On Wed, Jul 15, 2009 at 01:55:49AM -0000, Theodore Ts'o wrote:
> Eric,
>
> The problem is that your problem may be very different from other
> peoples' problem. First of all, in your test, you are continually
> dirtying the page cache with "cp /dev/zero /mnt/bigfile". This brings
> in filesystem effects, and VM effects, and so it's hard to tell what is
> actually going on.

I was showing the results of the test that Simon propoesed a few posts
earlier. Sorry I didn't attribute my reply properly. It was in response to
the following:

Simon Holm Th\370gersen wrote on 2009-06-09:
> It should literally be something as simple as the following
> (cp bigfile /media/disk/ &) ; sleep 10 ; time echo foo > /media/disk/bar
> If the time is really in seconds as you seem to suggest it is
> interesting I'd say.

Anyway I did also attach iostat output from during the test.

> And certainly the test which you did "(cp /dev/zero /media/disk/bigfile
> &); sleep 10 ; time echo foo > /media/disk/test" is not one that we can
> replicate on Windows, so the assertion that something is wrong is also
> not something which is immediately obvious given the very limited amount
> of information you have given. Maybe it's *somewhere* in this bug
> report, bug this is why it is really not useful to mingle multiple
> users' problems in one bug report.

I'm not sure what to tell you about it not being repeatable on Windows, as
I don't use it. But I think it would be silly to restrict debugging efforts
to only those that can be repeated in Microsoft operating systems,
especially since it may well turn out to be a VM system issue.
I agree that this particular bug report has degenerated into nonsense and
I'll be leaving it alone henceforth.

> [ ... ] you just want to whine
> about how horrible Ubuntu or Linux is, then keep on doing what you're
> doing here. It's not going to lead to anything useful though. But if

Maybe you're mistaking me for someone else. I haven't engaged in any
bashing of Ubuntu or Linux. Fwiw i've commercially deployed thousan...

[kulight (kulight)]

I've just remembered why the user base of Linux is so small

I do actually expect my mechanic to find and fix what's wrong with my car when I just tell him it won't start
And I don’t care if it's the engine the battery or the radiator
You have to understand that most users have no idea where the problem is coming from or if its two separate ones
They are just using the system and see the problems

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Theodore Ts'o
Sent: Wednesday, July 15, 2009 4:43 PM
To: <email address hidden>
Subject: [Bug 197762] Re: file transfers on USB disk are very slow

On Tue, Jul 14, 2009 at 08:45:56PM -0700, Ron Brogden wrote:
> Theodore, if I may ask you - are you just another random Linux user
> here or are you actually officially associated with Ubuntu in a
> support role? I see that you are associated with IBM, have written
> some file system utilities and appear to be connected with kernel
> development but am unsure your role here with respects to this bug.

I'm a linux kernel developer, and yes, I'm the maintaining of the ext4
filesystem and e2fsprogs. I started paying attention to this problem
because someone sent a note to the LKML (Linux Kernel Mailiung List)
complaining that kernel developers don't pay attention to Ubuntu bugs.

Keep in mind most Ubuntu users get Ubuntu for free, and pay no support fees; so it's not surprising that you will see very few Ubuntu support personnel paying attention to bugs like this one. So at some level, the Ubuntu user community and Canonical has a choice to make. If they
are willing to let users just whine and whine and whine and _not_ help us by running experiments and opening separate bugs for each problem, and give full information (and we will provide them with explicit instructions on how to run those experiments), fine. We can just go
back to ignoring Ubuntu launchpad as being totally, completely, worthless for actually fixing bugs. It can just be a place toxic sewer pit for users to whine, and if Canonical wants to pay someone to
try to extract useful information (lots of luck; theres very little here) after the fact, they can go right ahead. We'll just file Ubuntu into the "there's no intelligent life here" category, and ignore pleas for help on LKML when people ask developers to pay attention to Launchpad bugs.

Alternatively, if the goal is to have one part of the community helping
another, then the users have to be helpful. That means doing research
before filing bugs, and filing separate bugs for separate issues.
Launchpad fails miserably when there are more than 80 comments; it's
slow and painful to use. So I am trying to see if we can salvage the
Launchpad culture so it can be useful, but ultimately, maybe it's a
losing battle, and we should give up.

> Obviously most folks have no idea why their USB transfers are slow
> but they know something is up. It should be no surprise that they
> post unclear bug reports since there is no real way for them to tell
> whether their file manager, the kernel or some other bit of software
> is the cause. All they know is something is wrong and for those
> that dual b...

[Theodore Ts'o (tytso)]

On Tue, Jul 14, 2009 at 08:45:56PM -0700, Ron Brogden wrote:
> Theodore, if I may ask you - are you just another random Linux user
> here or are you actually officially associated with Ubuntu in a
> support role? I see that you are associated with IBM, have written
> some file system utilities and appear to be connected with kernel
> development but am unsure your role here with respects to this bug.

I'm a linux kernel developer, and yes, I'm the maintaining of the ext4 filesystem and e2fsprogs. I started paying attention to this problem because someone sent a note to the LKML (Linux Kernel Mailiung List) complaining that kernel developers don't pay attention to Ubuntu bugs.

Keep in mind most Ubuntu users get Ubuntu for free, and pay no support fees; so it's not surprising that you will see very few Ubuntu support personnel paying attention to bugs like this one. So at some level, the Ubuntu user community and Canonical has a choice to make. If they
are willing to let users just whine and whine and whine and _not_ help us by running experiments and opening separate bugs for each problem, and give full information (and we will provide them with explicit instructions on how to run those experiments), fine. We can just go
back to ignoring Ubuntu launchpad as being totally, completely, worthless for actually fixing bugs. It can just be a place toxic sewer pit for users to whine, and if Canonical wants to pay someone to
try to extract useful information (lots of luck; theres very little here) after the fact, they can go right ahead. We'll just file Ubuntu into the "there's no intelligent life here" category, and ignore pleas for help on LKML when people ask developers to pay attention to Launchpad bugs.

Alternatively, if the goal is to have one part of the community helping another, then the users have to be helpful. That means doing research before filing bugs, and filing separate bugs for separate issues. Launchpad fails miserably when there are more than 80 comments; it's slow and painful to use. So I am trying to see if we can salvage the Launchpad culture so it can be useful, but ultimately, maybe it's a losing battle, and we should give up.

> Obviously most folks have no idea why their USB transfers are slow
> but they know something is up. It should be no surprise that they
> post unclear bug reports since there is no real way for them to tell
> whether their file manager, the kernel or some other bit of software
> is the cause. All they know is something is wrong and for those
> that dual boot, it does not happen with Windows (not my situation
> but it is far from uncommon). Of course this is frustrating for the
> bug fixer but then the response should be a request for hardware
> that displays the problem at the very least (which I believe has
> been offered here).

The problem is that we need to separate out the reports. When something is as vague as "disk transfers are slow", there's a extremely high probability that there may be multiple things going on. Some people might be having genuine hardware problems; others might be having filesystem issues. Eric's (epv's) problem *might* be the classic ext3/fsync stalled writ...

[Night64 (roberto-berlim)]

Well, my linux support fix my problem, even when I can't explain what is the
problem. But I pay for it!
Canonical have a nice and not very pricey support service, maybe you could
hire them.
----------------------------
Linux: Because rebooting is for adding new hardware.

On Wed, Jul 15, 2009 at 8:33 AM, kulight <email address hidden> wrote:

> I've just remembered why the user base of Linux is so small
>
> I do actually expect my mechanic to find and fix what's wrong with my car
> when I just tell him it won't start
> And I don’t care if it's the engine the battery or the radiator
> You have to understand that most users have no idea where the problem is
> coming from or if its two separate ones
> They are just using the system and see the problems
>
> -----Original Message-----
> From: <email address hidden> [mailto:<email address hidden>] On Behalf Of
> Theodore Ts'o
> Sent: Wednesday, July 15, 2009 4:43 PM
> To: <email address hidden>
> Subject: [Bug 197762] Re: file transfers on USB disk are very slow
>
> On Tue, Jul 14, 2009 at 08:45:56PM -0700, Ron Brogden wrote:
> > Theodore, if I may ask you - are you just another random Linux user
> > here or are you actually officially associated with Ubuntu in a
> > support role? I see that you are associated with IBM, have written
> > some file system utilities and appear to be connected with kernel
> > development but am unsure your role here with respects to this bug.
>
> I'm a linux kernel developer, and yes, I'm the maintaining of the ext4
> filesystem and e2fsprogs. I started paying attention to this problem
> because someone sent a note to the LKML (Linux Kernel Mailiung List)
> complaining that kernel developers don't pay attention to Ubuntu bugs.
>
> Keep in mind most Ubuntu users get Ubuntu for free, and pay no support
> fees; so it's not surprising that you will see very few Ubuntu support
> personnel paying attention to bugs like this one. So at some level, the
> Ubuntu user community and Canonical has a choice to make. If they
> are willing to let users just whine and whine and whine and _not_ help us
> by running experiments and opening separate bugs for each problem, and give
> full information (and we will provide them with explicit instructions on how
> to run those experiments), fine. We can just go
> back to ignoring Ubuntu launchpad as being totally, completely, worthless
> for actually fixing bugs. It can just be a place toxic sewer pit for users
> to whine, and if Canonical wants to pay someone to
> try to extract useful information (lots of luck; theres very little here)
> after the fact, they can go right ahead. We'll just file Ubuntu into the
> "there's no intelligent life here" category, and ignore pleas for help on
> LKML when people ask developers to pay attention to Launchpad bugs.
>
> Alternatively, if the goal is to have one part of the community helping
> another, then the users have to be helpful. That means doing research
> before filing bugs, and filing separate bugs for separate issues.
> Launchpad fails miserably when there are more than 80 comments; it's
> slow and painful to use. So I am trying to see if we can salvage the
> Launchpad culture so it ca...

[Theodore Ts'o (tytso)]

On Wed, Jul 15, 2009 at 11:33:07AM -0000, kulight wrote:
> I've just remembered why the user base of Linux is so small
>
> I do actually expect my mechanic to find and fix what's wrong with
> my car when I just tell him it won't start And I don’t care if it's
> the engine the battery or the radiator You have to understand that
> most users have no idea where the problem is coming from or if its
> two separate ones They are just using the system and see the
> problems

Ah, but (a) you pay your mechanic, and (b) you bring your car in, and
don't expect him to fix it over the phone or over an web board.

There are plenty of services for which you can get pay-for-service for
Linux. For the right price, I've been flown into New York City to
work at a large financial institution or even into a classified
machine room at a major defense contractor.

But the difference with Ubuntu Launchpad is that no one pays any money
and they somehow expect gold-plated support. Sorry, for free support,
you have to expect a certain amount of "self service". It's still a
way better deal than you will get anywhere else --- but if you wan the
quality of service you get when you drive the car to your dealer, then
you will have to (a) pay the bug fixer, and (b) let the bug fixer have
direct access to your machine.

[Sergey V. Udaltsov (sergey-udaltsov)]

Same issue with Corsaire Voyager Mini 16G. Works reasonably fast under WinVista

[Ace Suares (acesuares)]

Luckily, the extermely slow USB transfer rate gives me ample time to enjoy the 173 comments in this bug report.

Whoooppeee!

[Edgardo Fredz (edgardo-fredz)]

309,3kb/s. 30 minutes to transfer 700 mb. Fucking awful.

[Tim Gardner (timg-tpi)]

There are 2 changes in recent Karmic kernels that should help:

2.6.31-10.35:
  * [Config] Set default I/O scheduler to DEADLINE
    CFQ seems to have some load related problems which are often exacerbated by sreadahead.
    - LP: #381300

2.6.31-11.38
  * sched: Disable NEW_FAIR_SLEEPERS for now
    - LP: #436342

[lebb (lebber)]

In Karmic transfering a file to the USB disk it is a slight bit faster compared to Jaunty, but moving a big file it is still painfully slow.

[Mikael Frykholm (mikael)]

I got this in dmesg when trying to rsync a windows 7 install dvd to a 4G flash drive:
[227761.643001] INFO: task pdflush:19635 blocked for more than 120 seconds.
[227761.643010] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[227761.643016] pdflush D 00000000ffffffff 0 19635 2 0x00000000
[227761.643029] ffff8801189f9d20 0000000000000046 0000000000000000 0000000000015880
[227761.643040] ffff88001af35e70 0000000000015880 0000000000015880 0000000000015880
[227761.643050] 0000000000015880 ffff88001af35e70 0000000000015880 0000000000015880
[227761.643060] Call Trace:
[227761.643078] [<ffffffff815291bd>] __down_read+0x8d/0xc6
[227761.643088] [<ffffffff81528589>] down_read+0x19/0x20
[227761.643097] [<ffffffff81120ed5>] sync_supers+0x75/0xe0
[227761.643107] [<ffffffff810e2615>] wb_kupdate+0x35/0x140
[227761.643116] [<ffffffff810e411e>] __pdflush+0x13e/0x260
[227761.643125] [<ffffffff810e4240>] ? pdflush+0x0/0x50
[227761.643133] [<ffffffff810e4288>] pdflush+0x48/0x50
[227761.643141] [<ffffffff810e25e0>] ? wb_kupdate+0x0/0x140
[227761.643149] [<ffffffff810e4240>] ? pdflush+0x0/0x50
[227761.643159] [<ffffffff81078746>] kthread+0xa6/0xb0
[227761.643168] [<ffffffff810130ea>] child_rip+0xa/0x20
[227761.643176] [<ffffffff810786a0>] ? kthread+0x0/0xb0
[227761.643183] [<ffffffff810130e0>] ? child_rip+0x0/0x20

[Mikael Frykholm (mikael)]

I can reproduce this every time, please tell me what kind of debug info you need.

[Sebastien Bacher (seb128)]

> trying to rsync a windows 7 install dvd

the issue seems different from a speed one, you should open a new bug rather than commenting there...

[Mikael Frykholm (mikael)]

I got the same slowness when trying dd to the same stick. It was fast at first and gradually got slower (according to kill -USR1 dd) and ended up at about 300k/s.

[Murz (murznn)]

Confirm this problem in Kubuntu Karmic RC amd64 fresh install: transfering a big file (more that 600 mb) to the USB disk is very slow and get about 100% of CPU usage (on AMD Athlon(tm) 7550 Dual-Core Processor with 4gb of RAM) all the copying time.
Can I help with some additional debug info?

[Martin Scharrer (martin-scharrer-online)]

I also suffer from this bug. The transfer speed seems to start high and then falls to kb/s after a short while, see below.
This happens with nautilus, dd, and rsync, etc.

$ rsync -aP * /media/usbmedibig --stats
sending incremental file list
file1
    367,481,854 100% 10.67MB/s 0:00:32 (xfr#1, to-chk=14/15)
file2
    367,362,570 100% 2.18MB/s 0:02:40 (xfr#2, to-chk=13/15)
file3
    367,235,908 100% 1.35MB/s 0:04:19 (xfr#3, to-chk=12/15)
file4
    125,108,224 33% 551.88kB/s 0:07:20 ^C

[Ulrich Lukas (ulrich-lukas)]

Martin and Murz,

maybe you can get help if you open up an individual, detailed bug report.

Given that this 184-message bug report is over one and a half year old,
I'm afraid adding comments here does not help.

This bugfix works, I've just tested it:

http://www.microsoft.com/windows/windows-7/default.aspx

[jamesnmandy (jamesnmandy)]

LOL, same patch applied here, having something as basc as USB communications
breaking down essentially breaks the entire OS for most people, i really
wanted Ubuntu to be my last OS but this bug killed it

On Thu, Oct 29, 2009 at 10:53 PM, Ulrich Lukas <
<email address hidden>> wrote:

> Martin and Murz,
>
> maybe you can get help if you open up an individual, detailed bug
> report.
>
> Given that this 184-message bug report is over one and a half year old,
> I'm afraid adding comments here does not help.
>
> This bugfix works, I've just tested it:
>
> http://www.microsoft.com/windows/windows-7/default.aspx
>
> --
> file transfers on USB disk are very slow
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in GVFS: Invalid
> Status in “gvfs” package in Ubuntu: Invalid
> Status in “linux” package in Ubuntu: Triaged
>
> Bug description:
> When transferring large files (800meg) or even medium-size files (200meg),
> the transfer rate decreases constantly. It starts at around 4meg/sec and
> goes down to even less than 800kb/s. When I cancel the download and start it
> over again, the transfer rate comes back up to 4meg/sec.
> I am using Nautilus on Ubuntu 8.04 (hardy heron)
>

[Sebastien Bacher (seb128)]

those comments are not constructive could you restain from adding such notes there?

[Shawn Reist (icabaud)]

Hello,

I am not a programmer and not that versed on how things in Linux work;
still quite Green.

Can a program be made to monitor a file transfer. i.e. track the
progress. there is something that is happening in the background.

The best file transfers I have is just after a reboot of the computer...
then things go slowly down hill.

I have noticed I can have a good transfer for 1/2 the file then there is
a pause and the rest of the transfer process is jerky/spurts of data
written/transferred, the process takes longer to complete as the
concurrent transfer packets or amount of data is slower/smaller.

Is it possible to have a packet sniffer for file transfers and it would
log the rate of the transfer packet size modules being used or errors
logged and have a time stamp. there has to be something that we just
don't see if a user pulls the logs manually or does a lsusb, df,
hdparm, . . .

one would think that a time chart or sequenced time line of the event
would help point in the direction of the problem logically?.

yes?

no?

[rom85 (rom85)]

I have tried several PCs. They were based upon Intel chipsets such as G33/31, i865, i915, i945 all with integrated video used. All of them had Celeron proccessors on board, 512MM of RAM and SATA 80GB drives. Tested versions of Ubuntu were 8.10, 9.04 and 9.10. None of them worked good, talking about USB speeds. I've tried different combinations, playing with enabling/disabling Usb Legacy Support in BIOS, setting "pci=noapc pci=routeirq" and "elevator" parameters in grub's configuration. These were some tricks discussed on forums. None of these experiments helped on any of the machines. The symptoms were: when copying files to flash drive the speed decreases from about 24BMps at start (according to speed indicator either in Nautilus or MC) to almost 1MBps->200-300kbps, than the progress freezes (doesn't answer for escape key or Cancel) as long as system starts hanging also, while CPU load isn't high at all. When you copy small files up to 50-150MB it's all right, written quite fast, but films are impossible to write as 1,4GB is a hard task. When you watch the copy-progress bar it's either moving fast or stopping for some time, than again. Even if copying is finished in a program I can see that usb stick is working for some time more indicating that with a lamp on it.
For those willing to say to buy a new flash drive - they are workable and well and of different manufacturers and USB 2.0 is enabled in BIOS.
Sincerely see further for bug fixing. I will supply all necessary information if requested.

[Dexter (pogany-tamas+bug)]

Bug confirmed on Karmic Koala

[Bais (bais)]

At this moment, the only way to survive at this bug is open an ftp or sftp
connection in local host and copy files with filezilla or similar client
O_O.

It's an absurd solution, I know, but the only one that not freeze my work.

by BAIS

2009/11/8 Dexter <<email address hidden> <pogany.tamas%<email address hidden>>>

> Bug confirmed on Karmic Koala
>
> --
> file transfers on USB disk are very slow
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>
> Status in GVFS: Invalid
> Status in “gvfs” package in Ubuntu: Invalid
> Status in “linux” package in Ubuntu: Triaged
>
> Bug description:
> When transferring large files (800meg) or even medium-size files (200meg),
> the transfer rate decreases constantly. It starts at around 4meg/sec and
> goes down to even less than 800kb/s. When I cancel the download and start it
> over again, the transfer rate comes back up to 4meg/sec.
> I am using Nautilus on Ubuntu 8.04 (hardy heron)
>

--
by BAIS

FreeFAX: [phone]

[rom85 (rom85)]

2 BAIS:
this doesn't help for my situation also. when copying by filezilla first 50mb are copied quite fast, but after that great slowdown as if it was copied from hdd.

[Shawn Reist (icabaud)]

I will agree the FTP does not work for me either.

the best thing for myself was to do a reboot. transfers are good for the
first hour or two... then get mediocre and after a day possibly two...
abysmal.

rom85 wrote:
> 2 BAIS:
> this doesn't help for my situation also. when copying by filezilla first 50mb are copied quite fast, but after that great slowdown as if it was copied from hdd.
>
>

[Denis Hamann (misterd)]

I have the same problem, but rebooting doesnt help me.

[Bais (bais)]

Strange I used this method for gigas and gigas and I continued to work well,
I have kubuntu 9.04 daily updated.
I'm using server kernel, I don't know why :) Probably for some application
I've installed for testing.

I have a dream... this software on linux -->>>>>>>>>>>>
www.killprog.com Killcopy ! best file manager , pause, anti crash,
buffer tuning, bandwidth,
different configuration for differente kind of devices... all in 1 little
program... parallel copy, queue... other is teracopy but kill copy rocks. If
I was a programmer I make it on linuxbox.

I will test on a scp / ftp session and I will post my results.

by BAIS

2009/11/17 Shawn Reist <email address hidden>

> I will agree the FTP does not work for me either.
>
> the best thing for myself was to do a reboot. transfers are good for the
> first hour or two... then get mediocre and after a day possibly two...
> abysmal.
>
>
> rom85 wrote:
> > 2 BAIS:
> > this doesn't help for my situation also. when copying by filezilla first
> 50mb are copied quite fast, but after that great slowdown as if it was
> copied from hdd.
> >
> >
>
> --
> file transfers on USB disk are very slow
> https://bugs.launchpad.net/bugs/197762
> You received this bug notification because you are a direct subscriber
> of a duplicate bug.
>
> Status in GVFS: Invalid
> Status in “gvfs” package in Ubuntu: Invalid
> Status in “linux” package in Ubuntu: Triaged
>
> Bug description:
> When transferring large files (800meg) or even medium-size files (200meg),
> the transfer rate decreases constantly. It starts at around 4meg/sec and
> goes down to even less than 800kb/s. When I cancel the download and start it
> over again, the transfer rate comes back up to 4meg/sec.
> I am using Nautilus on Ubuntu 8.04 (hardy heron)
>

--
by BAIS

FreeFAX: [phone]

[Shawn Reist (icabaud)]

I asked this some where but maybe not here...

I can't program, but is it possible to gather data via a script that
would put data into a sort of quasi time line format with certain data
appended to it..

i.e

start copy.

time bytes written rate
.
time bytes written rate
.
.
.

end time total bytes written

maybe one could set the program to be variable for time say 10, 30, 60.
. second intervals

other data append at the end, the modules used, Uname -a... and
computer information..

like I said I can't program... would need lots of help.. but for an idea...?

[Ulrich Lukas (ulrich-lukas)]

Dear Shawn,

"dstat" does essentially that.

Default is one line of output every second, which means the output
equals the MiB/sec throughput of your device.

Try "dstat -D sda,sdb". You can also adjust the delay; see the manual page.

[Luis Lobo (luislobo)]

I had this problem, and after reading all this comments, i decided to give it a try to the reconfiguration of fstab

I went from 4 MB to a steady 30MB (this is the REAL max throughput of a USB device in USB 2.0)

It worked perfectly. Also, setting my SATA Ports to use 32 bit and it made it a little better.

Setting "noatime,nodiratime" to the mount options in /etc/fstab solves the problem for me atleast

Ubuntu 64 here:
Linux lap-lub-lnx 2.6.31-15-generic #50-Ubuntu SMP Tue Nov 10 14:53:52 UTC 2009 x86_64 GNU/Linux
hardware: Intel Core 2 Duo (Centrino 2)

[Shawn Reist (icabaud)]

Thanks Ulrich,

I have started to log items and placed the data in a ODS file posted above. Message and attachment did not go through together.

I started with data being written to a .txt file to home directory. . . changed to the /dev/shm directory so data written to the file would not conflict with the data captured for the USB stick.

three sheets... labelled with hours, min and seconds of running time (machine uptime)

uname -a
    Linux icabaud 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:04:26 UTC 2009 i686 GNU/Linux

System
    Computer Supermicro P4D6C+

    1. Dual Intel® Xeon® processors 1.5 GHz
    2. Intel® 860 Chipset
    3. 512MB 600/800MHz RDRAM
    4. 1x Intel® 82559 10/100 Ethernet Controller
    5. Adaptec AIC-7899W Dual-Channel Ultra160 SCSI
    6. 2 x 64-bit PCI expansion slots
    7. 4xAGP Pro slot, nVidia GeForce FX 5500 AGP video card
    8. Award® BIOS
          DATED yes...
data sheet.ods

Sheet 1
    . . . is aprox 3 - 5 days, ONLY A GUESS... FORGOT HOW LONG THE SYSTEM WAS RUNNING before I had to do a reboot. due to instability problems with capturing data when trying to set the delay. . . eventually left with the standard 1 second capture so 1 line = one second .. appended CONKY with $timeup
    600s = 10 minutes aprox....

Sheet 2
    just after a reboot, 5 minutes 30 seconds aprox.

Sheet 3
    after reboot 2 hours 20 seconds aprox.

[mlx (myxal-mxl)]

@ Luis (#198): can you please post/link to the changes made to fstab? I have no lines in fstab for USB drives, as DeviceKit is supposed to take care of that.

@everyone having problems and still following: how many of you tried to bypass the filesystem by using dd or hdparm -t?

Running Karmic amd64, I'm having ridiculously slow speeds just reading from 3.5" HDD over USB (in the 3-8 MB/s range), right after bootup, using hdparm -t or dd'ing from to /dev/null:
$ sudo dd if=/dev/sdb of=/dev/null count=1024 bs=32k
1024+0 records in
1024+0 records out
33554432 bytes (34 MB) copied, 7,03421 s, 4,8 MB/s
$ sudo hdparm -t /dev/sdb

/dev/sdb:
 Timing buffered disk reads: 18 MB in 3.29 seconds = 5.47 MB/sec

Hardware is Intel ICH8, the same gives me over 12MiB/s in Windows NT 5.2 and Hardy i386. The hard drive puts over 40 MiB/s over ExpressCard->eSATA Silicon Image SiI 3132

[Carlos M (morenoce)]

From what I googled, it appears the 'noatime' option implies 'nodiratime' as well. I was able to specify this option in the USB->Properties->Volume->Mount options. It seemed to work when I transfered two 4.2GB movies but I'll keep an eye on it -- I'm not sure it's fixed in my case. Also, this issue is showing up in other distro's as well (fedora, arch linux, opensuse). Just google 'usb file copy very slow'. Some users in these posts think it's a kernel issue, but I don't know enough to say for sure. Coincidentally, I have 2 x64 machines - one with 8.10 (older box with the original installation) and one with 9.04 (newer box with all the latest updates). Interestingly enough, I've never seen this issue in the 8.10 box and I've been running both of them for months.

Carlos M

[Sergey V. Udaltsov (sergey-udaltsov)]

Carlos, where did you find USB->Properties->Volume->Mount menu?

[Carlos M (morenoce)]

Sergey
Once the USB thumbdrive is mounted, you can right-click on its desktop icon and select Properties. From there you will see the Volume tab and specify the mount options
Carlos M

[Shawn Reist (icabaud)]

I do not see this option in karmic

Carlos M wrote:
> Sergey
> Once the USB thumbdrive is mounted, you can right-click on its desktop icon and select Properties. From there you will see the Volume tab and specify the mount options
> Carlos M
>
>

[Sergey V. Udaltsov (sergey-udaltsov)]

I do not see this option in karmic, either

[Roozbeh Gholizadeh (roozbehid)]

Anything new on this bug?
i am having this issue with my usb :
Bus 001 Device 007: ID 058f:6387 Alcor Micro Corp. Transcend JetFlash Flash Drive

copying 170mg file after 70% tends to slow down.using krusador and also natilus.
i am willing to provide any other info if required.

using ubuntu 9.10 kernel 2.6.31-15-generic

[boratsuckdev (boratsuckdev1)]

I am using Karmicl, kernel 2.6.31 -15 -generic on my laptop. The time that takes to transfer files to my USB drives takes almost 20 times more than it used to take in Jaunty. Sometimes after waiting for a long time of transfering it will simply give a error message or the file will simply will not get transferred. Tried to transfer a 700MB (90 mins) avi file, i had to wait for almost 20 minutes for it to get transferred and an average speed of 200 kpbs was the maximum achieved. same complaints as the person above....plz help

[uMac (uvaio)]

I see this bug is very old but still here and painful for many people, is there anything we can do? provide some more info? is there anyone looking at it? Was this problem understood already?
I think priority should be risen, what should we do when we need to take larger file on USB? what is workaround? to use different OS for this purpose? Isn't issue that make people think to bin Ubuntu good reason to give higher priority than medium?

[mlx (myxal-mxl)]

I think the bug along with the reports needs to be split into:
>kernel issues - when even reading from/writing to the block device with hdparm/dd is unreasonably slow
>block filesystem issues - testing the above works,but copying files to/from the filesystem (preferably from something not impacted by random unexpected slowdowns, such as tmpfs) using CLI (cp/mv) is slow
>VFS/DE issues - the above works ok but using the gnome/kde is slow for whatever reason - KIO/GVFS bugs, choosing wrong block sizes, etc.

[Filofel (filofel)]

Same problem here copying 100MB from the disk to an USB key using Nautilus (drag n drop from Nautilus to Nautilus).

During the copy, the gnome desktop is mostly frozen, the copy progress bar only appears after after *minutes* and once appeared, is hardly refreshed at all before the end of the copy.
During that time, the desktop is totally unresponsive.
Karmic 2.6.31-15 and 16.

OTOH, reading from the key writing to the disk is perfectly normal, both in term of speed and behavior of the progress bar, desktopn etc. All cache effect has being eliminated (the data copied was not already in the cache).

I never saw anything comparable under Jaunty same hardware (key included).

[mlx (myxal-mxl)]

Latest kernel update (2.6.31-16-generic #53-Ubuntu x86_64) fixed my problem.

[Filofel (filofel)]

2.6.31-16-generic #53-Ubuntu x86_64 fixed my problem too.

[sbec67 (sbec)]

i have the same problem here....
read this: http://ubuntuforums.org/showthread.php?t=1306333

people are getting angry...
i think this Problem should be fixed with a Bug Fix and not within next release !!!!

Kind regards
Sbec

[rom85 (rom85)]

this problem lasts for me since 8.04 when i just started using linux. i have it by now and none of the proposed solutions helped

[uMac (uvaio)]

2.6.31-16-generic #53-Ubuntu x86_64 Did not fix anything for me ;o(

I did some further testing and the problem appears to be related to per file transfer and also to USB stick you use.

I did test with
1. Kingston micro SD HC 8GB memory card using with mini usb card reader.
    trasfered about 500MB of 700MB file then speed decreased to very very low almost stopped.
2. Kingston 8GB Traveler USB memory stick
    Transfered 700 MB file quickly but in the very end it took 2 minutes to complete transfer even it shown all 700MB copied.
    Then I tried 3 files of 700MB each in batch copy. And result is interesting. Overall speed was linearly decreasing from about 20MB/s to final 4 MB/s. After each about 700MB of transfer it got in about 2 minutes wait. Like copied 700, then progress didn't move for a while at all, copied another 700MB and then progress stayed at 1.4GB for another about 2-3 minutes. then last file transfered and waited again at the end.
I think the overall speed (I guess measured as average) may be decreasing as result of waiting at the end of file transfer.
I think it's doing the same thing with both memory cards it just does happen after different amount of data transfered.

[sbec67 (sbec)]

I also did some Tests...
Copying file to a MP3 Player ( FS MSDOS ) i had similar Problems as described above.
Copying Files to a external USB Hardisc ( FS EXT3 ) I don't see such problems.
Maybe the coses come from different File Systems ?

Regards Sbec

[Angelos Vlassopoulos (avlass)]

I have been following this bug report for a long time now and here is some advice to the people that have only recently begun to post here.

This bug has hundreds of postings and no reasonable person could possible read all of them in order to start fixing something.

Through time it has become obvious that this is not one single bug. It is very possible that there several bugs that each leads to reduced speed under different circumstances.

If you read several comments back, you will discover that some people tried to fix this bug, but they were unable to reproduce it. Apparently it is not a Dolphin or Nautilus issue and in any case you should start excluding stuff so that you can find the heart of the problem.

So, many postings earlier there was an agreement that people should stop posting to this bug. Instead, people should start new bug reports and post specific data that could be useful to bug fixers.

The best way to provide bug fixers with useful data is to follow the instructions that appear here.

https://help.ubuntu.com/community/DiskPerformance

If you have the time and you want to help, please conduct some tests and test your theory about what affects the USB performance. If you think you discover something, please post a new bug report.

[Digital5700 (edward-edwardh)]

sbec67: The difference isn't the file systems, it's that hard drives are treated differently to USB sticks. As I wrote before... copying to external drives gives me 15MB/s sustained.

[Jeff Wallace (tjwallace)]

I can concur with Digital5700. My 1TB external drive has no performance
issues, but my 8GB flash drive has trouble getting a 700MB file onto it in
a reasonable amount of time.

[Carlos M (morenoce)]

After playing around with the US mount options in my jaunty 64, I still see this issue.

In my case, I think this is the result of using ubuntu on an NTFS USB drive: If I copy and delete large files (4-8GB) in Jaunty, the USB drive gets fragmented (expectedly so). If I plug this device to my Windows box, it is defragmented by default (in vista or 7).
When I use Jaunty again to copy large files, I don't see this problem again.

[Botond Szász (boteeka)]

I have done some quick testing. I formatted my Kingston DataTraveler G2 16GB drive as a FAT32, FAT32 (lba), EXT2, EXT3 disk one at a time and tried copying over a file little over 1GB in size. I monitored the actual disk operations with dstat.

What I have found is that the first 100 - 200MB for which the copying speed is normal, does not actually get written to the disk. Up to the point when Nautilus reports that 100 - 200MB is copied, dstat shows that no writing takes place - which is (let's say) OK, because of the caching. After this the operation starts to slow down and at this same time data start to get written to the disk but with an incredibly slow pace (< 100KB/s). If I do not cancel the operation Nautilus starts to become unresponsive (dims out then back in - because of compiz).

This is what dstat shows (this is for the test with EXT3 but others are similar):

boteeka@kryzisworx:~$ dstat -D sdb1
----total-cpu-usage---- --dsk/sdb1- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
 14 5 64 16 0 0| 147B 32k| 0 0 |2485B 16k|1000 2158
 12 4 83 0 0 0| 0 0 |2473B 2763B| 0 0 | 813 1504
 10 4 85 0 0 0| 0 0 |1447B 1574B| 0 0 | 863 1561
 12 3 84 0 0 0| 0 0 |5631B 4587B| 0 0 | 749 1463
  5 4 89 0 1 0| 0 0 |4600B 4273B| 0 0 | 814 1255
  8 7 83 2 0 0| 0 0 |2068B 3283B| 0 0 | 877 1733
 11 4 84 0 0 0| 0 0 |3703B 4359B| 0 0 |1015 1751
 14 6 41 37 2 0|8192B 0 |2015B 3422B| 0 0 |1235 2340
 12 16 43 27 1 1| 0 0 |3336B 4627B| 0 0 |1595 2560
 15 14 30 40 0 0| 0 0 |1398B 1492B| 0 0 |1456 2945
 15 14 36 34 0 1| 0 0 |1267B 915B| 0 0 |1472 2756
 15 15 39 29 0 1|4096B 0 | 569B 712B| 0 0 |1449 2680
 15 18 26 40 0 0| 0 0 | 132B 168B| 0 0 |1550 2943
 18 14 24 43 0 0| 0 52k| 673B 985B| 0 0 |1391 2520
 11 8 2 79 1 0| 0 1920k| 144B 746B| 0 0 | 967 1751
 11 5 0 83 1 0| 0 4224k|2211B 2500B| 0 0 |1085 1960
  8 9 0 81 1 0| 0 4100k| 234B 504B| 0 0 |1178 2133
 10 4 0 85 0 0| 0 2020k| 964B 778B| 0 0 | 866 1538
  7 7 0 85 0 0| 0 28k| 764B 1410B| 0 0 | 785 1475
  6 7 27 59 0 0| 0 568k|1354B 1696B| 0 0 | 963 1605
 13 8 8 70 1 1| 0 52k|1579B 1925B| 0 0 |1236 2144
 13 4 0 83 0 0| 0 56k|1811B 3758B| 0 0 | 829 1473
 10 5 0 84 1 1| 0 80k|1700B 3975B| 32k 80k| 900 1508
----total-cpu-usage---- --dsk/sdb1- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
  9 7 0 84 0 0| 0 88k|2122B 3456B| 0 0 | 953 1539
  8 7 0 84 0 0| 0 56k| 546B 731B| 0 0 | 845 1478
 11 8 0 80 0 0| 0 40k| 12k 17k| 0 0 |1018 1631
 13 3 0 83 0 0| 0 60k|4218B 12k| 0 0 |1027 1861
 12 3 0 84 1 0| 0 28k|1230B 3793B...

[Botond Szász (boteeka)]

One more thing...

I don't know if this is related, but whenever I try to "Safely remove" the disk I always got this error:

Unable to stop drive

Error detaching: helper exited with exit code 1: sense buffer empty
Error SYNCHRONIZE CACHE for /dev/sdb: Success
start stop unit: transport: Host_status=0x07 [DID_ERROR]
Driver_status=0x00 [DRIVER_OK, SUGGEST_OK]

Error STOP UNIT for /dev/sdb: No such file or directory

[Theodore Ts'o (tytso)]

Botond,

Again, I would ask that you take a look at the procedures found here:

https://help.ubuntu.com/community/DiskPerformance

... and then open another bug, complete with details of kernel version, dstat -D <dev> when doing a "dd if=/dev/zero of=<dev> bs=32k", and then the output of "dmesg | grep usb" so we can see the usb-related messages, etc.

I just did my own testing, using a 2.6.32-git6 based kernel (so it has a lot of freshly merged ext4 improvements), and what I can see when copying a large file to an Aptiva 2GB USB stick (purchased from Microcenter; identifies as a Sandisk device via lsusb -v), is a slowdown when doing a cp bigfile /mnt, when copying to ext2 and ext3, sometimes to a crawl (4k/second according to dstat), but when a I do dd if=/dev/zero of=/dev/sdc, I see 6MB/sec. I see the same 6MB/sec while mke2fs is zeroing out the inode table, and if I copy the large file using ext4, I see 6MB/sec (very rarely it drops down to 3-5MB/sec, but only for a second). When the same USB stick is formatted using vfat, I see a wide range of write bandwidths, from 3-6MB/sec, with an average of maybe 4-5 MB/sec. My hardware is a Lenovo T400 with 4GB of memory, and the writes to the disk start showing up with dstat after at most a 1 second after I hit the return key to initiate the cp from the shell prompt.

So as you can see, these are very different results than what you have gotten. I get very similar (although perhaps slightly degraded) results as far as write speends are concerned when I mount the ext2 or ext3 file system using ext4 by the way. And I can consistently replicate these results, which indicates that at least on my system, raw sequential writes (i.e., via dd if=/dev/zero ...) are 6MB/sec, and using the ext4 file system driver gets me close the same raw sequential write rate, regardless of whether the file system is formatted using ext2, ext3, or ext4 (although things are a bit worse when extents are disabled). I am getting perhaps 75-80% of the raw write speed when using vfat to a USB stick, consistently over writing a 1GB file to a 2GB USB stick. With ext2 and ext3, I lost patience before the file copy completed, but after 10 seconds or so with ext2, and perhaps 30 seconds with ext3, the write speeds as reported by dstat -D had dropped down to 4-8 kb/sec, and it would stay there for a few minutes, and then suddenly pop back up to around 3-4 mb/sec, only to later (after some time had passed) drop back down to 4-8 kb/sec.

The bottom line is that everyone is reporting slightly different symptoms, and so opening separate bug reports, with detailed information is the only hope of figuring out what is going on. I am going to posit that some people are losing because of bad interactions between the kernel and their USB controller (which should show up when they see bad results with raw writes to the device). Others may be seeing interesting interactions between the write order and size of the writes by a particular file system driver and their particular USB stick. Depending on the sophistication of the flash controller on the USB stick, the USB stick may handle small (4k) writes much less efficiently than ...

[Przemek K. (azrael)]

I've did some test and I managed to narrow this bug somehow.
My setup: Macbook 2,1; Intel USB controller; Ubuntu 9.10 64-bit.
1st try: Patriot Xporter 4GB flash drive - no problems.
2nd try: Patriot Xporter 16GB drive - slows down when writing to USB drive under Gnome.
It worked fine when I booted to recovery mode (pure CLI, no Gnome), mounted the drive manually and copied over 700MB of data.
It slowed down in Gnome, both with automounting, and with manual mounting (though this time I can't guarantee that Gnome wasn't involved in that because I just remounted the drive manually after the Gnome's autodetection).
So it seems that some subsystem invoked by Gnome (hal? udev? gvfs?) is responsible for the slowdown.
Please tell me if this is relevant to this bug or should I file a new one?

I'm attaching the output of dstat ran while copying in Gnome.

[Przemek K. (azrael)]

Here's my dmesg.
Please note that I also have a USB 3.5" hard disk and it works fine. The problem is only with the pendrive.

[Przemek K. (azrael)]

Here's dstat run in recovery mode - everything is fine.

[Przemek K. (azrael)]

And here's dstat ran when I tried manual mounting while in Gnome, but it might be the same as automount because I just remounted the drive manually after an automounting in Gnome.

[Yann Papouin (yann-papouin)]

@Przemysław Kulczycki: It's not because your usb hard drive is working fine that it's the same for others (in my case my LACIE 3.5" 160Go is slowing down).

And with more than 200 comments, we can see that everyone has the same symptoms but different causes. This bug report is not usefull anymore due to its life time and the quantity of reports.

It seems a low level issue, lower than "cp" and "move" commands. Maybe a Kernel or Kernel compilation issue?

[slumbergod (slumbergod)]

I've endured painfully slow file transfer with USB flash drives ever since Hardy. It has never improved. It has never been resolved. I realise this issue doesn't affect everyone but it has annoyed me for almost my entire two years with linux. During that time I have used dozens of different USB flash drives (some good brands, some cheap) and across different hardware. I had hoped my latest laptop would be different but no, it is still pathetic. 1-2 Mb/s is hopeless. If I boot into Windows it is always significantly faster i.e. the expected USB 2.0 speed for file transfer.

PLEASE PLEASE PLEASE place some importance on this. It has been an issue for far too long.

[Simon Holm (odie-cs)]

søn, 20 12 2009 kl. 20:34 +0000, skrev Przemysław Kulczycki:
> So it seems that some subsystem invoked by Gnome (hal? udev? gvfs?)
> is responsible for the slowdown.
> Please tell me if this is relevant to this bug or should I file a new one?

Everyone should open new reports, you should too.

Your problem seems related to mount options, try to find the specific
option that makes the difference in performance such that you can mount
your disk manually from cli and consistently reproduce good and bad
numbers depending on which options you mount your disk with.

I've updated https://help.ubuntu.com/community/DiskPerformance to
include the part "Mounting disk manually gives better performance than
auto-mounting and semi-automounting" to address your problem. Please see
that for more details.

To Botond, follow the instructions by Theodore Ts'o and
https://help.ubuntu.com/community/DiskPerformance for using dd.

slumbergod and everyone else, see
https://help.ubuntu.com/community/DiskPerformance. After careful study
and if there is still a problem, open a new bug and include a reference
to it in this bug.

[Torandi (torandi)]

I have been experimenting some with this problem today. I have been using rsync to copy the files (to get a progress bar), and I noticed that rsync stops updating in periods when this bug is in effect:
I first get high transfer speed for a couple of minutes, after this the transfter speed drop down to 100-200kb/s and during this rsync only updates approximately once every half minute.
Loading the module xhci seemed to delay the problem for a couple of minutes.

[ktp (kari-petersen)]

I'm pretty sure that comment 177 will help most of you all. This bug is probably a duplicate of #381300
See https://bugs.launchpad.net/ubuntu/+source/linux/+bug/381300 for more information.

Try to set your I/O scheduler to noop and see if transfer rates stay constant.
To do this follow these steps:

sudo gedit /boot/grub/menu.lst
search for the line begining with #kopt and add "elevator=noop" without the quotes to the end of the line
save & exit
sudo update-grub
reboot

HTH, ktp.

[Julian Lam (julian-lam)]

Thanks ktp. I've applied the change, I'll let you know how it goes.

[RishiRamraj (thereisnocowlevel)]

I tried elevator=noop with no success. I've also tried removing the sync option as per the following with no success:
http://en.opensuse.org/SDB_Discussion:Automounting_without_the_sync_Option

My only symptom so far has been slow data transfer on usb keys and usb external hard drives. I don't notice the degrading speed over time, or any effects to my network speed (only tested scp). I'm running a 64 bit AMD processor with 8 gigs memory on an updated cut of 9.10.

I've attached my dmesg log. I'll also try and run some performance tests.

Thanks for the help,
- Rishi

[Murz (murznn)]

Setting elevator=noop solve the problem for me, the write speed to usb stick is stable, about 5-6 mb/sec all the time. But now sometimes I see the HDD performance problems while system write to usb.
Maybe try other elevator types?
Completely Fair Queuing — elevator=cfq
Deadline — elevator=deadline
NOOP — elevator=noop
Anticipatory — elevator=as

[Digital5700 (edward-edwardh)]

elevator=noop solved it for me also, but just as murz says: sometimes under heavy disk activity (dd iso -> empty sata drive) some parts of Xorg will pause, sometimes for several minutes.

I can also see that noop writes to the usb drives without pausing. Normal elevator writes for a few seconds, pause for 1.5 seconds and then keep writing. Noop writes without pauses.

[alex123 (stry90dis)]

I just had this problem re-appaer after I fixed it, see my commentsin bug "https://bugs.launchpad.net/ubuntu/+source/linux/+bug/334914". Still running with same hardware. Now I'm on 9.10 with kernel:

alex@alex-desktop:~$ uname -r
2.6.31-17-generic

After I installed 9.10 with its default kernel on release day, I remember to have made backups to my 1TB USB2.0 drive that went at expected speeds. Today I wanted to backup again and it started off fast at ~35MB/s in nautilus but went to ~2MB/s after ~400MB was copied. "dstat showed it was writing at ~1.5MB/s.

The thing to note here is I connected it to the front-panel of my computer case which is connected to the motherboard. The case is an Antec 900, "http://www.antec.com/Believe_it/product.php?id=MTEyOQ==". Bought it 2 years ago so it may have undergone some revisions.
Next I connected the USB drive to the backpanel (USB port directly on motherboard) and now it had normal speeds again, dstat ~30MB/s-40MB/s.
The front panel has given me some troubles before since it seems that the slots on it do not always make a good electrical connection, resulting in the device not being recognised or disappearing. But my USB drive stayed reliably connected, just at slow speeds.
So now I'm wondering if there may be a technical difference in the front- and backpanel connections that limit the speed.

[Simon Holm (odie-cs)]

RishiRamraj:

You can see https://help.ubuntu.com/community/DiskPerformance for general advice on testing performance of a disk and reporting your issue. Your snip from dmesg tells that you plugged in a USB 2.0 device, but that is about what can be said from that information, so further performance tests are certainly required.

Murz and Digital5700:

Your issues with noop as a much better scheduler than cfq are very interesting, but I'm not sure what the best way forward is, bug #381300 and #131094 already exists without really going anywhere. If you can make a good report with dstat numbers etc. and with the newest upstream kernel compiled from git there should be a good chance of getting Jens Axboe looking into the issue (he's the upstream kernel developer that maintains the I/O scheduler subsystem).

alex123:
You should verify that your front panel USB connections are not USB 1.1, see https://help.ubuntu.com/community/DiskPerformance.

[Murz (murznn)]

Simon Holm Thøgersen, I can try to do the reports, can you give me some step-by-step instructions and examples how I can do it better?
I can try to install new kernel and do some tests, if it will not too hard and long process.
At now I try deadline scheduler and see the problem again after copying about 1.5gb to the flash. At start when copying first gb the speed was about 5-6 mb/sec, but after 1.5gb slows down to 0.2-1.5 mb.
I have attach part of output "iostat -m 1" command at ~1.6-1.7 gb copied and speed is slow down in deadline scheduler.

P.S. I have found an easy way yo check current scheduler and switch it without reboot at http://www.cyberciti.biz/faq/linux-change-io-scheduler-for-harddisk/:
View Current Disk scheduler
# cat /sys/block/{DEVICE-NAME}/queue/scheduler
# cat /sys/block/sda/queue/scheduler
Sample output:
noop anticipatory deadline [cfq]

Set I/O Scheduler For A Hard Disk
To set a specific scheduler, simply type the command as follows:
# echo {SCHEDULER-NAME} > /sys/block/{DEVICE-NAME}/queue/scheduler

[u356 (pavelkok)]

Murz, thanks! Change queue to noop solve my problem without reboot.
Now I have stable 8-9 Mbyte/sec with SD class 6 card.
This trouble affected my desktop (64 bit) and notebook (32 bit) Ubuntu 9.10 installs.

[u356 (pavelkok)]

Update
After 10 Gbyte transfer to sd card speed is down again. Scheduler fix is not enought for me.

May be it would help for someone - http://ubuntuforums.org/showpost.php?p=8653721&postcount=29
"yes its fixed, but not in the 2.6.31 kernel branch. I experienced the same problem you described when trying to copy files from my internal hard disk to either a usb external hard disk or a usb thumbdrive. the speed would start at about 30 MB/s then quickly drop down to 5 MB/s and sometimes get down to even 1.5 MB/s. I thought it was pretty funny that i could download files from the Internet at 2.6 MB/s but couldnt even transfer them via usb that fast. It also spiked my CPU and caused my wifi to hang and disconnect as well.

Last night I installed the Linux 2.6.33RC3 kernel from:
http://kernel.ubuntu.com/~kernel-ppa...e/v2.6.33-rc3/

I ran some tests and found that my USB file transfers ran at full speed and did not drop like before. I should point out that I am running the 64bit kernel so i dont know if this issue exists with the 32bit kernel or not."

[Marc MAURICE (dooblem)]

I did all the following tests, and always the same problem : speed decrease to 2-3 MB/s.

Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.31-17-generic + Nautilus
Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.31-17-generic + rsync
Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.33-rc6 + Nautilus
Thinkpad T42p + Ubuntu 9.10 + Linux 2.6.33-rc6 + rsync

Thinkpad X31 + Archlinux + Linux 2.6.32 + Dolphin
Thinkpad X31 + Archlinux + Linux 2.6.32 + rsync

[tonyo (toverfield)]

I have this problem too. HP Pavilion (Core 2 Quad, 3 GB RAM, 2.6.31-17-generic x86_64)

I was unable to use dd to write a memstick image to a Sandisk USB stick. It was crawling to a virtual stop before reaching 40% complete. I hope my observations will help to solve this problem.

My first observation was that I could use dstat output to anticipate the slow down and then press "Ctrl-Z" to suspend the dd so that the write buffers could drain. Carefully nursing the transfer along resulted in a successful and quick copy.

My other observation was that I could work around this problem using the "oflag=direct" option to dd, which presumably bypasses the lazy write buffering.

# dd if=sandisku3.dd of=/dev/sdh bs=256k oflag=direct
7839+1 records in
7839+1 records out
2055021056 bytes (2.1 GB) copied, 217.913 s, 9.4 MB/s

I think these observations suggest that the extremely high write backlog (over 700MB "Dirty" in /proc/meminfo) causes a performance fall-over when available memory resources become exhausted.

[Mitch Towner (kermiac)]

Setting this task back to invalid.

@ Fail2Ban: Please do not change the status of tasks without commenting to advise why you have done this.
Thanks in advance!

[gdneil (nsy999)]

Hello.... Just commenting on the same problem - which for me also appeared with external USB hard drives.... Transfer rate slowing down with large files.

Tried all of the suggestions...

-Starting with pci=routeirq helped a little maybe?
-Turning off image preview in Nautilus might have helped a little?
-Reformatting externals (flash and HDs) to ntfs instead of fat helped the most
-Recompiling the latest ntfs-3g helped a little more

And then in frustration, I installed Debian(Squeeze) on an extra partition and timed transfers of big files of different types (fat32, ntfs, ext3, ext4) and found that Debian was much much much faster in all cases. Unfortunately though, I seem to have lost the notes with my actual timings.... Not sure if this helps, but hope it might... the problem (for me at least) seems to be coming from the Ubuntu side...

I've used Ubuntu since 8.10 and it has always been slower at transferring files than Windows, but 9.10 was significantly worse. Kind of useable with the mods above, but nowhere near debian. Tried putting some of the relevant debian packages into my ubuntu installation, but no luck (and not enough knowledge/experience in these things I admit) but maybe a possible direction? For now though, finding myself more often booting Debian.

All the best,
-Neil

Toshiba Libretto U100
1gb RAM, lots of external HDs

[kasrawis (kasrawis)]

same problem here :( seems to be bugs in kernal, slow copying :(

[RishiRamraj (thereisnocowlevel)]

gdneil, kasrawis:

Have you tried setting elevator=noop as per post #233?

all:
I suspect that there might be no singular cause for this problem. If I find the time, I'll try to weed through all of the bug reports (including duplicates) and identify the suspected causes, platforms affected and proposed workarounds.

[Digital5700 (edward-edwardh)]

echo 10 > dirty_writeback_centisecs
echo 10 > dirty_expire_centisecs

Now it should basically write to the stick continously instead of every 3000 centisecs (30 secs).

I learned when I found that transferring terabytes through my gigabit results in 30s long pauses, then 80mb/s for a few seconds and then another pause.

The same principle should be applicable to usb sticks, although it won't explain why usb harddrives are much faster...

[Pi (sub2008-deactivatedaccount-deactivatedaccount)]

I believe this is the same bug, I asked in the ubuntuforums.org and someone told me to post my discoveries here:

Whenever I plug-in my USB 8gb pendrive to copy a large file, it takes eons.

Using Ubuntu 9.10 for AMD64.

At first it copies a file at high speeds, because it's not actually writing to the USB but to the cache; at 500-700mb, the write cache is full and flushes, *while* still copying the rest of the file. Since the system is trying to access the USB in two different places at the same time, the transfer speed bunks down, often to 200-300kb/s. The final process, instead of taking 2 minutes, takes one hour. This continues until 1.4gb, when the cache finishes the flush, the copy can continue streamlined and the last 200-300mb are copied again at quicker speeds.

I've made a test, copying a 1.6gb file to the USB and waiting for the cache to get full. Once it happens, I copy another file of the same size to the USB at the same time, when it starts copying I cancel the previous copy - cancelling it could take 1-2 minutes, but once it's cancelled, the second copy goes amazingly quick since the pendrive doesn't have any access troubles anymore. However, that means for the trick to work that I can't copy four 1.6 files to the 8gb pendrive, but only three. The fourth one would need an hour unless I start copying it just before the first three finish so the write cache is still not in use or something like that.

IMHO the problem is using the cache in certain USB drives. For example the same problem happens with my usb-connected HDD; but due to its faster access time, the slowdown is less noticeable and disappears quickly. The low access time in usb pendrives makes the slowdown due to the cache to last for too long to be bearable.

I hope this clarifies the situation; sorry if this has been mentioned before but I didn't have time to check such a long bug report before adding my own experience.

Is there simply a method to disable cache (or lower it to a minimum) to an already mounted device?

[Ulrich Lukas (ulrich-lukas)]

Pi, this is consistent with my observations.

[Olaf Geibig (olafgeibig)]

So it worked for me. I added these two parameters to my kopt line in /boot/grub/menu.lst as described in post #233

elevator=noop pci=routeirq

It's still not a speed king but it doesn't take ages now.

[Angelos Vlassopoulos (avlass)]

How can I do these changes on Karmic's Grub2? There is no menu.lst anymore

[mlx (myxal-mxl)]

Grub now uses binary configuration format, which has to be generated
from a text file. The text file is /etc/default/grub. Edit it, then
run update-grub.

[kwstas tsob (grandamra)]

hi guys,

i thought i could post here something i found out..

i had similar issues but also when i was trying to copy a file(around 50-700mb or larger) from my local hdd to another folder of the same hdd!!!
so what i found out was that this was happening only in files that i had downloaded with "transmission"!
i tried copying with files i downloaded with "firefox" or "qbittorrent" and until now i have no problem!

hope it helps some of you

[Theodore Ts'o (tytso)]

>i had similar issues but also when i was trying to copy a file(around 50-700mb or larger) from my
>local hdd to another folder of the same hdd!!!
>so what i found out was that this was happening only in files that i had downloaded with "transmission"!

What file system were you using on your hdd, and what version of Ubuntu are you using?

This isn't surprising, because bittorrent downloads tend to write the file in random order (depending on what is made available from the bt seeders and the bt peers) and in multiple threads. Using ext4 and a relatively modern kernel, it won't be too bad, because ext4 has better allocation algorithms than ext3. It's best though, if the bt client uses fallocate(), which is I'll bet what qbittorrent is doing.

So this is a useful bit of information, but I suspect some users have different causes for their problems, which is one of the reasons why most kernel developers have refused to use Launchpad. Too many ubuntu users are glomming onto a bug report, and they post different performance problems all in a single bug report, without bothering to give full information about their hardware, software versions, what file system they are using, etc. And then on top of that Launchpad is slow as a dog when there it gets too long.

Which is why this bug is marked invalid and why I beg people, if they have performance problems, to open new bugs and to give full details about their hardware configuration, what version of Ubuntu they are running, the exact specifics of the USB device, etc. Otherwise this can be a great place for people to vent, but most people who aren't paid to pay attention to dysfunctional bug forums will simply say, "Beam me up Scotty, there's no intelligent life here".

[allisterbrizan (allisterbrizan)]

Please don't brush this aside as "poor performance". This is an issue in long disk activities. I notice this bug in both disk copies and dd.
I use dd quite frequently to clone disk images onto identical computers and very regularly have to copy these 75.4GB disk images. While i have not done a forensic investigation this is what i have noticed after facing this issue MANY times:

1. It definitely affects Ububtu 9.10, Ububtu 9.04, Ububtu 8.10, Ububtu 9.10UNR, Ububtu 9.10UNR. (It seems to be a bigger problem UNR distros)

2. In Ubuntu 9.04 i get this slowdown in approximately 25% of the dd clones i do (2 out of 7 computers this weekend). Canceling and restarting the computer don't fix this issue.
Note: For dd clones i boot off a USB stick and invoke dd in a terminal window. The drive images copy from a file on a NTFS drive to /dev/sda or vice-versa.

3. Using the computer makes this bug less likely to happen. (I truly believe this has something to do with efficient power consumption or reducing disk writes.)

4. The read/write rate decreases over time in a predictable manner. A dd clone which works normally copies the 75.4GB file in 3054s give or take 10. If the bug occurs it takes approx. 23200s.

This is a real problem. I will do some proper testing and analysis based on suggestion.

[Theodore Ts'o (tytso)]

@allisterbrizan,

Please, do the investigation, and then open a new bug!!!

Your issue with dd clones is completely different from kwstas tsob's problem in #256 which had to do with source file fragmentation because of bittorrent.

You see? Both are performance problems, and both are complete different!!

And discussing them in the same bug report in Launchpad is completely unproductive!!!

So the issue is not writing off performance problems; just this specific Launchpad bug, which is clearly degenerated beyond any hope of help. (It doesn't help that launchpad is very slow when there are over 80 comments in the bug report.)

If you want to have any hope of anyone paying attention, OPEN A NEW BUG.

[kwstas tsob (grandamra)]

Theodore Ts'o

as i said in my first post i only posted this so i could help others understand better the problem. i had already posted to the forum of transmission about it...

i'm on karmic 9.10 with ext3.

[kwstas tsob (grandamra)]

it seems that i solved the problem by changing the preallocation number of transmission's conf file from 1 to 2.

[Zordid (zordid-gmx)]

This bug is the most annoying, most critical and nerve-wracking one in the whole Ubuntu universe!
This makes me hate my beloved Ubuntu cause I am not able to use ANY USB-connected memory device! I can also constitute that when copying large files (>100MB) to any USB device (memory stick, hard drive, does not matter...) the transfer starts "normal" with a couple of MB/sec, everything's normal - but then, after 100 MB have been transfered, the speed drops dramatically down to a few KB/sec - when copying 2 GB files, I sometimes get the remaining time up to over 1 hour!!

What I can see then, is that the load of the system rises to 6 or even 8 - only because of that file transfer going on. When the copy finally is done, the system behaves normal again and the load is where it belongs, 0.something or so.

This problem has been there for quite some time now, at least the last 1,5 to 2 years! I am not sure though when it first hit me...

I am using the 64bit Ubuntu 9.10 currently on a Dell Latitude D830.

Is there ANYBODY out there who knows how to fix this?? I don't want to have to use Windows just because file copying is so damn slow! Also, I am always kind of in a trouble when I want to copy files for friends: when they see how long it takes I cannot find a single positive argument for them to switch to Ubuntu anymore!! :-(

[zvaral (z-varallyay)]

This issue seems to be quite old and still exist in Ubuntu Lucid Lynx 10.04 beta.

(Bug #541937 is a duplicate of this bug report but I reported it because this problem occurred in a newer Ubuntu release mentioned here.)

[cheffe76 (p-letna)]

The priority should be changed to high. This is a BIG issue, someone capable should fix it.

Bug description:
Ubuntu 9.10 -> Slow USB transfer speed, while copying (starts with high speed for about 150m, then speed drops to <1 MB/s.

Tried it with several USB sticks. Did not try the terminal yet.

Cheers

[Timm (tbaumeister)]

Similar problems occur when copying large files from a virtualbox guest to the ubuntu host (see Bug #557796) While this may be entirely unrelated, I am posting this because both problems occur on my system and have similar symptoms.

[adri58 (adri58)]

El 08/04/2010 5:58, Timm escribió:
> Similar problems occur when copying large files from a virtualbox guest
> to the ubuntu host (see Bug #557796) While this may be entirely
> unrelated, I am posting this because both problems occur on my system
> and have similar symptoms.
>
>
But that problem is from virtalbox, i think, not linux

[David Malizia (david-malizia)]

Same Problem.

I use VMWARE player and it can take control of USB devices. If I insert any pendrive Windows 2003 through VMWARE recognizes it as a USB 1.1 device. If I try the same trick booting my PC from windows the pendrive is recognized as USB 2.0 both by the host windows XP and the vmware windows 2003.

$uname -a
Linux ubuntu-best 2.6.31-20-generic-pae #58-Ubuntu SMP Fri Mar 12 06:25:51 UTC 2010 i686 GNU/Linux

$lsusb

Bus 006 Device 002: ID 046d:c521 Logitech, Inc. MX620 Laser Cordless Mouse
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 004: ID 0781:5406 SanDisk Corp. Cruzer Micro 1/2/4GB Flash Drive
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 002: ID 04f2:b064 Chicony Electronics Co., Ltd
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 002: ID 08ff:1600 AuthenTec, Inc. AES1600
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

$ lsusb -vv -s 002:004

Bus 002 Device 004: ID 0781:5406 SanDisk Corp. Cruzer Micro 1/2/4GB Flash Drive
Device Descriptor:
  bLength 18
  bDescriptorType 1
  bcdUSB 2.00
  bDeviceClass 0 (Defined at Interface level)
  bDeviceSubClass 0
  bDeviceProtocol 0
  bMaxPacketSize0 64
  idVendor 0x0781 SanDisk Corp.
  idProduct 0x5406 Cruzer Micro 1/2/4GB Flash Drive
  bcdDevice 2.00
  iManufacturer 1
  iProduct 2
  iSerial 3
  bNumConfigurations 1
  Configuration Descriptor:
    bLength 9
    bDescriptorType 2
    wTotalLength 32
    bNumInterfaces 1
    bConfigurationValue 1
    iConfiguration 0
    bmAttributes 0x80
      (Bus Powered)
    MaxPower 200mA
    Interface Descriptor:
      bLength 9
      bDescriptorType 4
      bInterfaceNumber 0
      bAlternateSetting 0
      bNumEndpoints 2
      bInterfaceClass 8 Mass Storage
      bInterfaceSubClass 6 SCSI
      bInterfaceProtocol 80 Bulk (Zip)
      iInterface 0
      Endpoint Descriptor:
        bLength 7
        bDescriptorType 5
        bEndpointAddress 0x81 EP 1 IN
        bmAttributes 2
          Transfer Type Bulk
          Synch Type None
          Usage Type Data
        wMaxPacketSize 0x0200 1x 512 bytes
        bInterval 0
      Endpoint Descriptor:
        bLength 7
        bDescriptorType 5
        bEndpointAddress 0x02 EP 2 OUT
        bmAttributes 2
          Transfer Type Bulk
          Synch Type None
          Usage Type Data
        wMaxPacketSize 0x0200 1x 512 bytes
        bInterval 0
can't get de...

[mlx (myxal-mxl)]

#267: That's a vmWare problem. See here> http://communities.vmware.com/thread/118175

@Canonical: Is it a good idea to have people with completely different issues dogpile on this evergreen bug? Marking any new bug as a duplicate of this one instead of telling them to get more info, referring them to https://help.ubuntu.com/community/DiskPerformance probably isn't helping, either :-\

[Theodore Ts'o (tytso)]

Comment which I added to bug #541937:

No, no, no!!!!!

Please, can we not tell people to dogpile onto bug #197762?

That bug is so dogpiled as to be useless.

We need to have separate bugs for separate performance problems, instead of assuming that all reported performance problems with USB drives should be directed to a bug that multiple people have been trying to shut down as "there's no intelligent life here, Scotty", which was opened over two years ago, and for which there are at least three separate and distinct problems, and probably more, all horribly mixed together.

Can someone official in Ubuntu step up and say whether or not this horrible practice should be continued?

I'm seriously thinking about writing a blog entry, "Why Launchpad is terminally broken, and why all upstream devs should ignore it as a waste of their time", and pointing to #197762, and misguided attempts to get people to dogpile onto this misbegotten bug, as a exhibit number #1 as to why kernel developers are ignoring Launchpad and why it is more of a menance than a help to upstream developers. If canonical isn't providing upstream development assistance, and they're not providing enough talented people to do bug triage, and they are claiming that the value they are bringing to upstream projects is lots of users and lots of bug reports, this is a prime example of why This Isn't Working For Us.

[Jeremy Foshee (jeremyfoshee)]

Ted,
     I couldn't agree more. I'm not sure I fall into what you would call "someone official in Ubuntu", but I am the Kernel Bug Triager. My stance on these is to have reporters, with any deviation from the reported bug they think is affecting them, file a new bug while subscribing to the bug they find similar. In this fashion, they are able to test fixes against their reported issue for the other reported bugs. The difficulty is in the push to consolidate that seems to be afflicting some bug tragers. This is difficult to address, as I am sure you can imagine.

For what it is worth, I appreciate that you take the time to look at these issues and comment on them. Given that the nature of this bug ticket has changed so much since it was reported, and given the fact that no one here seems to be willing to continue work on their respective bugs. I am marking this particular bug as Invalid. Those of you affected by this whose bugs are marked as duplicates will need to remove the duplicate designation. To the reporter, Amar Boudjerida, please open a new bug for your issue and we will begin again.

To those of you who did not file a bug in favor of marking this bug 'me too', please file a bug with your collected data and a clear description of the issue in both the bug title and the description fields so that we may work them as appropriate.

Ted, I'd like for you and I to chat a bit about the best way to go about the ubuntu kernel bug filing process if you are open to it. I think between us we can get this to a better place.

Thanks!

~JFo

[Shawn Reist (icabaud)]

I have been following this post for some time.

I have installed Debian Lenny and started using that as a primary system
and the Ubuntu system I have I tinker with.

Call me a rebel or such.. but I have noticed differences on how the
systems' (Ubuntu and Debian) handle file transfers.

I am not a super geek and can't devote a lot of time to tracking down
the differences.

One thing I have noticed is that Ubuntu does not write some of the
configuration files for USB devices. . .or that some of the utilities
required to read the USB information can't retrieve anything because the
required files do not exist. there are increasing wait periods or data
written/read from a USB device, something in the wait states. . .
someone with more technical familiarity needs to look at these.

As far as I can see there are several issues, but without some help or
guidance I can't give more information. Most people from what I see are
quoting the same fixes over and over; it is or has become a circular
enigma. Specific data has to be collected from a range of systems in a
specific order and collated/sorted through.

Shawn.

[Theodore Ts'o (tytso)]

On Tue, Apr 20, 2010 at 09:06:13PM -0000, Jeremy Foshee wrote:
> Ted,
> I couldn't agree more. I'm not sure I fall into what you would
> call "someone official in Ubuntu", but I am the Kernel Bug
> Triager. My stance on these is to have reporters, with any deviation
> from the reported bug they think is affecting them, file a new bug
> while subscribing to the bug they find similar. In this fashion, they
> are able to test fixes against their reported issue for the other
> reported bugs. The difficulty is in the push to consolidate that
> seems to be afflicting some bug tragers. This is difficult to
> address, as I am sure you can imagine.

So let me suggest some technological fixes first --- not that they
will completely solve the problem, but Launchpad desperately needs them.

The first is I think we need to have a way for someone to put a
statement at the very top of the Launchpad bug. Either right before
or right after the bug description. This would be a place for someone
to put a message such as "for people who think that they have this
problem, please see this wiki page first, which will describe how to
do more bug-specific diagnosis --- i.e., how to collect various bits
of diagnostic data and then to suggest which of several subsidary bugs
that they can check the "this bug affects me too" box.

One of the major problem is that Launchpad is !@#@! slow once it has
more than several hundred comments, and if people try to put helpful
information in the two hundred and seventieth comment, most users
never bother to read that far. So you really need a way to put
information to users who are first getting redirected to the bug that
they will actually *see*.

The other thing that you really need is a way for a bug to be closed.
A bug which is closed is one which can't be used as a duplicate, and
for which no one but an admin is allowed to post further comments, or
change the state of the bug, or add some other package as being
affected by a bug. It's basically a way of saying, "This bug is a EPA
Superfund toxic waste site, and please open a new bug if you think you
have a similar problem". Combined with the first feature, where
someone can place a message explaining why a bug is closed, and what
people should do, would be a big, big help.

Another thing that might be useful (and which would also require some
number of triagers to have privilege bits; you can't give it out to
everyone), is some way to mark certain comments as being irrelevant.
If the bug is really about USB1 vs. USB2 detection, the last thing you
need is for someone to say, "me too! I have a HDD performance problem
when using XFS". When we have a vast number of non-technical users,
we ****desperately**** need a way to moderate out the crap comments.
Maybe for political reasons they are displayed by default, but if an
upstream developer is going to viewing the bug, they really will want
a way to hit a button and get rid of the irrelevant comments. (Maybe
there would be a moderation reason where the triager can say,
"unrelated problem; please open a new bug report with full details of
your hardware and software configuration" before marking a comment
with the crap...

[David Tombs (dgtombs)]

OK Ted, I have taken your advice and changed the title and description. Hope this will solve some problems.

[Martin Pool (mbp)]

Hi Ted,

> Launchpad is slow

Sadly often true. I'm told this may be the major focus for later in 2010.

> Put a message at the top

The usual thing is to change the description. There is a different bug asking for ways for packagers to display more messages later. Unfortunately some users won't read them before talking.

> Close/lock bugs

https://bugs.edge.launchpad.net/malone/+bug/73122

> Filtered view

Doing this as an opt-in mode for upstreams would be interesting. There are some other things like code reviews where we do at least highlight comments by well-known users.

[Pavol Klačanský (pavolzetor-deactivatedaccount)]

same problem, with external USB hard drive, speed falls from 20 MB/s to 1 MB/s, I need back up 100GB, its ridiculous

[Charlton (charlton-grant)]

Linux home 2.6.32-25-generic #44-Ubuntu SMP Fri Sep 17 20:05:27 UTC 2010 x86_64 GNU/Linux

AMD 64 quad processor same problem
USB thumb drives formatted to fat, vfat, ntfs, ext3.
Various sizes 1gb - 16gb
various manufacturers

initial fast copy +320mb (or so it says) then slows to a crawl if it finishes at all. via command line and nautilus.
When I do boot into windows on the same machine no problems whatsoever other than the fact I am in windows. :(
I have had this problem since 9.04 maybe before but I am using ubuntu for everything now.

[raimondohc (raimondohc)]

Same problem here: transfer starts well (about 15-20 Mb) but immediately (in less than 20 seconds falls to 7-5 Mb) slows down upt to 1,5 Mb or less. That problem doesn't appear in Windows XP SP3.
Here's my specs:
Ubuntu 10.04.1 LTS 64 bits
Kernel version: Linux home 2.6.32-25-generic
System up-to-date
CPU: Intel Pentium E2160
MB: Nvidia MCP73
GPU: GeForce 7300 LE
HDD: Samsung SATA 500 Gb

[Nathanel Titane (nathanel.titane)]

running up-to-date ubuntu natty amd64 and reporting (AGAIN!) to state that it took over 45 minutes to get 3.2 Gb in my usb flash drive.

The mainline kernel "fix" does not seem to affect my system nor does it even show any sort of improvement eversince i've been on ubuntu (hardy heron amd64)

The transfer rate was at about 1.1 Mb/sec throughout the whole thing and I'm using a main port (USB 2.0) to transfer my stuff in...

SYSTEM REPORTNOT ATTACHED.. OOPS ERROR

UNAME -A--------------------------------------------------------------------------------------------------------------------------------------------------

Linux Mercury 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

HWINFO-----------------------------------------------------------------------------------------------------------------------------------------------------

============ start debug info ============
libhd version 16.0u (x86-64)
using /var/lib/hardware
kernel version is 2.6
----- /proc/cmdline -----
  BOOT_IMAGE=/boot/vmlinuz-2.6.38-8-generic root=UUID=7ec66c7c-c592-4c97-a211-1993619209b0 ro quiet splash vga=795 vt.handoff=7
----- /proc/cmdline end -----
debug = 0xff7ffff7
probe = 0x00138fc4aa17fcf9fffe (+memory +pci +isapnp +net +floppy +misc +misc.serial +misc.par +misc.floppy +serial +cpu +bios +monitor +mouse +scsi +usb -usb.mods +modem +modem.usb +parallel +parallel.lp +parallel.zip -isa -isa.isdn +isdn +kbd +prom +sbus +int +braille +braille.alva +braille.fhp +braille.ht -ignx11 +sys -bios.vbe -isapnp.old -isapnp.new -isapnp.mod +braille.baum -manual +fb +pppoe -scan +pcmcia -fork -parallel.imm +s390 -cpuemu -sysfs -s390disks +udev +block +block.cdrom +block.part +edd +edd.mod -bios.ddc -bios.fb -bios.mode +input +block.mods +bios.vesa -cpuemu.debug -scsi.noserial +wlan -bios.crc -hal -bios.vram -bios.acpi -bios.ddc.ports -modules.pata -net.eeprom -x86emu -max -lxrc)
shm: attached segment 10059815 at 0x7f396443d000
>> hal.1: read hal data
  hal: connected to: :1.38
  hal: empty device list
>> floppy.1: get nvram
>> floppy.2: klog info
>> bios.1: cmdline
>> bios.1.1: apm
>> bios.2: ram
  bios: 0 disks
>> bios.2: rom
>> bios.3: smp
----- BIOS data 0x00400 - 0x004ff -----
  400 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  420 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  430 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  440 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  450 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  460 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  470 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  480 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  490 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  4a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  4b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "................"
  4c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 "...............