[regression]Slow disk transfer rate on Hardy

[f3a97]

Hi,
     upgrading from Gutsy to Hardy I've seen a considerable HD transfer rate slowdown.

I have a machine with two IDE disks, both are udma5 capable. In Gutsy, they were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:

/dev/hdb:
 Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec

which is quite correct for that kind of drive.

In Hardy, they are recognized as 'sd[ab]', and the performance regarding transfer rate is LESS THAN HALF:

/dev/sdb:
 Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec

That slowdown is visible also during file transfer operations, obviously. One thing I've noticed is that in the dmesg, my drive is set at UDMA/33, and also hdparm reports that udma2 is currently configured:

sudo hdparm -i /dev/sdb

/dev/sdb:

 Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08, SerialNo= WD-WCAL78023606
 Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
 RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=74
 BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=?16?
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=488397168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes: pio0 pio1 pio2 pio3 pio4
 DMA modes: mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 *udma2 udma3 udma4 udma5
 AdvancedPM=no WriteCache=enabled
 Drive conforms to: Unspecified: ATA/ATAPI-1,2,3,4,5,6

 * signifies the current active mode

I've seen a number of users complaining about that, a number of bugs with status Incomplete or Release Fixed. I can say that with the latest Hardy patches (as of now), the problem is still there. Will that be fixed before the final release? If not, that can be a huge performance penalty!

Here are some infos. I'm willing to help to solve the problem, just ask if you need any info.
ste@ste-ubuntu:~$ lsb_release -rd
Description: Ubuntu hardy (development branch)
Release: 8.04

Pertinent dmesg output:

[ 35.085132] libata version 3.00 loaded.
[ 35.460830] pata_via 0000:00:11.1: version 0.3.3
[ 35.463781] scsi0 : pata_via
[ 35.465345] scsi1 : pata_via
[ 35.465441] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xcc00 irq 14
[ 35.465446] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xcc08 irq 15
[ 35.637026] ata1.00: ATA-6: Maxtor 5T030H3, TAH71DP0, max UDMA/100
[ 35.637034] ata1.00: 60030432 sectors, multi 16: LBA
[ 35.638417] ata1.01: ATA-6: WDC WD2500JB-00GVC0, 08.02D08, max UDMA/100
[ 35.638424] ata1.01: 488397168 sectors, multi 16: LBA48
[ 35.638446] ata1.00: limited to UDMA/33 due to 40-wire cable
[ 35.638450] ata1.01: limited to UDMA/33 due to 40-wire cable
[ 35.652916] ata1.00: configured for UDMA/33
[ 35.669962] ata1.01: configured for UDMA/33
[ 36.136471] ata2.00: ATAPI: LG DVD-ROM DRD-8160B, 1.01, max UDMA/33
[ 36.136502] ata2.01: ATAPI: HL-DT-ST GCE-8160B, 1.02, max MWDMA2
[ 36.300210] ata2.00: configured for UDMA/33
[ 36.464034] ata2.01: configured for MWDMA2
[ 36.464239] scsi 0:0:0:0: Direct-Access ATA Maxtor 5T030H3 TAH7 PQ: 0 ANSI: 5
[ 36.464959] scsi 0:0:1:0: Direct-Access ATA WDC WD2500JB-00G 08.0 PQ: 0 ANSI: 5

Kernel:

Linux ste-ubuntu 2.6.24-16-generic #1 SMP Thu Apr 10 13:23:42 UTC 2008 i686 GNU/Linux

For other info, just let me know! BTW, congrats for the GREAT WORK!!!

[f3a97 (f3a97)]

Hi,
   I forgot to add an important note: I can confirm that my cables are 80 wire (just disassebled my PC to be 100% sure :)

[TenLeftFingers (tenleftfingers)]

I'm getting only 2 to 5 MB/s transferring files from one SATA II drive to another. Seems very slow. The files are all in the 700mb size and I'm using nautilus with all hardy updates on a fresh install.

Up to date as of July 20th 2008.

[zcat (zcat)]

same problem here; tried two different cables, all definitely 80-wire. Also tried
ide0=ata66 ide1=ata66 idebus=66 as kernel parameters but no effect.

root@ubuntu:~# uname -a
Linux ubuntu 2.6.24-16-server #1 SMP Thu Apr 10 13:58:00 UTC 2008 i686 GNU/Linux
root@ubuntu:~# dmesg | grep ata
[ 0.000000] BIOS-e820: 000000000f7f3000 - 000000000f800000 (ACPI data)
[ 0.000000] Kernel command line: root=UUID=bb4bf196-20fd-40c5-bc70-7626b06e3f86 ide0=ata66 ide1=ata66 idebus=66 ro quiet splash
[ 21.503717] Memory: 239472k/253888k available (2255k kernel code, 13812k reserved, 1032k data, 384k init, 0k highmem)
[ 21.503752] .data : 0xc0333e95 - 0xc0435fe4 (1032 kB)
[ 27.498517] libata version 3.00 loaded.
[ 27.594861] pata_via 0000:00:07.1: version 0.3.3
[ 27.618377] scsi0 : pata_via
[ 27.650709] scsi1 : pata_via
[ 27.650857] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 irq 14
[ 27.650864] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xc008 irq 15
[ 27.828715] ata1.00: ATA-6: ST340810A, 3.39, max UDMA/100
[ 27.828728] ata1.00: 78165360 sectors, multi 16: LBA
[ 27.828752] ata1.00: limited to UDMA/33 due to 40-wire cable
[ 27.868899] ata1.00: configured for UDMA/33
[ 28.208652] ata2.00: ATAPI: GCR-8481B, 1.06, max UDMA/33
[ 28.408728] ata2.00: configured for UDMA/33
[ 29.901673] EXT3-fs: mounted filesystem with ordered data mode.

[SabreWolfy (sabrewolfy)]

Same as bug #155511?

[MountainX (dave-mountain)]

I'm seeing slow disk transfers with Hardy (8.4.1) on a server. Windows 2003 server on older/slower hardware is performing better. I need to improve the Ubuntu server performance! :)

Here are some examples of hdparm results on my Ubuntu Hardy server. These are from a RAID-5 array on an LSI MegaRAID 320-2E SCSI controller with 4x Seagate 15k rpm drives.

/dev/mapper/sdX_crypt:
 Timing cached reads: 1378 MB in 2.00 seconds = 688.89 MB/sec
 Timing buffered disk reads: 166 MB in 3.01 seconds = 55.12 MB/sec

/dev/mapper/sdX_crypt:
 Timing cached reads: 1368 MB in 2.00 seconds = 683.69 MB/sec
 Timing buffered disk reads: 170 MB in 3.02 seconds = 56.37 MB/sec

I have all the relevant info (dmesg, lspci, lsmod, hdparm, bonnie++ output, etc.) I will post it here if requested. Thanks.

[Hal (halbower)]

The problem with 8.04 Hardy Heron's use of libata and improperly setting DMA
characteristics of systems is inhibiting effective use of solid-state media
in systems. On a number of systems I have confirmed that this does not
exist on Feisty (7.04) and Dapper (6.06) systems where full-speed is
attained on Flash-Memory-based systems. Here are some sample excerpts
from dmesg on different machines (All CF-IDE adapters verified as being DMA
capable, 4 different types):

1. Koolu (aka FIC A603). AMD Geode at 500 MHz, CF with 2 inches of cable
   between CF-44-pin adapter.
 [ 24.433986] SCSI subsystem initialized
 [ 24.491024] libata version 3.00 loaded.
 [ 24.502688] scsi0 : pata_cs5536
 [ 24.503175] scsi1 : pata_cs5536
 [ 24.504264] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xff00 \
        irq 14
 [ 24.504632] ata2: DUMMY
 [ 24.668330] ata1.00: ATA-4: ELITE PRO CF CARD 4GB, 20071207, max UDMA/66
 [ 24.668687] ata1.00: 7831152 sectors, multi 0: LBA
 [ 24.668989] ata1.00: limited to UDMA/33 due to 40-wire cable
 [ 24.684233] ata1.00: configured for UDMA/33
 [ 24.684815] scsi 0:0:0:0: Direct-Access ATA ELITE PRO CF \
        CAR 2007 PQ: 0 ANSI: 5

2. Biostar Turiun64 3200 MicroATX Mobo, sda is IDE-CF Adapter with 80-wire
   cable, sdb is UDMA/133 Hard Drive.
 [ 24.398426] SCSI subsystem initialized
 [ 24.440126] libata version 3.00 loaded.
   ...
 [ 24.500091] sata_via 0000:00:0f.0: version 2.3
 [ 24.500109] ACPI: PCI Interrupt 0000:00:0f.0[B] -> Link [ALKA] -> GSI 20
        (level, low) -> IRQ 1
 [ 24.500860] sata_via 0000:00:0f.0: routed to hard irq line 11
 [ 24.508043] scsi0 : sata_via
 [ 24.520038] USB Universal Host Controller Interface driver v3.0
 [ 24.528029] scsi1 : sata_via
 [ 24.529176] ata1: SATA max UDMA/133 cmd 0xde00 ctl 0xe400 bmdma 0xdd00
        irq 16
 [ 24.529734] ata2: SATA max UDMA/133 cmd 0xe500 ctl 0xdc00 bmdma 0xdd08
        irq 16
   ...
 [ 24.731771] ata1: SATA link down 1.5 Gbps (SStatus 0 SControl 300)
 [ 24.943531] ata2: SATA link down 1.5 Gbps (SStatus 0 SControl 300)
   ...
 [ 25.894752] pata_via 0000:00:0f.1: version 0.3.3
 [ 25.894773] ACPI: PCI Interrupt 0000:00:0f.1[A] -> Link [ALKA] -> GSI 20
        (level, low) -> IRQ 1
 [ 25.918239] scsi2 : pata_via
 [ 25.940463] scsi3 : pata_via
 [ 25.962812] ata3: PATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0xdf00
        irq 14
 [ 25.984729] ata4: PATA max UDMA/133 cmd 0x170 ctl 0x376 bmdma 0xdf08
        irq 15
 [ 26.118214] usb 1-2: new full speed USB device using uhci_hcd and address 3
 [ 26.170481] ata3.00: ATA-4: TRANSCEND, 20080128, max UDMA/66
 [ 26.192508] ata3.00: 15662304 sectors, multi 0: LBA
 [ 26.214508] ata3.00: limited to UDMA/33 due to 40-wire cable
 [ 26.250259] ata3.00: configured for UDMA/33
 [ 26.341366] usb 1-2: configuration #1 chosen from 1 choice
 [ 26.434370] ata4.00: ATA-7: Maxtor 6Y120P0, YAR41BW0, max UDMA/133
 [ 26.456746] ata4.00: 240121728 sectors, multi 16: LBA
 [ 26.494165] ata4.00: configured for UDMA/133
 [ 26.516437] scsi 2:0:0:0: Direct-Access ATA TRANSCEND \
        2008 PQ: 0 ANSI: 5
 [ 26.539610] scsi 3:0:0:0: Direct-Access...

[xteejx (xteejx-deactivatedaccount)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. You reported this bug a while ago and there hasn't been any activity in it recently. We were wondering is this still an issue for you? Can you try with latest Ubuntu release? Thanks in advance.

Note: Why was this never set to confirmed by anyone if so many users are experiencing this same problem?

[f3a97 (f3a97)]

Hi Teej,
    if you take a look at the bug report, you'll discover that this
but is STILL present in Intrepid.

There hasn't been any activity since nobody from the bug squad ever
replied to this bug, but I can confirm definitely that this is there.

Let me know what kind of info do you need to spot the problem.

2008/11/6 Teej <email address hidden>:
> Thank you for taking the time to report this bug and helping to make
> Ubuntu better. You reported this bug a while ago and there hasn't been
> any activity in it recently. We were wondering is this still an issue
> for you? Can you try with latest Ubuntu release? Thanks in advance.
>
> Note: Why was this never set to confirmed by anyone if so many users are
> experiencing this same problem?
>
> ** Changed in: ubuntu
> Status: New => Confirmed
>
> --
> Slow disk transfer rate on Hardy
> https://bugs.launchpad.net/bugs/216878
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Ubuntu: Confirmed
>
> Bug description:
> Hi,
> upgrading from Gutsy to Hardy I've seen a considerable HD transfer rate slowdown.
>
> I have a machine with two IDE disks, both are udma5 capable. In Gutsy, they were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:
>
> /dev/hdb:
> Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec
>
> which is quite correct for that kind of drive.
>
> In Hardy, they are recognized as 'sd[ab]', and the performance regarding transfer rate is LESS THAN HALF:
>
> /dev/sdb:
> Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec
>
> That slowdown is visible also during file transfer operations, obviously. One thing I've noticed is that in the dmesg, my drive is set at UDMA/33, and also hdparm reports that udma2 is currently configured:
>
> sudo hdparm -i /dev/sdb
>
> /dev/sdb:
>
> Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08, SerialNo= WD-WCAL78023606
> Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
> RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=74
> BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=?16?
> CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=488397168
> IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
> PIO modes: pio0 pio1 pio2 pio3 pio4
> DMA modes: mdma0 mdma1 mdma2
> UDMA modes: udma0 udma1 *udma2 udma3 udma4 udma5
> AdvancedPM=no WriteCache=enabled
> Drive conforms to: Unspecified: ATA/ATAPI-1,2,3,4,5,6
>
> * signifies the current active mode
>
> I've seen a number of users complaining about that, a number of bugs with status Incomplete or Release Fixed. I can say that with the latest Hardy patches (as of now), the problem is still there. Will that be fixed before the final release? If not, that can be a huge performance penalty!
>
> Here are some infos. I'm willing to help to solve the problem, just ask if you need any info.
> ste@ste-ubuntu:~$ lsb_release -rd
> Description: Ubuntu hardy (development branch)
> Release: 8.04
>
> Pertinent dmesg output:
>
> [ 35.085132] libata version 3.00 loaded.
> [ 35.460830] pata_via 0000:00:11.1: versi...

[xteejx (xteejx-deactivatedaccount)]

Hi Stek79,

I didn't notice anything about it being a problem in Intrepid, this is obviously quite a long regression and I have tagged it accordingly.

Thank you

[f3a97 (f3a97)]

ste@ste-ubuntu:~$ cat /proc/version_signature
Ubuntu 2.6.27-7.16-generic

[Charlie Kravetz (cjkgeek)]

Set to triaged; importance medium since this is a regression affecting hardware

[Launchpad Janitor (janitor)]

Per a decision made by the Ubuntu Kernel Team, bugs will longer be assigned to the ubuntu-kernel-team in Launchpad as part of the bug triage process. The ubuntu-kernel-team is being unassigned from this bug report. Refer to https://wiki.ubuntu.com/KernelTeamBugPolicies for more information. Thanks.

[Gord Wait (gordwait)]

Can you explain what this means to those of us suffering from the extremely poor IDE performance on Hardy etc?

I read thru the link (KernelTeamBugPolicies) and the impression I get is that this bug has been de-prioritized, and no one is looking into why any more. I understand if there are not enough people to work on bugs, but please let us know the status of any fix for this.
If it's not going to be fixed I will install an older version of ubuntu on my server box to avoid it, as this bug is causing my server to be painfully slow.

Does anyone know how to revert the PATA drives back to the old drivers to avoid this slowdown?

[Hal (halbower)]

This bug seems to be fixed in 8.10 (Intrepid Ibex). All systems I reported above exhibit improved performance in 8.10 as well as several more with Compact Flash adapters. A Fix for this in Hardy would be good to allow continued use of an LTS system.

[f3a97 (f3a97)]

Hal,
  it is just the opposite.

This bug is not present in Hardy, but it is there in Intrepid.

The fact that you notice an improvement, IMHO means that you are not hitting
this bug actually.

2009/1/7 Hal <email address hidden>

> This bug seems to be fixed in 8.10 (Intrepid Ibex). All systems I
> reported above exhibit improved performance in 8.10 as well as several
> more with Compact Flash adapters. A Fix for this in Hardy would be good
> to allow continued use of an LTS system.
>
> --
> [regression]Slow disk transfer rate on Hardy
> https://bugs.launchpad.net/bugs/216878
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in "linux" source package in Ubuntu: Triaged
>
> Bug description:
> Hi,
> upgrading from Gutsy to Hardy I've seen a considerable HD transfer rate
> slowdown.
>
> I have a machine with two IDE disks, both are udma5 capable. In Gutsy, they
> were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:
>
> /dev/hdb:
> Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec
>
> which is quite correct for that kind of drive.
>
> In Hardy, they are recognized as 'sd[ab]', and the performance regarding
> transfer rate is LESS THAN HALF:
>
> /dev/sdb:
> Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec
>
> That slowdown is visible also during file transfer operations, obviously.
> One thing I've noticed is that in the dmesg, my drive is set at UDMA/33, and
> also hdparm reports that udma2 is currently configured:
>
> sudo hdparm -i /dev/sdb
>
> /dev/sdb:
>
> Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08, SerialNo=
> WD-WCAL78023606
> Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
> RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=74
> BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=?16?
> CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=488397168
> IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
> PIO modes: pio0 pio1 pio2 pio3 pio4
> DMA modes: mdma0 mdma1 mdma2
> UDMA modes: udma0 udma1 *udma2 udma3 udma4 udma5
> AdvancedPM=no WriteCache=enabled
> Drive conforms to: Unspecified: ATA/ATAPI-1,2,3,4,5,6
>
> * signifies the current active mode
>
> I've seen a number of users complaining about that, a number of bugs with
> status Incomplete or Release Fixed. I can say that with the latest Hardy
> patches (as of now), the problem is still there. Will that be fixed before
> the final release? If not, that can be a huge performance penalty!
>
> Here are some infos. I'm willing to help to solve the problem, just ask if
> you need any info.
> ste@ste-ubuntu:~$ lsb_release -rd
> Description: Ubuntu hardy (development branch)
> Release: 8.04
>
> Pertinent dmesg output:
>
> [ 35.085132] libata version 3.00 loaded.
> [ 35.460830] pata_via 0000:00:11.1: version 0.3.3
> [ 35.463781] scsi0 : pata_via
> [ 35.465345] scsi1 : pata_via
> [ 35.465441] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xcc00 irq
> 14
> [ 35.465446] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xcc08 irq
> 15
> [ 35.637026] ata...

[stat (maibpenrai)]

Is there any update on this? I have looked for weeks now for some fix or update. The transfer speeds are horribly slow and wonder how this can not be a priority fix. It renders Ibex almost unusable for normal operations. Is this being tracked somewhere else or am I missing something?

[ZhangInSeattle (yzhang-ca)]

My situation might be related to this bug:

I bought a Thinkpad T61 in last October. I erased Vista and installed Ubuntu 8.7 and later upgraded to 8.10. The disk speed seemed slow. I tried the following command to test the real read speed:

$ date; cp a_100_MB_file /dev/null; date;

The copying takes about 20 seconds. So the speed is about 5MB/sec.

But when I try to use hdparm to test the read speed, the result is like this:

$ sudo hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 130 MB in 3.03 seconds = 42.84 MB/sec

Can anybody tell me why the real read speed is far from the 42.84MB/sec?

[stat (maibpenrai)]

I can't for one tell you as think that is the key to the issue, why? For
example here are my hdparm rates:

/dev/sda:
 Timing buffered disk reads: 54 MB in 3.14 seconds = 17.19 MB/sec
stat@stat:~$ sudo hdparm -t /dev/sdb

/dev/sdb:
 Timing buffered disk reads: 218 MB in 3.02 seconds = 72.18 MB/sec
stat@stat:~$ sudo hdparm -t /dev/sdc

/dev/sdc:
 Timing buffered disk reads: 90 MB in 3.03 seconds = 29.71 MB/sec

As you can see they are dramatically different and the actual transfer rates
are approximately:

6.7 MB/sec
10.9 MB/sec
3.4 MB/sec

This is really pretty pathetic both the rates themselves and the variance.

I am intrigued though by something that I have been considering and wonder
if you might be able to confirm it. How did you upgrade from Hardy to
Ibex? I did not do a clean install but used the upgrade tool and wonder if
you did the same. If so it might be related to that and the fix could be as
simple (well not really simple but direct anyway) as doing a clean install
and starting over.

On Mon, Feb 16, 2009 at 2:07 AM, ZhangInSeattle <email address hidden> wrote:

> My situation might be related to this bug:
>
> I bought a Thinkpad T61 in last October. I erased Vista and installed
> Ubuntu 8.7 and later upgraded to 8.10. The disk speed seemed slow. I
> tried the following command to test the real read speed:
>
> $ date; cp a_100_MB_file /dev/null; date;
>
> The copying takes about 20 seconds. So the speed is about 5MB/sec.
>
> But when I try to use hdparm to test the read speed, the result is like
> this:
>
> $ sudo hdparm -t /dev/sda
>
> /dev/sda:
> Timing buffered disk reads: 130 MB in 3.03 seconds = 42.84 MB/sec
>
> Can anybody tell me why the real read speed is far from the 42.84MB/sec?
>
> --
> [regression]Slow disk transfer rate on Hardy
> https://bugs.launchpad.net/bugs/216878
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in "linux" source package in Ubuntu: Triaged
>
> Bug description:
> Hi,
> upgrading from Gutsy to Hardy I've seen a considerable HD transfer rate
> slowdown.
>
> I have a machine with two IDE disks, both are udma5 capable. In Gutsy, they
> were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:
>
> /dev/hdb:
> Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec
>
> which is quite correct for that kind of drive.
>
> In Hardy, they are recognized as 'sd[ab]', and the performance regarding
> transfer rate is LESS THAN HALF:
>
> /dev/sdb:
> Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec
>
> That slowdown is visible also during file transfer operations, obviously.
> One thing I've noticed is that in the dmesg, my drive is set at UDMA/33, and
> also hdparm reports that udma2 is currently configured:
>
> sudo hdparm -i /dev/sdb
>
> /dev/sdb:
>
> Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08, SerialNo=
> WD-WCAL78023606
> Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
> RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=74
> BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=?16?
> CurCHS=16383/16/63, CurSects=16514...

[ZhangInSeattle (yzhang-ca)]

Thanks stat!

Yes, I used the upgrade tool to install Ibex..

Did you try to do a clean install and fix the issue?

(I remember that I tried to boot from the clean 8.10 CD and mount the same disk to test the speed. The speed is the same as I boot from my hard drive. So it seemed the clean install might not work for me. Anyway I'd like to hear if you succeeded.)

[f3a97 (f3a97)]

Hello Zhang,
       this is a common issue, and is not related to Linux.

It is important to understand the testing that we're doing and how a
rotational disk works.

hdparm test the _sequential_ transfer speed, which is the speed at which
data can be transferred from sequential sectors.

Real life data transfer can be much lower indeed - as you've measured -
since disks are mechanical parts, they have a moving arm inside. This
implies that if your workload is random (as opposed to sequential), you end
up with another bottleneck, which is arm movement. This slows down your
throughput to 5 MB/s.

If you generate a pure random workload for your disk, you'll see that you
can barely reach 1-2 MB/s! You can try that issuing a:

$ find / -ls > /dev/null

and watching disk stats with

# iostat -x 10

Bye!

2009/2/15 ZhangInSeattle <email address hidden>

> My situation might be related to this bug:
>
> I bought a Thinkpad T61 in last October. I erased Vista and installed
> Ubuntu 8.7 and later upgraded to 8.10. The disk speed seemed slow. I
> tried the following command to test the real read speed:
>
> $ date; cp a_100_MB_file /dev/null; date;
>
> The copying takes about 20 seconds. So the speed is about 5MB/sec.
>
> But when I try to use hdparm to test the read speed, the result is like
> this:
>
> $ sudo hdparm -t /dev/sda
>
> /dev/sda:
> Timing buffered disk reads: 130 MB in 3.03 seconds = 42.84 MB/sec
>
> Can anybody tell me why the real read speed is far from the 42.84MB/sec?
>
> --
> [regression]Slow disk transfer rate on Hardy
> https://bugs.launchpad.net/bugs/216878
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in "linux" source package in Ubuntu: Triaged
>
> Bug description:
> Hi,
> upgrading from Gutsy to Hardy I've seen a considerable HD transfer rate
> slowdown.
>
> I have a machine with two IDE disks, both are udma5 capable. In Gutsy, they
> were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:
>
> /dev/hdb:
> Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec
>
> which is quite correct for that kind of drive.
>
> In Hardy, they are recognized as 'sd[ab]', and the performance regarding
> transfer rate is LESS THAN HALF:
>
> /dev/sdb:
> Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec
>
> That slowdown is visible also during file transfer operations, obviously.
> One thing I've noticed is that in the dmesg, my drive is set at UDMA/33, and
> also hdparm reports that udma2 is currently configured:
>
> sudo hdparm -i /dev/sdb
>
> /dev/sdb:
>
> Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08, SerialNo=
> WD-WCAL78023606
> Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
> RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=74
> BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=?16?
> CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=488397168
> IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
> PIO modes: pio0 pio1 pio2 pio3 pio4
> DMA modes: mdma0 mdma1 mdma2
> UDMA modes: udma0 udma1 *udma2 udma3 udma4 udma5
> AdvancedPM=no Writ...

[ZhangInSeattle (yzhang-ca)]

Hi Stek79,

Thanks for your reply!

I understand the hdparm result is the best possible speed. But the difference between 42.84MB/sec and 5MB/sec still shocked me.
It kinda implied that the file system (I'm using ext3) needs more optimization to store big files in the way the hard disk can sequentially read (with minimum arm movement). (I don't know much about ext3, maybe it did the optimization already).

My laptop (thinkpad t61) is purchased in October 2008. It has a SATA disk. It should have a much better performance than 5MB/sec imho :-)

Thanks!
ZhangInSeattle

[f3a97 (f3a97)]

Hi Zhang,
  you're welcome.

It is actually possible to get very high performance from SATA disks,
comparable to those of SAS or SCSI, by using a technique called
short-stroking.

If you understand the mechanics of the disks, you can actually build a
system with very low latency. You can get this by placing the most accessed
files on the outer zone of your disk, thus getting the max rotational speed
and reducing head movement.

This way you can greatly reduce the arm movement and get higher throughputs!
:)

2009/2/16 ZhangInSeattle <email address hidden>

> Hi Stek79,
>
> Thanks for your reply!
>
> I understand the hdparm result is the best possible speed. But the
> difference between 42.84MB/sec and 5MB/sec still shocked me.
> It kinda implied that the file system (I'm using ext3) needs more
> optimization to store big files in the way the hard disk can sequentially
> read (with minimum arm movement). (I don't know much about ext3, maybe it
> did the optimization already).
>
> My laptop (thinkpad t61) is purchased in October 2008. It has a SATA
> disk. It should have a much better performance than 5MB/sec imho :-)
>
>
> Thanks!
> ZhangInSeattle
>
> --
> [regression]Slow disk transfer rate on Hardy
> https://bugs.launchpad.net/bugs/216878
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in "linux" source package in Ubuntu: Triaged
>
> Bug description:
> Hi,
> upgrading from Gutsy to Hardy I've seen a considerable HD transfer rate
> slowdown.
>
> I have a machine with two IDE disks, both are udma5 capable. In Gutsy, they
> were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:
>
> /dev/hdb:
> Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec
>
> which is quite correct for that kind of drive.
>
> In Hardy, they are recognized as 'sd[ab]', and the performance regarding
> transfer rate is LESS THAN HALF:
>
> /dev/sdb:
> Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec
>
> That slowdown is visible also during file transfer operations, obviously.
> One thing I've noticed is that in the dmesg, my drive is set at UDMA/33, and
> also hdparm reports that udma2 is currently configured:
>
> sudo hdparm -i /dev/sdb
>
> /dev/sdb:
>
> Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08, SerialNo=
> WD-WCAL78023606
> Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
> RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=74
> BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=?16?
> CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=488397168
> IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
> PIO modes: pio0 pio1 pio2 pio3 pio4
> DMA modes: mdma0 mdma1 mdma2
> UDMA modes: udma0 udma1 *udma2 udma3 udma4 udma5
> AdvancedPM=no WriteCache=enabled
> Drive conforms to: Unspecified: ATA/ATAPI-1,2,3,4,5,6
>
> * signifies the current active mode
>
> I've seen a number of users complaining about that, a number of bugs with
> status Incomplete or Release Fixed. I can say that with the latest Hardy
> patches (as of now), the problem is still there. Will that be fixe...

[stat (maibpenrai)]

Haven't done it yet but will this weekend. I am not a guru but find it hard
to believe that I can get greater transfer speeds between machines than I
can within the actual machine. That just makes no sense to me. I have an
XP box here and will be testing it with the same drives before I do
anything. I suspect that those speeds will be different. If they are not
then ... well I will accept that it is not a linux issue. Also I took the
drives out and moved them to an old Hardy box and the parm test resutls were
even higher (around the 70's for two of the disks and 40's for the one that
is slower). While I agree that placement on the disk is important and
recognize the affects of the mechanical side of things I again find it hard
to accept that there is a 85% drop off due to inefficiency. Also one of the
disks I use is nearly clean which begs the question. I kind of live by the
rule that if there is even one exception to the hypothesis that you need a
new hypothesis :)

I will let you know what I find out and appreciate the other comments you
passed along. BTW, not sure that if you booted from the live disk that will
count as an accurate test. I get 1-2 mb throughput going to a USB so why
should we see nearly that internal?

Also would note that I have have seen many, many threads on this and have
even seen (not sure I can place it now) the Ubuntu team admit that there is
an issue.

On Mon, Feb 16, 2009 at 12:38 PM, ZhangInSeattle <email address hidden>wrote:

> Thanks stat!
>
> Yes, I used the upgrade tool to install Ibex..
>
> Did you try to do a clean install and fix the issue?
>
> (I remember that I tried to boot from the clean 8.10 CD and mount the
> same disk to test the speed. The speed is the same as I boot from my
> hard drive. So it seemed the clean install might not work for me. Anyway
> I'd like to hear if you succeeded.)
>
> --
> [regression]Slow disk transfer rate on Hardy
> https://bugs.launchpad.net/bugs/216878
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in "linux" source package in Ubuntu: Triaged
>
> Bug description:
> Hi,
> upgrading from Gutsy to Hardy I've seen a considerable HD transfer rate
> slowdown.
>
> I have a machine with two IDE disks, both are udma5 capable. In Gutsy, they
> were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:
>
> /dev/hdb:
> Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec
>
> which is quite correct for that kind of drive.
>
> In Hardy, they are recognized as 'sd[ab]', and the performance regarding
> transfer rate is LESS THAN HALF:
>
> /dev/sdb:
> Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec
>
> That slowdown is visible also during file transfer operations, obviously.
> One thing I've noticed is that in the dmesg, my drive is set at UDMA/33, and
> also hdparm reports that udma2 is currently configured:
>
> sudo hdparm -i /dev/sdb
>
> /dev/sdb:
>
> Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08, SerialNo=
> WD-WCAL78023606
> Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
> RawCHS=16383/16/63, TrkSize=57600, SectSize=...

[f3a97 (f3a97)]

Hi,
      it all depends on the benchmark.

How do you plan to reproduce on Windows?

If you take a look at recent benchmarks, you'll see that Ubuntu performs
faster here:

http://www.tuxradar.com/content/benchmarked-ubuntu-vs-vista-vs-windows-7

Let me know which kind of test you've done and what are the results.

 I kind of live by the
> rule that if there is even one exception to the hypothesis that you need a
> new hypothesis :)

I agree! But it is imperative to perform a sound benchmarking, it is not
easy as it seems.

>
>
> I will let you know what I find out and appreciate the other comments you
> passed along. BTW, not sure that if you booted from the live disk that
> will
> count as an accurate test. I get 1-2 mb throughput going to a USB so why
> should we see nearly that internal?
>
> Also would note that I have have seen many, many threads on this and have
> even seen (not sure I can place it now) the Ubuntu team admit that there is
> an issue.
>
> On Mon, Feb 16, 2009 at 12:38 PM, ZhangInSeattle
> <email address hidden>wrote:
>
> > Thanks stat!
> >
> > Yes, I used the upgrade tool to install Ibex..
> >
> > Did you try to do a clean install and fix the issue?
> >
> > (I remember that I tried to boot from the clean 8.10 CD and mount the
> > same disk to test the speed. The speed is the same as I boot from my
> > hard drive. So it seemed the clean install might not work for me. Anyway
> > I'd like to hear if you succeeded.)
> >
> > --
> > [regression]Slow disk transfer rate on Hardy
> > https://bugs.launchpad.net/bugs/216878
> > You received this bug notification because you are a direct subscriber
> > of the bug.
> >
> > Status in "linux" source package in Ubuntu: Triaged
> >
> > Bug description:
> > Hi,
> > upgrading from Gutsy to Hardy I've seen a considerable HD transfer
> rate
> > slowdown.
> >
> > I have a machine with two IDE disks, both are udma5 capable. In Gutsy,
> they
> > were recognised as 'hd[ab]' and I with hdparm sequential thoughput was:
> >
> > /dev/hdb:
> > Timing buffered disk reads: 172 MB in 3.01 seconds = 57.19 MB/sec
> >
> > which is quite correct for that kind of drive.
> >
> > In Hardy, they are recognized as 'sd[ab]', and the performance regarding
> > transfer rate is LESS THAN HALF:
> >
> > /dev/sdb:
> > Timing buffered disk reads: 78 MB in 3.01 seconds = 25.92 MB/sec
> >
> > That slowdown is visible also during file transfer operations, obviously.
> > One thing I've noticed is that in the dmesg, my drive is set at UDMA/33,
> and
> > also hdparm reports that udma2 is currently configured:
> >
> > sudo hdparm -i /dev/sdb
> >
> > /dev/sdb:
> >
> > Model=WDC WD2500JB-00GVC0 , FwRev=08.02D08,
> SerialNo=
> > WD-WCAL78023606
> > Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq
> }
> > RawCHS=16383/16/63, TrkSize=57600, SectSize=600, ECCbytes=74
> > BuffType=DualPortCache, BuffSize=8192kB, MaxMultSect=16, MultSect=?16?
> > CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=488397168
> > IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
> > PIO modes: pio0 pio1 pio2 pio3 pio4
> > DMA modes: mdma0 mdma1 mdma2
> > UDMA...

[stat (maibpenrai)]

HI:

I agree it isn't all that easy but since I have the machines to play around
with I am going to take my main machine (the one I am experiencing the
issues with and install Win 7 and then use the Richard's python script
(cross platform) as described on the site you linked to. It is pretty funny
since I had created an outline of what I was going to do from the very site
you mentioned, what are the odds. I already have just about everything put
together. The difference for me of course will be that I will be using my
8.1 install as is and will perform those tests first (excluding the install
tests) and then redoing them using Win 7 and the same hardware. I have all
the tools I need including using a couple of Aspire ones and a DNS-323 that
I will use to measure throughput externally. I believe that by following
what they did on Tuxradar, connecting some separate machine tests etc. I
should be able to get a true test and relatively close to the true picture
of things.

Once it is done I will post somewhere (here?) my steps and what results I
got. If my plan is flawed let me know, I am open to listen and am hoping to
get this done this weekend (it's good to be a lonely geek :) Kind of
looking forward to this, despite the work it entails and the fact that I
will have to turn my xbmc box into a production machine temporarily :(

On Tue, Feb 17, 2009 at 6:41 PM, stek79 <email address hidden> wrote:

> Hi,
> it all depends on the benchmark.
>
> How do you plan to reproduce on Windows?
>
> If you take a look at recent benchmarks, you'll see that Ubuntu performs
> faster here:
>
> http://www.tuxradar.com/content/benchmarked-ubuntu-vs-vista-vs-windows-7
>
> Let me know which kind of test you've done and what are the results.
>
> I kind of live by the
> > rule that if there is even one exception to the hypothesis that you need
> a
> > new hypothesis :)
>
>
> I agree! But it is imperative to perform a sound benchmarking, it is not
> easy as it seems.
>
>
> >
> >
> > I will let you know what I find out and appreciate the other comments you
> > passed along. BTW, not sure that if you booted from the live disk that
> > will
> > count as an accurate test. I get 1-2 mb throughput going to a USB so why
> > should we see nearly that internal?
> >
> > Also would note that I have have seen many, many threads on this and have
> > even seen (not sure I can place it now) the Ubuntu team admit that there
> is
> > an issue.
> >
> > On Mon, Feb 16, 2009 at 12:38 PM, ZhangInSeattle
> > <email address hidden>wrote:
> >
> > > Thanks stat!
> > >
> > > Yes, I used the upgrade tool to install Ibex..
> > >
> > > Did you try to do a clean install and fix the issue?
> > >
> > > (I remember that I tried to boot from the clean 8.10 CD and mount the
> > > same disk to test the speed. The speed is the same as I boot from my
> > > hard drive. So it seemed the clean install might not work for me.
> Anyway
> > > I'd like to hear if you succeeded.)
> > >
> > > --
> > > [regression]Slow disk transfer rate on Hardy
> > > https://bugs.launchpad.net/bugs/216878
> > > You received this bug notification because you are a direct subscriber
> > > of the bug.
> > >
> > > Statu...

[f3a97 (f3a97)]

Hi stat,
 I'm looking forward to see the results!

Only suggestions is that perhaps this is not the right place to publish
them, maybe a blog post is more appropriate.

Have fun with the benchmarking :)

2009/2/18 stat <email address hidden>

> HI:
>
> I agree it isn't all that easy but since I have the machines to play around
> with I am going to take my main machine (the one I am experiencing the
> issues with and install Win 7 and then use the Richard's python script
> (cross platform) as described on the site you linked to. It is pretty
> funny
> since I had created an outline of what I was going to do from the very site
> you mentioned, what are the odds. I already have just about everything put
> together. The difference for me of course will be that I will be using my
> 8.1 install as is and will perform those tests first (excluding the install
> tests) and then redoing them using Win 7 and the same hardware. I have all
> the tools I need including using a couple of Aspire ones and a DNS-323 that
> I will use to measure throughput externally. I believe that by following
> what they did on Tuxradar, connecting some separate machine tests etc. I
> should be able to get a true test and relatively close to the true picture
> of things.
>
> Once it is done I will post somewhere (here?) my steps and what results I
> got. If my plan is flawed let me know, I am open to listen and am hoping
> to
> get this done this weekend (it's good to be a lonely geek :) Kind of
> looking forward to this, despite the work it entails and the fact that I
> will have to turn my xbmc box into a production machine temporarily :(
>
> On Tue, Feb 17, 2009 at 6:41 PM, stek79 <email address hidden> wrote:
>
> > Hi,
> > it all depends on the benchmark.
> >
> > How do you plan to reproduce on Windows?
> >
> > If you take a look at recent benchmarks, you'll see that Ubuntu performs
> > faster here:
> >
> > http://www.tuxradar.com/content/benchmarked-ubuntu-vs-vista-vs-windows-7
> >
> > Let me know which kind of test you've done and what are the results.
> >
> > I kind of live by the
> > > rule that if there is even one exception to the hypothesis that you
> need
> > a
> > > new hypothesis :)
> >
> >
> > I agree! But it is imperative to perform a sound benchmarking, it is not
> > easy as it seems.
> >
> >
> > >
> > >
> > > I will let you know what I find out and appreciate the other comments
> you
> > > passed along. BTW, not sure that if you booted from the live disk that
> > > will
> > > count as an accurate test. I get 1-2 mb throughput going to a USB so
> why
> > > should we see nearly that internal?
> > >
> > > Also would note that I have have seen many, many threads on this and
> have
> > > even seen (not sure I can place it now) the Ubuntu team admit that
> there
> > is
> > > an issue.
> > >
> > > On Mon, Feb 16, 2009 at 12:38 PM, ZhangInSeattle
> > > <email address hidden>wrote:
> > >
> > > > Thanks stat!
> > > >
> > > > Yes, I used the upgrade tool to install Ibex..
> > > >
> > > > Did you try to do a clean install and fix the issue?
> > > >
> > > > (I remember that I tried to boot from the clean 8.10 CD and mount the
> > > > same disk to test th...