Slow disk writes (ubuntu kernel)

[Erik de Castro Lopo]

Runing the above kernel on a Dell Latitude X1 laptop. Disk writes
are very much slower with this kernel than with linux-image-2.6.15-23-686.

I have a test program which does a large amount of reading and
writing of files. On the -23 kernel this program runs in 16-18
seconds. On the -25 kernel the same binary takes from 1 -4
minutes. The machine is not doing much else at the time.

Here's the dstat output from the -23 kernel (write throughput about a Meg a second)

----total-cpu-usage---- -disk/total -net/total- ---paging-- ---system--
usr sys idl wai hiq siq|_read write|_recv _send|__in_ _out_|_int_ _csw_
 27 16 52 5 1 0| 111k 55k| 0 0 | 0 0 |1014 125
  0 0 100 0 0 0| 0 0 |1364B 3230B| 0 0 |1021 61
 43 29 29 0 0 0| 0 580k|1306B 3452B| 0 0 |1076 307
 25 13 38 25 0 0| 0 1160k| 956B 5238B| 0 0 |1151 458
 10 30 20 30 0 10| 0 1372k|1164B 6426B| 0 0 |1143 452
 29 14 29 29 0 0| 0 1136k| 714B 4972B| 0 0 |1108 401
 22 11 33 22 11 0|4096B 1224k| 918B 5894B| 0 0 |1155 480
 22 22 33 22 0 0| 0 1356k|1377B 7408B| 0 0 |1164 490
 27 9 27 27 9 0| 0 1424k| 950B 5588B| 0 0 |1177 665
 29 14 29 29 0 0| 0 1216k|1020B 5288B| 0 0 |1155 508
 25 25 38 13 0 0| 0 1100k| 816B 5206B| 0 0 |1154 439
 22 22 33 22 0 0|4096B 1264k|1148B 5620B| 0 0 |1151 440
 36 36 18 9 0 0| 0 1056k|1610B 8432B| 0 0 |1154 452
 14 14 43 29 0 0| 0 1104k| 714B 4590B| 0 0 |1122 496
 22 22 33 22 0 0| 0 1304k|1020B 6566B| 0 0 |1155 511
 22 11 22 33 11 0| 0 1300k|1170B 3910B| 0 0 |1141 408
 25 25 25 25 0 0| 0 980k|1402B 6814B| 0 0 |1133 432
 11 33 33 22 0 0|4096B 1204k|1224B 6444B| 0 0 |1136 448
 25 13 38 25 0 0| 0 1152k|1456B 5898B| 0 0 |1160 495
 30 20 30 20 0 0| 0 1280k|1358B 8108B| 0 0 |1146 496
 43 14 43 0 0 0| 0 348k|2303B 5166B| 0 0 |1098 206

And now the -25 output (about 1/10th the disk throughput):

----total-cpu-usage---- -disk/total -net/total- ---paging-- ---system--
usr sys idl wai hiq siq|_read write|_recv _send|__in_ _out_|_int_ _csw_
  0 0 50 50 0 0| 0 136k| 166B 666B| 0 0 | 338 57
 33 33 0 33 0 0| 0 60k| 448B 634B| 0 0 | 327 42
  0 0 100 0 0 0| 0 124k| 166B 748B| 0 0 | 332 59
  0 0 50 50 0 0| 0 40k| 166B 634B| 0 0 | 363 42
  0 50 50 0 0 0| 0 140k| 246B 634B| 0 0 | 355 68
 33 0 33 33 0 0| 0 84k| 236B 956B| 0 0 | 333 46
  0 33 67 0 0 0| 0 96k| 344B 694B| 0 0 | 286 59
100 0 0 0 0 0| 0 80k| 166B 666B| 0 0 | 339 47
  0 0 50 50 0 0| 0 68k| 294B 700B| 0 0 | 336 48
  0 0 100 0 0 0| 0 132k| 230B 634B| 0 0 | 275 55
  0 0 50 50 0 0| 0 100k| 294B 634B| 0 0 | 285 59
  0 33 33 33 0 0| 0 72k| 470B 634B| 0 0 | 306 50
  0 0 0 100 0 0| 0 88k| 358B 1242B| 0 0 | 349 55
 50 0 50 0 0 0| 0 80k| 358B 650B| 0 0 | 324 47
 33 0 33 33 0 0| 0 112k| 102B 666B| 0 0 | 356 63
  0 50 50 0 0 0| 0 72k| 483B 1070B| 0 0 | 327 32
  0 0 50 50 0 0| 0 120k| 306B 650B| 0 0 | 353 62

The hdparm settings are identical under both kernels:

/dev/hda:
 multcount = 0 (off)
 IO_support = 0 (default 16-bit)
 unmaskirq = 0 (off)
 using_dma = 1 (on)
 keepsettings = 0 (off)
 readonly = 0 (off)
 readahead = 256 (on)
 geometry = 16383/255/63, sectors = 117210240, start = 0

I can't debug this any further, but happy to do testing if someone
has an idea.

Note: the original reporter indicated the bug was in package 'linux-image-2.6.15-25-686'; however, that package was not published in Baltix.

[Peter Miller (pmiller-opensource)]

I have noticed that it takes linux-image-2.6.15-26-686 about 15 seconds to mount root (most of it with no disk activity) as the second step of the boot process, where linux-image-2.6.15-23-686 took a faction of a second.

[Erik de Castro Lopo (erikd)]

Fix found.

echo 1 > /sys/module/processor/parameters/max_cstate

fixes slow disk writes.

This laptop only shows 1 CPU in /proc/cpuinfo.

[Andrew Ash (ash211)]

Glad you found the fix. If it becomes a problem again, let us know and we'll see if we can help. Thanks for your contribution to Ubuntu.

[dr4cul4 (dr4cul4)]

Trying to do:

#echo 1 > /sys/module/processor/parameters/max_cstate

result in:

bash: /sys/module/processor/parameters/max_cstate: No such file or directory

directory exist, but trying to do anything with it result in write errors.

This slow read/write problems randers my system totaly unusable (saving 192MB fie takes 1 hour !!! That's unimaginable) (Apears on 7.10/8.04/8.10, didn't have motivation to test older releases)

[Andrew Ash (ash211)]

I think you have a different bug, dr4cul4. Please open a new bug and Ubuntu members can help you try to debug it there.