Comment 84 for bug 190754

As crazy as it may seem, reducing the buffer may help. Also I noticed reduced CPU usage when I manually set the sample rate to my soundcard's native rate, 48000Hz.

Try changing your /etc/pulse/daemon.conf to reflect the following changes:

; default-sample-format = s16le
default-sample-rate = 48000
; default-sample-channels = 2

default-fragments = 8
default-fragment-size-msec = 10

This has eliminated stuttering for me with both ICH4 and ICH5 (AC'97) cards, but it may vary for other soundcards. Perhaps some AC'97 based cards have a small buffer, hence if the fragment number or fragment size is too high, it cannot fit in the buffer...