I use a raspberry pi as a sound server with pulseaudio. Works fine but after around 3-4 minutes it crash.
I have to restart pulseaudio and then same thing happen again and again.
I tried to comment the #load-module module-suspend-on-idle but same issue persist.

Here is the trace log I have
( 294.556|   0.000) D: [alsa-sink-USB Audio] module-rtp-recv.c: Estimated target rate: 44096 Hz, using average of 44097 Hz  (α=0,800)
( 294.556|   0.000) D: [alsa-sink-USB Audio] module-rtp-recv.c: Updated sampling rate to 44100 Hz.
( 298.330|   3.773) D: [alsa-sink-USB Audio] sink.c: Found underrun 6568 bytes ago (28696 bytes ahead in playback buffer)
( 298.355|   0.024) D: [alsa-sink-USB Audio] sink.c: Found underrun 15384 bytes ago (19880 bytes ahead in playback buffer)
( 298.358|   0.003) D: [pulseaudio] module-rtp-recv.c: Checking for dead streams ...
( 298.380|   0.021) D: [alsa-sink-USB Audio] sink.c: Found underrun 24168 bytes ago (11096 bytes ahead in playback buffer)
( 298.405|   0.024) D: [alsa-sink-USB Audio] sink.c: Found underrun 32984 bytes ago (2280 bytes ahead in playback buffer)
( 318.379|  19.974) D: [pulseaudio] module-rtp-recv.c: Checking for dead streams ...
( 338.399|  20.020) D: [pulseaudio] module-rtp-recv.c: Checking for dead streams ...
( 338.399|   0.000) I: [pulseaudio] module-rtp-recv.c: Freeing session 'PulseAudio RTP Stream on boulaga'
( 338.400|   0.000) D: [pulseaudio] module-suspend-on-idle.c: Sink alsa_output.usb-Yamaha_Corporation_AG06_AG03-00-AG06AG03.analog-stereo becomes idle, timeout in 5 seconds.
( 338.400|   0.000) D: [alsa-sink-USB Audio] alsa-sink.c: Requested to rewind 35264 bytes.
( 338.400|   0.000) D: [alsa-sink-USB Audio] alsa-sink.c: Limited to 35008 bytes.
( 338.400|   0.000) D: [alsa-sink-USB Audio] alsa-sink.c: before: 4376
( 338.401|   0.000) D: [alsa-sink-USB Audio] alsa-sink.c: after: 4376
( 338.401|   0.000) D: [alsa-sink-USB Audio] alsa-sink.c: Rewound 35008 bytes.
( 338.401|   0.000) D: [alsa-sink-USB Audio] sink.c: Processing rewind...
( 338.401|   0.000) D: [alsa-sink-USB Audio] source.c: Processing rewind...
( 338.401|   0.000) D: [pulseaudio] module-suspend-on-idle.c: Sink alsa_output.usb-Yamaha_Corporation_AG06_AG03-00-AG06AG03.analog-stereo becomes idle, timeout in 5 seconds.
( 338.401|   0.000) D: [pulseaudio] core.c: Hmm, no streams around, trying to vacuum.
( 338.403|   0.002) I: [pulseaudio] sink-input.c: Freeing input 0 "RTP Stream (PulseAudio RTP Stream on boulaga)"
( 343.406|   5.002) I: [pulseaudio] module-suspend-on-idle.c: Sink alsa_output.usb-Yamaha_Corporation_AG06_AG03-00-AG06AG03.analog-stereo idle for too long, suspending ...
( 343.410|   0.003) D: [pulseaudio] sink.c: Suspend cause of sink alsa_output.usb-Yamaha_Corporation_AG06_AG03-00-AG06AG03.analog-stereo is 0x0004, suspending
( 343.412|   0.001) I: [alsa-sink-USB Audio] alsa-sink.c: Device suspended...
( 343.412|   0.000) D: [pulseaudio] core.c: Hmm, no streams around, trying to vacuum.
( 343.414|   0.001) D: [pulseaudio] module-udev-detect.c: /dev/snd/controlC1 is accessible: yes
( 343.414|   0.000) D: [pulseaudio] module-udev-detect.c: Resuming all sinks and sources of card alsa_card.usb-Yamaha_Corporation_AG06_AG03-00-AG06AG03.
( 358.419|  15.005) D: [pulseaudio] module-rtp-recv.c: Checking for dead streams ...
( 378.439|  20.020) D: [pulseaudio] module-rtp-recv.c: Checking for dead streams ...
( 398.460|  20.020) D: [pulseaudio] module-rtp-recv.c: Checking for dead streams ...