Do you have vlc-plugin-pulse installed and selected "default" or "pulseaudio" as output plugin? I can't reproduce your problems with pulseaudio.
Do you have vlc-plugin-pulse installed and selected "default" or "pulseaudio" as output plugin? I can't reproduce your problems with pulseaudio.