I get this message too. Also not using pulseaudio before I upgraded, but karmic installed it for me on upgrade.
My sound buttons are now broken after I uninstalled pulse audio.
Was on an irc channel earlier talking with a bunch of people about what to do who had the same problem.
I had to restart ubuntu after I uninstalled pulseaudio, as the uninstall did not stop pulse audio.
A bunch of applications started using up all my cpu once pulse audio was stopped, and I had to restart my laptop. I guess the pulseaudio client library is buggy with regard to when pulse audio stops.
I had to uninstall pulse audio, since it used up a constant 3% cpu even when there was no sound coming out... making my battery flat, and keeping my fans running reducing the lifetime of my laptop.