Comment 3 for bug 398619

What do you think about something like - Don't let earcandy be obtrusive when it just started. When I plug in an USB headset after it's started, fine, but for devices detected initially, just leave it be, and let the device that had audio also be the "fallback" audio device when I plug in a headset and then plug it out again. That would fix it for me, and still leave that nice feature around.
If you want, you could even add a menu item to your right-click menu that says "Switch audio device" to do it manually if the user already had a headset connected and wants to switch control to that (and they don't have pavucontrol running or so). :)