I've learned this is not a bug Idillon. It's by design because some people like to listen to music without having the window in the way. They use the notification area icon to control it once the window is closed. If you want the X to close the application and stop playback, here's what you need to do:
1. open Rhythmbox
2. Edit > Plugins
3. scroll to the end of the list and find the entry marked "Status Icon"
4. deselect "Status Icon"
5. close the window
clicking the X on the Titlebar will now close Rhythmbox and stop its playback.