Comment 4 for bug 448710

doesn't setting /apps/rhythmbox/plugins/status-icon/status-icon-mode to 2 do what you want?