Comment 27 for bug 576352

>Are you using the nVidia driver instead, or are you using the "somehow leftover"
>nouveau module resulting from after xserver-xorg-video-nouveau removal?

Indeed, that's my understanding (I played so much with those that I might have missed something but...):
-Nvidia drivers are not installed (have been removed)
-"somehow leftover" nouveau module is what I use.

>Are you successfully running stable after the removal of that package under 10.04
>or newer or is this still after your downgrade?

I am running 10.04.
At some point, I installed a PPA to get more recent xorg:
http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu

After installing the (automatically) proposed xorg related package, freeze/hangs disappeared.
But one day a new upgrade was proposed, containing that "xserver-xorg-video-nouveau" package...and hangs came back. So I only uninstalled that package and it is running fine again.