Comment 10 for bug 1768906

Difference between remove & purge:
- 'remove' uninstall the package, but does not remove the package settings, which can disturb the system and conflict with an other driver choice.
- 'purge' remove both package & its settings; that is way recommended.

About /etc/X11/xorg.conf, this is no more used since a couple releases. so your system should not finf such a path anyway.

Your real problem is not about 'xorg' as you have reported but 'gdm' and/or 'gnome-session'
******************
May 3 09:20:18 madbox /usr/lib/gdm3/gdm-x-session[2748]: (EE) NVIDIA(GPU-0): Failed to acquire modesetting permission.
May 3 09:20:18 madbox /usr/lib/gdm3/gdm-x-session[2748]: (EE) NVIDIA(0): Failing initialization of X screen 0
******************