Comment 6 for bug 1228570

Ever since a recent update I get this problem after using an X client remotely over ssh. It seems they may be (incorrectly) overwriting the ~/.Xauthority file, preventing the local X server from starting on the next login.

The X server failing to start because a remote application has been run is not a Unity problem. Perhaps an xlib problem?