I think it is the same problem as https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/380360.
I noticed another workaround though:
ssh-agent is already initialised in /etc/gdm/Xsession. Empirically, t seems /etc/gdm/Xsession is initialised before .profile whereas /etc/X11/Xsession.options is initialised after .profile.
As ssh-agent needs to be initialised only once, simply disabling ssh-agent in /etc/X11/Xsession.options stops LD_LIBRARY_PATH from being set to null after .profile has been sourced. This can be done by changing use-ssh-agent to no-use-ssh-agent in /etc/X11/Xsession.options. This still requires root but is simpler...