After some more searching, I found out that adding
Option "AccelMethod" "uxa"
to the device-section of xorg, solves the problem for my dual display setup.
After some more searching, I found out that adding
Option "AccelMethod" "uxa"
to the device-section of xorg, solves the problem for my dual display setup.