Comment 6 for bug 1332220

For anyone finding this in the future, this is how I set up the out of spec modes and set one to come up automatically.

Create the file /etc/X11/xorg.conf.d/90-monitor.conf (you may have to create the directory)

Add the contents of the file as:

Section "Monitor"
        Identifier "HDMI1"
        VendorName "KJT
        ModelName "78c5"
    Option "PreferredMode" "2560x1440"
    ModeLine "2560x1440" 241.50 2560 2608 2640 2720 1440 1442 1447 1481 +hsync +vsync
    ModeLine "1920x1080" 172.80 1920 2040 2248 2576 1080 1081 1084 1118 -hsync +vsync
endSection

Modify the Identifier to match the name used by xrandr to show the connected monitor. VendorName and ModelName are optional, or you can modify to match your hardware. Use the modelines as appropriate for your monitor. You only need to add modes that are over 165MHz (the first number after the mode name). The rest of the modes will be added from the EDID. You must specify the PreferredMode or it will not use overscan modes by default.

VendorName and ModelName can usually be found in Xorg.0.log. If the ModeLines are not displayed in your Xorg.0.log file, you may wish to add a "Device" section that has a line 'Option "ModeDebug" "True"' to get more output from the modesetting module.

I'm still trying to figure out how to set the kernel frame buffer mode.