SDL applications crash X server

[Malte S. Stretz]

Binary package hint: xserver-xorg

I'm not sure what should happen when I start an SDL application (like qemu/kvm) under X, but the X server crashing is definitely the wrong choice.

From the Xorg.0.log (attached):
Backtrace:
0: /usr/bin/X(xf86SigHandler+0x65) [0x480f35]
1: /lib/libc.so.6 [0x7f30fb7e4060]
2: /usr/bin/X(VidModeGetFirstModeline+0x85) [0x485995]
3: /usr/bin/X(VidModeGetNumOfModes+0x42) [0x485a32]
4: /usr/lib/xorg/modules/extensions//libextmod.so [0x7f30fb0fb031]
5: /usr/bin/X(Dispatch+0x364) [0x44d6e4]
6: /usr/bin/X(main+0x45d) [0x4336fd]
7: /lib/libc.so.6(__libc_start_main+0xe6) [0x7f30fb7cf466]
8: /usr/bin/X [0x432ad9]
Saw signal 11. Server aborting.

As you can see from the log I've got some X issues on this system anyway, so it might be a radeon driver bug as well. Might also be related to my other bug 298094.

mss@Otherland:~$ lsb_release -rd
Description: Ubuntu 8.10
Release: 8.10
mss@Otherland:~$ uname -a
Linux Otherland 2.6.27-7-generic #1 SMP Tue Nov 4 19:33:06 UTC 2008 x86_64 GNU/Linux
mss@Otherland:~$ apt-cache policy xserver-xorg xserver-xorg-video-ati xserver-xorg-video-radeon
xserver-xorg:
  Installed: 1:7.4~5ubuntu3
  Candidate: 1:7.4~5ubuntu3
  Version table:
 *** 1:7.4~5ubuntu3 0
        500 http://de.archive.ubuntu.com intrepid/main Packages
        100 /var/lib/dpkg/status
xserver-xorg-video-ati:
  Installed: 1:6.9.0+git20081003.f9826a56-0ubuntu2
  Candidate: 1:6.9.0+git20081003.f9826a56-0ubuntu2
  Version table:
 *** 1:6.9.0+git20081003.f9826a56-0ubuntu2 0
        500 http://de.archive.ubuntu.com intrepid/main Packages
        100 /var/lib/dpkg/status
xserver-xorg-video-radeon:
  Installed: 1:6.9.0+git20081003.f9826a56-0ubuntu2
  Candidate: 1:6.9.0+git20081003.f9826a56-0ubuntu2
  Version table:
 *** 1:6.9.0+git20081003.f9826a56-0ubuntu2 0
        500 http://de.archive.ubuntu.com intrepid/main Packages
        100 /var/lib/dpkg/status

[Bryce Harrington (bryce)]

Please collect a full backtrace on this crash. See http://wiki.ubuntu.com/X/Backtracing for directions.

[Malte S. Stretz (mss)]

Seems like this is yet another radeon bug (like so many I encountered). I switched to fglrx a week or so ago and now SDL doesn't crash the xserver anymore. I'll have a try with the radeon driver later to get a full backtrace.

[Malte S. Stretz (mss)]

Here is the full backtrace:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f827623b6e0 (LWP 8148)]
VidModeGetFirstModeline (scrnIndex=<value optimized out>, mode=0x7fff7e25b778, dotClock=0x7fff7e25b784)
    at ../../../../hw/xfree86/common/xf86VidMode.c:229
229 ../../../../hw/xfree86/common/xf86VidMode.c: No such file or directory.
        in ../../../../hw/xfree86/common/xf86VidMode.c
(gdb) backtrace full
#0 VidModeGetFirstModeline (scrnIndex=<value optimized out>, mode=0x7fff7e25b778,
    dotClock=0x7fff7e25b784) at ../../../../hw/xfree86/common/xf86VidMode.c:229
        pScrn = (ScrnInfoPtr) 0x25925b0
#1 0x0000000000485a32 in VidModeGetNumOfModes (scrnIndex=39526608)
    at ../../../../hw/xfree86/common/xf86VidMode.c:451
        mode = (pointer) 0x0
        dotClock = 0
        nummodes = 0
#2 0x00007f82738f5031 in ProcXF86VidModeGetAllModeLines (client=0x289ea60)
    at ../../../../../hw/xfree86/dixmods/extmod/xf86vmode.c:534
        rep = {type = 40 '(', pad1 = 69 'E', sequenceNumber = 126, length = 0, modecount = 4527588,
  pad2 = 0, pad3 = 0, pad4 = 0, pad5 = 42592864, pad6 = 0}
        mdinf = {dotclock = 0, hdisplay = 0, hsyncstart = 0, hsyncend = 0, htotal = 0, hskew = 0,
  vdisplay = 0, vsyncstart = 0, vsyncend = 0, vtotal = 0, pad1 = 16751, flags = 0, reserved1 = 0,
  reserved2 = 0, reserved3 = 39886816, privsize = 0}
        oldmdinf = {dotclock = 8275232, hdisplay = 0, hsyncstart = 0, hsyncend = 134, htotal = 0,
  vdisplay = 0, vsyncstart = 0, vsyncend = 30928, vtotal = 633, flags = 0, privsize = 5319427}
        mode = <value optimized out>
        modecount = <value optimized out>
        dotClock = <value optimized out>
        ver = 2
#3 0x000000000044d6e4 in Dispatch () at ../../dix/dispatch.c:454
        result = <value optimized out>
        client = (ClientPtr) 0x289ea60
        nready = 0
        start_tick = 460
#4 0x00000000004336fd in main (argc=8, argv=0x7fff7e25ba48, envp=<value optimized out>)
    at ../../dix/main.c:441
        i = 1
        error = 0
        xauthfile = <value optimized out>
        alwaysCheckForInput = {0, 1}

[Bryce Harrington (bryce)]

Hi Malte,

Thanks I had a chance to investigate this a bit. The backtrace seems pretty clear from that where the segmentation fault is happening, although it's a bit perplexing how the server can get into that particular state. Maybe SDL is futzing up the server's modes data somehow. Seems quite plausible that this and bug 298094 have the same root cause. I reviewed the edid in that bug and it *seems* okay, although your xrandr output indicates that something obviously isn't wrong. Maybe the xserver's edid parser has a bug in it, but we'll leave that analysis to the other bug.

Anyway, whatever the root cause, this seems to be a crash in the xserver's video selection logic, attached is a change that should patch up that crash at least.

[Frans Pop (elendil)]

Created an attachment (id=22086)
Full Xorg log

I have a HP 2510p notebook with docking station running Debian Lenny, kernel is self-compiled 2.6.28.

When I log in to KDE, a script is started automatically that "listens" for dock/undock events and automatically calls xrandr to enable/disable dual-display (Xinerama).

I get an Xserver crash if I start a VirtualBox 2.1.0 virtual machine after undocking the notebook while the LVDS is off. After the crash Xorg.0.log.old contains:
<snip>
Backtrace:
0: /usr/bin/X(xf86SigHandler+0x6a) [0x48dd0a]
1: /lib/libc.so.6 [0x7fe702829f60]
2: /usr/bin/X(VidModeGetFirstModeline+0x5b) [0x49053b]
3: /usr/bin/X(VidModeGetNumOfModes+0x42) [0x490772]
4: /usr/lib/xorg/modules/extensions//libextmod.so [0x7fe701c60040]
5: /usr/bin/X(Dispatch+0x342) [0x44f7e2]
6: /usr/bin/X(main+0x4a5) [0x436bd5]
7: /lib/libc.so.6(__libc_start_main+0xe6) [0x7fe7028161a6]
8: /usr/bin/X(FontFileCompleteXLFD+0x281) [0x435e99]

Fatal server error:
Caught signal 11. Server aborting
</snip>

After the crash I see display corruption of VT1 (after switching consoles or during shutdown).
I am using the VESA framebuffer on VT1 (vga=791). The corruption is that after a console switch most of the display is either black or shows part of the KDE desktop; at the bottom there is an area of grey horizontal stripes, and at the top there is a small bar where the text of the console looks to be displayed, but in an extremely small font (unreadable, but you can just recognize console messages "scrolling" in that area).
Switching back from VT1 to X.Org works correctly, and if I log in to X.Org again and retry VirtualBox it works correctly. A reboot is needed to get VT1 displayed correctly again.

The issue can be reproduced both with PAT and without PAT enabled for the kernel and also happens with kernel version 2.6.26.3. I've seen it happen with both with VirtualBox 2.0.1 and 2.1.0.

The crash can be reproduced as follows:
- boot with notebook docked
- log in to KDE and activate dual display with:
  xrandr --output VGA --left-of LVDS --mode 1280x1024
- disable the laptop display:
  xrandr --output LVDS --off
- undock the notebook, which results in:
  xrandr --output LVDS --auto --output VGA --off
- start VirtualBox virtual machine

The crash does not happen if I redock the notebook before starting VirtualBox. The display corruption on VT1 only happens after an X.Org crash.

I can also reproduce it if I disable the dock/undock script and instead run these xrandr commands manually.

Interesting is that the crash does not happen if I split the last xrandr command and first activate the LVDS and only then disable the VGA.
So if instead of:
xrandr --output LVDS --auto --output VGA --off
I do:
xrandr --output LVDS --auto
xrandr --output VGA --off

[Frans Pop (elendil)]

Additional info

1) I also see the crash with other programs than VirtualBox. Example is the
   game "einstein".
2) The workaround I mentioned (to first enable LVDS) does not prevent
   the crash after all.

[Julien Cristau (jcristau)]

*** Bug 19948 has been marked as a duplicate of this bug. ***

[Bryce Harrington (bryce)]

Created an attachment (id=22576)
check_null_modes.patch

[Frans Pop (elendil)]

The proposed patch fails to compile for me with Debian's xserver-xorg package (1.4.2) as the pVidMode struct does not have a modes variable.

I tried the following instead and with that I no longer got the crash in a quick first test:
     pScrn = xf86Screens[scrnIndex];
     pVidMode = VMPTR(pScrn->pScreen);
+ if (pScrn->modes == NULL)
+ return FALSE;
+
     pVidMode->First = pScrn->modes;
     pVidMode->Next = pVidMode->First->next;

On IRC Julien Cristau confirmed that this should be correct.

[Bryce Harrington (bryce)]

Hi Malte,

I've forwarded your bug upstream to https://bugs.freedesktop.org/show_bug.cgi?id=19948 - please subscribe to that bug in case upstream needs further information or wishes you to test something. Thanks ahead of time!

[Bryce Harrington (bryce)]

Created an attachment (id=22595)
157_check_null_modes.patch

Fixed

[Bryce Harrington (bryce)]

Patch updated and committed to xserver git; will be uploaded after archive is unfrozen.

[Launchpad Janitor (janitor)]

This bug was fixed in the package xorg-server - 2:1.5.99.902-0ubuntu2

---------------
xorg-server (2:1.5.99.902-0ubuntu2) jaunty; urgency=low

  [Bryce Harrington]
  * Add 157_check_null_modes.patch: Catch null pointer dereference in
    video mode selection, which can cause xserver crash when using SDL
    applications with qemu/kvm.
    (LP: #300310)
  * Add 158_raise_maxclients.patch to raise max number of clients from 256
    to 512. Trade-off is that this reduces client resources available to
    1,048,576 total resources (which should still be ample).
    (LP: #260138)

  [Steven Harms]
  * 159_xinerama_focus.patch: Resolves xinerama focus issues
    with multiple screens
    (LP: #41301)

 -- Bryce Harrington <email address hidden> Wed, 04 Feb 2009 22:33:28 -0800