segfaults in VESAPreInit, corrupt mode list

[Tormod Volden]

I first thought this was a duplicate of bug #89853, but it is not, although many comments in that bug might regard this very issue.

Running Tribe 5 on ATI X700, forcing the vesa driver, the X server crashes and leaves a backtrace in its log:

(WW) VESA(0): No valid modes left. Trying less strict filter...

Backtrace:
0: /usr/bin/Xorg(xf86SigHandler+0x81) [0x80e6471]
1: [0xffffe420]
2: /usr/bin/Xorg(InitOutput+0x9a4) [0x80a8e34]

Running under gdb shows that it crashes in line 684 of vesa.c, due to a corrupt linked list pScrn->display->modes

[Tormod Volden (tormodvolden)]

Seems identical to a crash on X1400, reported in April: https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/89853/comments/98

[brokencrystal.com (admin-brokencrystal)]

https://bugs.launchpad.net/ubuntu/+bug/131799

[Tormod Volden (tormodvolden)]

It is quite possible there is a relation to bug 89853 and bug 103945, but in this case there is a segfault crash. Many of the me-too'ers on those bugs have the same backtrace as here.

[Tormod Volden (tormodvolden)]

My theory is that pScrn->display->modes is not initialized or is overwritten. In some cases its value happens to be null, and you get no crash but only "no valid modes", in my case it's not null and the data structure is random and you get the crash in line src/vesa.c:684.

[Tormod Volden (tormodvolden)]

Should be fixed with 1:1.3.0-1ubuntu5

[Tormod Volden (tormodvolden)]

Verified with Tribe 5 live cd + updated vesa driver.

xserver-xorg-video-vesa (1:1.3.0-1ubuntu5) gutsy; urgency=low

  * debian/patches/100-fedora-vesa-1.3.0-mode-heuristics.patch
    - update, this should fix #89853 for good.

 -- Timo Aaltonen <email address hidden> Thu, 30 Aug 2007 23:21:03 +0300