[RV380] X-Server segmentation fault (signal 11) when totem or xine render a video beyond screen dimensions (using overlay mode due to nomodeset kernel option)

[Oliver Joos]

Binary package hint: xserver-xorg-video-ati

[This bug is in Lucid 10.04.1 too - see my comment #3]

I use the brand-new iso released today to boot Ubuntu 10.10. By default no crash occurs. I see this bug only if Kernel parameter "nomodeset" is added. Unfortunately kernel mode setting still makes kernel freezes on most ATI systems I know including mine. Dual screen, googleearth, sleep-resume only work reliable if nomodeset is added. This bug report is NOT about problems with kernel mode setting!

Steps to reproduce:
1. boot with Kernel parameter "nomodeset"
2. add "universe" to package repositories and reload package lists
3. double-click a video (see below for a crashing example video)
4. let the assistant install gstreamer0.10-ffmpeg and gstreamer0.10-plugins-bad
5. close totem, then double-click the video again
6. press F for fullscreen
7. press ctrl+R to zoom in
=> X-Server crashes

My laptop graphics adapter:
01:00.0 VGA compatible controller: ATI Technologies Inc M24 1P [Radeon Mobility X600]

A video that crashes is downloadable here: http://kai3.tv/cc2/CC-Zwei-47-NRWTV.mp4

In Ubuntu Lucid 10.04 this crash did never occur!

Excerpt of /var/log/Xorg.0.log.old after the crash:
-----------------------------------------------------------------
[ 854.333] (II) RADEON(0): Output: VGA-0, Detected Monitor Type: 0
[ 854.338] (II) RADEON(0): Output: DVI-0, Detected Monitor Type: 0
[ 854.341] (II) RADEON(0): Output: LVDS, Detected Monitor Type: 2
[ 854.341] (II) RADEON(0): Added native panel mode: 1680x1050
[ 854.341] (II) RADEON(0): Output: S-video, Detected Monitor Type: 0
[ 855.012] Changing OV0_BASE_ADDR from 0xc0000000 to 0xc3c00000
[ 859.516]
Backtrace:
[ 859.516] 0: /usr/bin/X (xorg_backtrace+0x3b) [0x80e82fb]
[ 859.516] 1: /usr/bin/X (0x8048000+0x5da8d) [0x80a5a8d]
[ 859.516] 2: (vdso) (__kernel_rt_sigreturn+0x0) [0xdce40c]
[ 859.516] 3: /usr/lib/xorg/modules/drivers/radeon_drv.so (0x20a000+0x40204) [0x24a204]
[ 859.516] 4: /usr/lib/xorg/modules/drivers/radeon_drv.so (0x20a000+0x4264b) [0x24c64b]
[ 859.516] 5: /usr/bin/X (0x8048000+0x139e66) [0x8181e66]
[ 859.516] 6: /usr/lib/xorg/modules/extensions/libextmod.so (XvdiPutImage+0x186) [0x1322e6]
[ 859.517] 7: /usr/lib/xorg/modules/extensions/libextmod.so (0x128000+0xe31b) [0x13631b]
[ 859.517] 8: /usr/lib/xorg/modules/extensions/libextmod.so (ProcXvDispatch+0x3a) [0x13727a]
[ 859.517] 9: /usr/bin/X (0x8048000+0x26087) [0x806e087]
[ 859.517] 10: /usr/bin/X (0x8048000+0x1a5ba) [0x80625ba]
[ 859.517] 11: /lib/libc.so.6 (__libc_start_main+0xe7) [0x493ce7]
[ 859.517] 12: /usr/bin/X (0x8048000+0x1a191) [0x8062191]
[ 859.517] Segmentation fault at address 0xa64c9000
[ 859.517]
Caught signal 11 (Segmentation fault). Server aborting
[ 859.517]
Please consult the The X.Org Foundation support
  at http://wiki.x.org
 for help.
[ 859.517] Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[ 859.517]
-----------------------------------------------------------------

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: xserver-xorg-video-radeon 1:6.13.1-1ubuntu5
ProcVersionSignature: Ubuntu 2.6.35-22.33-generic 2.6.35.4
Uname: Linux 2.6.35-22-generic i686
Architecture: i386
Date: Sun Oct 10 20:59:00 2010
GdmLog1: Error: command ['gksu', '-D', 'Apport', '--', 'cat', '/var/log/gdm/:0.log.1'] failed with exit code 1: cat: /var/log/gdm/:0.log.1: No such file or directory
GdmLog2: Error: command ['gksu', '-D', 'Apport', '--', 'cat', '/var/log/gdm/:0.log.2'] failed with exit code 1: cat: /var/log/gdm/:0.log.2: No such file or directory
LiveMediaBuild: Ubuntu 10.10 "Maverick Meerkat" - Release i386 (20101007)
MachineType: Hewlett-Packard HP Compaq nx8220 (PY517EA#ABD)
PccardctlIdent:
 Socket 0:
   no product info available
PccardctlStatus:
 Socket 0:
   no card
ProcCmdLine: BOOT_IMAGE=(loop)/casper/vmlinuz boot=casper iso-scan/filename=/boot/iso/hostname-10.10-desktop-i386.iso noeject noprompt nomodeset usbhid.mousepoll=8 --
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: xserver-xorg-video-ati
dmi.bios.date: 07/24/2009
dmi.bios.vendor: Hewlett-Packard
dmi.bios.version: 68DTV Ver. F.18
dmi.board.name: 0934
dmi.board.vendor: Hewlett-Packard
dmi.board.version: KBC Version 43.1D
dmi.chassis.asset.tag: oliverhp
dmi.chassis.type: 10
dmi.chassis.vendor: Hewlett-Packard
dmi.modalias: dmi:bvnHewlett-Packard:bvr68DTVVer.F.18:bd07/24/2009:svnHewlett-Packard:pnHPCompaqnx8220(PY517EA#ABD):pvrF.18:rvnHewlett-Packard:rn0934:rvrKBCVersion43.1D:cvnHewlett-Packard:ct10:cvr:
dmi.product.name: HP Compaq nx8220 (PY517EA#ABD)
dmi.product.version: F.18
dmi.sys.vendor: Hewlett-Packard
glxinfo: Error: [Errno 2] No such file or directory
system:
 distro: Ubuntu
 codename: maverick
 architecture: i686
 kernel: 2.6.35-22-generic

[Oliver Joos (oliver-joos)]

The crash is reproducible with the final LiveCD of 10.10, zooming into a x264-Video. Zooming out (image gets smaller) works. Other video formats (xvid in .avi or .ogm) do not crash.

xine chooses "xv" as video_out plugin. With video_out="opengl" the crash does not occur, because zooming is not possible then. With video_out="xvmc" xine crashes the x-server immediately after its start, even without a video!!

So far I guess:
As totem is also affected, it seems to be a problem with an underlying x264-library feeding reckless dimensions through xv to the the driver xserver-xorg-video-ati, which then crashes. If this is true, then both the x264-library and the xserver-xorg-video-ati would need a fix, while the latter is more important to avoid future crashes caused by problematic libraries. I'll try to find the guilty file by mixing versions of Maverick and Lucid, where this bug did never occur.

[Oliver Joos (oliver-joos)]

Today the same segmentation fault occurred in my up-to-date Lucid 10.04.1 (installed on HD)! While xine was playing a divx video I changed the monitor layout to dual screen and moved the xine window to the second screen (right of the main screen). As soon as the window got a few pixels too much right (outside of the second screen) the X-Server crashed. Looking e.g. at line 6 of backtrace make me think that this is the same problem.

Backtrace:
0: /usr/bin/X (xorg_backtrace+0x3b) [0x80e938b]
1: /usr/bin/X (0x8048000+0x61c8d) [0x80a9c8d]
2: (vdso) (__kernel_rt_sigreturn+0x0) [0x1b7410]
3: /usr/lib/xorg/modules/drivers/radeon_drv.so (0x4bb000+0x40fd4) [0x4fbfd4]
4: /usr/lib/xorg/modules/drivers/radeon_drv.so (0x4bb000+0x46d27) [0x501d27]
5: /usr/bin/X (0x8048000+0x1370be) [0x817f0be]
6: /usr/lib/xorg/modules/extensions/libextmod.so (XvdiPutImage+0x186) [0xdbc546]
7: /usr/lib/xorg/modules/extensions/libextmod.so (0xdb2000+0xe366) [0xdc0366]
8: /usr/lib/xorg/modules/extensions/libextmod.so (ProcXvDispatch+0x3a) [0xdc13ca]
9: /usr/bin/X (0x8048000+0x2a477) [0x8072477]
10: /usr/bin/X (0x8048000+0x1ed7a) [0x8066d7a]
11: /lib/tls/i686/cmov/libc.so.6 (__libc_start_main+0xe6) [0x1cebd6]
12: /usr/bin/X (0x8048000+0x1e961) [0x8066961]
Segmentation fault at address 0xaaa95000

Caught signal 11 (Segmentation fault). Server aborting

Now my only options are to go back to Karmic or trying another disto. :-(

[bugbot (bugbot)]

Hey Oliver,

Hi, have you had a chance to test if this bug is still present in natty?

If it does (and if you're the original reporter), please boot into natty
and run the command:

  apport-collect <bug-number>

which will update the bug with fresh logs and tag the bug as affecting
natty. (It is best to run this right after reproducing the problem.)

[Oliver Joos (oliver-joos)]

Upto Lucid I had no problems with UMS (fast & stable 2D+3D), but many problems with KMS. That's why this bug has hurt me quite much in Maverick.

Now in Natty it's the opposite: with KMS the only problem is video playback tearing (not in sync with vblank), but without KMS there are severe redraw problems in 2D (at least with Composite enabled) and 3D is unusably slow (due to new Gallium driver, I guess)

In short: I was not able to reproduce this bug with Natty (no wonder with all these changes in gfx subsystem). So I'll skip Maverick, leave UMS and overlay mode, and hope that despite KMS and textured video it will soon be possible to stay in sync with vblank again...

[bugbot (bugbot)]

We're closing this bug since it is has been some time with no response from the original reporter. However, if the issue still exists please feel free to reopen with the requested information. Also, if you could, please test against the latest development version of Ubuntu, since this confirms the bug is one we may be able to pass upstream for help.