Compiz r3275 breaks VirtualBox Guest Additions: black screen or just wallpaper

Bug #1030891 reported by Łukasz Zemczak on 2012-07-30
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Compiz
High
Sam Spilsbury
Virtualbox
Fix Released
Unknown
virtualbox-guest-additions (Ubuntu)
Undecided
Sam Spilsbury

Bug Description

I already reported this to Daniel, but the lp:compiz trunk does not work on the VirtualBox VM - while the current quantal version (1:0.9.8+bzr3249-0ubuntu2) works fine. With some hints from Daniel, I was able to bisect and find that the problem has been introduced by fix in revision 3275. Both 3273 and 3274 seem to work fine on VirtualBox, while starting with 3275 - compiz does not start and seems to enter some endless loop somewhere, making the system unresponsive and unusable.

After logging in with this broken compiz in VirtualBox, the only thing that appears on screen is either the background or a black screen - the mouse can be moved, but no action can be performed (not even opening the terminal by keybindings).

From a previous run, this is what Ctrl+C'ing from the locked compiz (when running with unity --debug) gives me from gdb:
http://paste.ubuntu.com/1109878/

Related branches

Sam Spilsbury (smspillaz) wrote :

This indicates a broken driver to me, as the spec says you're not supposed to do that:

#5 0x00007ffff6d22a41 in XOpenDisplay () from /usr/lib/x86_64-linux-gnu/libX11.so.6
No symbol table info available.
#6 0x00007fffea05fc62 in vboxstub_glXBindTexImageEXT () from /usr/lib/x86_64-linux-gnu/dri/vboxvideo_dri.so
No symbol table info available.

---

http://developer.download.nvidia.com/opengl/specs/GLX_EXT_texture_from_pixmap.txt

5. Should users be required to re-bind the drawable to a texture after
    the drawable has been rendered to?

    It is difficult to define what the contents of the texture would be if
    we don't require this. Also, requiring this would allow implementations
    to perform an implicit copy at this point if they could not support
    texturing directly out of renderable memory.

    The problem with defining the contents of the texture after rendering
    has occured to the associated drawable is that there is no way to
    synchronize the use of the buffer as a source and as a destination.
    Direct OpenGL rendering is not necessarily done in the same command
    stream as X rendering. At the time the pixmap is used as the source
    for a texturing operation, it could be in a state halfway through a
    copyarea operation in which half of it is say, white, and half is the
    result of the copyarea operation. How is this defined? Worse, some
    other OpenGL application could be halfway through a frame of rendering
    when the composite manager sources from it. The buffer might just
    contain the results of a "glClear" operation at that point.

    To gurantee tear-free rendering, a composite manager would run as follows:

    -receive request for compositing:
    XGrabServer()
    glXWaitX() or XSync()
    glXBindTexImageEXT()

    <Do rendering/compositing>

    glXReleaseTexImageEXT()
    XUngrabServer()

In any case we'll have to work around it, I don't want to revert this revision as it will cause regressions on nvidia.

Changed in compiz:
assignee: nobody → Sam Spilsbury (smspillaz)
milestone: none → 0.9.8.0
Sam Spilsbury (smspillaz) wrote :

Okay,

I had a quick look into the driver and this is what is happening:

 -> glxBindTexImageEXT is bound to DECLEXPORT(void) VBOXGLXTAG(glXBindTexImageEXT)(Display *dpy, GLXDrawable draw, int buffer, const int *attrib_list)
 -> If a pGlxPixmap isn't found it calls stubInitGlxPixmap, which then calls stubInitXDamageExtension
 -> This opens a new X connection in the same process, which deadlocks.

The driver should never be opening a new X connection, but there's a comment there which suggests that they have to do it, possibly because they were working around some broken client's bugs.

"/* Open second xserver connection to make sure we'd receive all the xdamage messages
     * and those wouldn't be eaten by application even queue */
    pContext->damageDpy = XOpenDisplay(DisplayString(pContext->dpy));"

So I think there are two ways around this:

 1) Fix virtualbox so that it does this during glXCreateContext (also broken, but slightly less so)
 2) Work around it in compiz by not doing the server grab for the first pixmap we bind

Sam Spilsbury (smspillaz) wrote :

I have no idea if this patch on virtualbox will even work or compile, but it should work from a 5 minute gloss over the source.

It causes the XDamage extension and subsequent call to XOpenDisplay to happen in glXCreateContext and not glXBindTexImageEXT

Changed in compiz:
status: New → Invalid
Changed in virtualbox:
assignee: nobody → Sam Spilsbury (smspillaz)
Changed in virtualbox-guest-additions (Ubuntu):
assignee: nobody → Sam Spilsbury (smspillaz)
Changed in virtualbox:
assignee: Sam Spilsbury (smspillaz) → nobody
Sam Spilsbury (smspillaz) wrote :

I've marked the compiz bug invalid

The attachment "Do xdamage init a bit earlier" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags: added: patch

Thanks for the VirtualBox driver fix Sam! I will try testing this in a moment, compiling VB additions from source. But anyway, it seems to be a bug that probably should be reported upstream, don't you think?

Changed in compiz:
status: Invalid → Opinion
Changed in compiz:
milestone: 0.9.8.0 → 0.9.8.1
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in virtualbox-guest-additions (Ubuntu):
status: New → Confirmed
Sam Spilsbury (smspillaz) wrote :

Bug forwarded upstream.

no longer affects: virtualbox
Changed in compiz:
status: Opinion → Invalid
Changed in virtualbox:
status: Unknown → New
Changed in compiz:
milestone: 0.9.8.2 → none
summary: - Revision 3275 breaks compiz in VirtualBox
+ Compiz r3275 breaks VirtualBox Guest Additions: black screen or just
+ wallpaper
Changed in compiz:
status: Invalid → In Progress
milestone: none → 0.9.8.4
Changed in compiz:
status: In Progress → Fix Committed
Daniel van Vugt (vanvugt) wrote :

Workaround committed into lp:compiz at revision 3392

Changed in compiz:
status: Fix Committed → Fix Released
Changed in virtualbox:
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.