orca: alt-tabbing between windows - descriptions become interleaved confusingly

[Alan Jenkins]

Ubuntu 11.04
gnome-orca-3.0.0-0ubuntu2
"Gnome classic" environment

1. Enable orca screen reader.
2. You should now have the orca window open. Close any other active applications.
3. Start another application - I've tried gnome-terminal and Calculator.
4. Press alt-tab to switch to the other application. Orca will read
  a) it's title
  b) a description of the active widget
 "e.g. Orca Screen reader slash magnifier frame, preferences button".
5. Press alt-tab again, to switch to the other application

After just a few repetitions - giving orca plenty of time to read out the full description, not interrupting it half way through - Orca starts saying insane things like

"Orca Screen Reader slash magnifier frame, Terminal frame, preferences button, terminal".

[William-walker-z (william-walker-z)]

a11y == accessibility

I work on the Orca screen reader project (http://live.gnome.org/Orca). As we have done with metacity, we are trying to provide compelling presentation as the user presses Alt+Tab to navigate between windows on the desktop using Compiz.

As seen in http://bugzilla.gnome.org/show_bug.cgi?id=466841#c8, we are noticing some really strange behavior when the user releases the Tab key after pressing Alt+Tab. In particular, given the scenario where the user Alt+Tab's from window A to window B, we see the following sequence of AT-SPI events:

(Alt+Tab initially pressed here)

A: window:deactivate

(Alt+Tab released here)

A: window-activate
B: window:activate
A: window:deactivate

This overlapping of window activation events is really quite strange (it says two windows are active at once). Ideally we'd just get this:

(Alt+Tab initially pressed here)

A: window:deactivate

(Alt+Tab released here)

B: window:activate

Thanks!

[William-walker-z (william-walker-z)]

In testing more with compiz 0.7.7 on the latest Ubuntu Intrepid, I'm also seeing this event ordering when Alt+Tab'ing from A to B:

(Alt+Tab initially pressed here)

A: window:deactivate

(Alt+Tab released here)

B: window-activate
A: window:activate
A: window:deactivate

Again, ideally we'd just get this:

(Alt+Tab initially pressed here)

A: window:deactivate

(Alt+Tab released here)

B: window:activate

Thanks!

[Maniac-y (maniac-y)]

What exactly triggers those window:activate and window:deactivate events? I mean, what X event is the root of those?

[William-walker-z (william-walker-z)]

(In reply to comment #2)
> What exactly triggers those window:activate and window:deactivate events? I
> mean, what X event is the root of those?

I'm not sure. I'm guessing FocusIn and FocusOut.

This might be ultimately done by moveInputFocusToWindow in http://cgit.freedesktop.org/xorg/app/compiz/tree/src/window.c, which is called by activateWindow and hideWindow in the same module.

But don't let the above trick you into thinking I know anything about compiz. :-) I don't. I was just scouring sources.

Speaking of scouring, I did a little more, but this is just naive scouring: on the GTK+ side, gtk+/gdk/x11/gdkevents-x11.c listens for FocusIn/FocusOut events and generates a GDK_FOCUS_EVENT. This is processed by gtk+/gdk/gdkevents.c, and things end up getting bubbled up to gtk+, which turns them into 'focus-in-event' and 'focus-out-event' events. The AT-SPI GAIL module picks these up and issues "window:activate" and "window:deactivate" events accordingly.

[Maniac-y (maniac-y)]

Ok, it most likely _is_ FocusIn / FocusOut and also most likely is grab related.

Here are the FocusIn and FocusOut events happening on a switch gnome-terminal -> gcalctool:

focusOut 119537694 (gnome-terminal), mode 1, detail 1
focusIn 422 (), mode 1, detail 2
focusIn 119537694 (gnome-terminal), mode 1, detail 5
---> switcher plugin initiated
focusOut 119537694 (gnome-terminal), mode 1, detail 5
focusOut 422 (), mode 1, detail 2
focusIn 146800729 ((null)), mode 1, detail 0
moveInputFocusToWindow 136314883 (galculator)
focusOut 146800729 ((null)), mode 2, detail 3
focusIn 119537694 (gnome-terminal), mode 2, detail 4
focusOut 119537694 (gnome-terminal), mode 0, detail 4
focusIn 136314883 (galculator), mode 0, detail 3
focusOut 136314883 (galculator), mode 0, detail 2

The first three are the keyboard grab becoming active due to Alt+Tab being pressed. The next three are the pointer grab becoming active through switcher. The last 5 happen after releasing Alt+Tab and thus releasing both grabs and moving the focus.
The printed data are the window, mode and detail members of the XFocusChangeEvent struct.

I don't have the time to do a deeper analysis at this moment (will do as soon as I have some time), but if you have some X/Gdk expert at hand, that'd be helpful ;)

[William-walker-z (william-walker-z)]

(In reply to comment #4)
> Ok, it most likely _is_ FocusIn / FocusOut and also most likely is grab
> related.

Nice analysis. Thanks! How did you do it? It would be really interesting to analyze the events from metacity in the same way. As we see in http://bugzilla.gnome.org/show_bug.cgi?id=466841#c9, it seems to be able to do the right thing. So, hopefully it will just be a matter of trying to duplicate that.

[Maniac-y (maniac-y)]

(In reply to comment #5)
> (In reply to comment #4)
> > Ok, it most likely _is_ FocusIn / FocusOut and also most likely is grab
> > related.
>
> Nice analysis. Thanks! How did you do it?

I added debug output to the FocusIn / FocusOut event handlers of Compiz. As it selects for FocusChange events on all toplevel windows anyway, all events are printed.

[Joanmarie (joanmarie-diggs-deactivatedaccount)]

Ping. :-)

As more and more distros are shipping with Compiz as the default window manager, this bug is impacting more and more users who are blind.

[Alan Jenkins (aj504)]

Secondly, the description is often cut off altogether.

I think this reproduces quite reliably with the gnome help browser, i.e.:

1. Click on the "Help" button in the Orca window.
You should now see the documentation for Orca, and start to hear the entire index page being read out.

2. Press alt-tab to switch back to the main Orca window. Wait for both the window title and button name to finish being read out.
3. Press alt-tab to switch back to the Help window.

The description you now hear will sound something like "Ye.." (yelp) or "Or..." (Orca ...).

[Alan Jenkins (aj504)]

Aha! This doesn't happen in "Ubuntu classic (no effects)". So it doesn't happen with metacity; it only happens if you're running compiz.

[Alan Jenkins (aj504)]

This is also described in comments to ubuntu bug #127705

[Alan Jenkins (aj504)]

Sorry, my previous comment is wrong. I must have confused the Ubuntu bug with something else - I'm not sure what.

[Luke Yelavich (themuso)]

Compiz 0.9 currently doesn't show the window title on-screen, or expose it via accessibility. Compiz 0.8 used to do this, but it was lost in the 0.9 rewrite, which is not so good.

[Alan Jenkins (aj504)]

This specific report is about what happens after you've switched windows with alt-tab, and Orca describes the new window (including the word "frame", to separate the window title from the active widget description) Apparently (see linked bugs) compiz is generating a spurious "focus" event for the old window.

The issue with the window switcher being silent is a separate bug.

[Alan Jenkins (aj504)]

(a separate bug which I didn't bother to report, because I thought it was so obvious, I wasn't originally sure what the desired behavior was, and then I saw it reported elsewhere anyway, so there didn't seem much point).

[Alan Jenkins (aj504)]

Ubuntu 11.10: the descriptions are no longer interleaved, but there still seem to be spurious focus events. So when switching from Orca to Terminal, I get

"Orca Frame, Preferences button
 Terminal Frame, Terminal"

when what I should hear is just " Terminal Frame, Terminal".

I think this only happens under compiz. So I can only reproduce this by running Orca under Unity 3D.

[Alan Jenkins (aj504)]

Ubuntu 12.04: unity-3d is still affected. Haven't noticed any problem in unity-2d.