Some applications don't 'auto-show' onboard

[David López]

I'm testing onboard 0.97 in 2 different computers (one with oneiric: 0.97.0+tr756-0ppa~oneiric1, another one with precise alpha2: 0.97.0-0ubuntu1), both of them with auto-show when editing, show when unlocking, show tooltips, always on top.

Most applications work fine with the new 'auto-show when editing' option, but I've found a few 'text applications' with wrong behaviour:

- If onboard is hidden, they don't unhide (the right bevaviour is unhide)
- If onboard was automatically unhided by another application , they hide onboard (the right behaviour is do nothing). Please note that in case of onboard was unhidden by a 'manual' clicking on onboard's icon, those applications don't hide onboard (right behaviour)

I post all of wrong applications I've found in only one bug, if you prefer I can file different bugs.

- opening the dash: this bug is different than #915250 , and this new bug is present even in unity-2d and with 'always on top' marked
New bug reported here:
https://bugs.launchpad.net/ubuntu/+source/unity/+bug/960537

- wxmaxima: cells and text boxes
- qtoctave: the editor
- skype: the user/password window
- tuxpaint: the option to write text in a image (in some other applications as gimp or xournal this feature works fine)

- the right-click option to rename files in nautilus
New bug reported here:
https://bugzilla.gnome.org/show_bug.cgi?id=672498

- wine: tested with Adob Digital Editions
- texmaker
- fbreader
- vlc (text boxes in preferences)

- chromium-browser (all text entries except the url-bar)
Upstream bug report for Orca here:
https://code.google.com/p/chromium/issues/detail?id=24585

[David López (david-lopez-upct)]

Some new applications I've found:

- texmaker
- fbreader
- vlc (text boxes in preferences)

I think that all of them, skype and obviously qtoctave are qt-aplications, so maybe there's a problem with qt interface. I wish this information helps.

[Francesco Fumanti (frafu)]

Hi David,

Thanks for your interest in Onboard and for helping us make it better.

Concerning your second point in your bug description: when Onboard gets manually unhidden when auto-show is running, the auto-show feature gets paused until the user hides Onboard again. This behaviour was intended by design. However, meanwhile, we wonder whether we should not also add a timeout to restart auto-show after it has been paused by a manual unhide.

Could you please give us more details about what you mean by "opening the dash" and "the right-click option to rename files in nautilus"? What is the relation to auto-show.

Finally, I think that it is fine to list all the applications that do not behave properly with auto-show in the same bug.

[David López (david-lopez-upct)]

Hi Francesco. I'll try to be clearer.

- Open dash (click on the left upper botton with the ubuntu logo). The cursor is in the text box to find applications, so onboard must auto-unhide. However it doesn't.
- Open nautilus, right-click on a file and select Rename... It is not exactly a "text box", but the name of the file is selected and the cursor is on it expecting for a new name, so onboard must auto-unhide, but it remains hidden.

In my opinion the importance of this bugs is low (maybe the dash a little bigger) because manual unhide allows to use onboard even if auto-unhide fails. The design of auto-show is fine, and a timeout will also be nice. Both options are OK while manual unhide/hide "pauses" auto-hide/unhide.

[marmuta (marmuta)]

I finally went through the list. There was one unlisted bug with qtoctave I could fix in Onboard, but the rest would probably need to be filed elsewhere.

- dash
Needs a bug report for unity.
There doesn't appear to be support for at-spi yet.

- wxmaxima: cells and text boxes
Probably needs a bug report for wxwidgets.
Apparently wxwidgets applications don't support at-spi (yet).

- qtoctave: the editor
Needs a bug report for qtoctave.
The editor is marked editable, but has it's role set to ATSPI_ROLE_FILLER. I don't think we should unhide onboard in this case.
Comboboxes in qtoctave didn't unhide the keyboard either, but I've been able to fix this from onboard's side.

- skype: the user/password window
Please run oboard as "onboard -ddebug" and make note of At-spi focus events. If there aren't any I probably can't help unfortunately. Skype is closed source.

- tuxpaint: the option to write text in a image (in some other applications as gimp or xournal this feature works fine)
Needs a bug report for tuxpaint.
This seems to be some custom made text entry without support for at-spi.

- the right-click option to rename files in nautilus
Needs a bug report for nautilus if none exists yet.

- wine: tested with Adob Digital Editions
Needs a bug report for wine. I have no idea if this can ever be fixed though.

- texmaker
Needs a bug report for texmaker.
Same as qtoctave, the editor is marked as editable but has role of ATSPI_ROLE_FILLER. This doesn't seem right.

- fbreader
Preferences are fine apparently. Crashes on me when adding books, though. Where do I have to look?

- vlc (text boxes in preferences)
Works here. Any specific entry where it fails for you?

[marmuta (marmuta)]

Some general notes: onboard's auto-show is based on at-spi focus events. All frameworks and applications supporting at-spi should in theory work with onboard out of the box. AFAIK supported frameworks are gtk, qt and java. Everything outside of those would have to have been coded with at-spi in mind or be ignored by onboard.

There is still a possibility that onboard misses some accessible widgets, e.g. if they are marked with roles and states we haven't encountered before. This can be fixed from Onboard's side. However, I believe by now most of the remaining problems will have to be taken care of by the developers of the affected applications.

There are probably hundreds of applications in the repositories where auto-show doesn't work. We, the Onboard devs, cannot possibly go through all of them. I'd suggest we concentrate on a few high profile ones and file bug reports there.

[David López (david-lopez-upct)]

Thanks for opening the dash and nautilus bugs, marmuta. IMHO only dash is important, the rest of misfunctions are very low importance and don't worth.

Only for your information, in several applications as vlc, fbreader and calibre, unhide only works bad in some textboxes, and it works fine in another ones. skype is quite absurd example: it works fine with password textbox:

2012-04-02 01:44:36,654:DEBUG:KeyboardGTK: At-spi focus event: name=, role=<enum ATSPI_ROLE_COMBO_BOX of type Role>(combo box), editable=True, states=1124075906, extents=Rect(x=766 y=408 w=258 h=27)]
2012-04-02 01:44:36,674:DEBUG:KeyboardGTK: At-spi focus event: name=, role=<enum ATSPI_ROLE_PASSWORD_TEXT of type Role>(password text), editable=True, states=1128274306, extents=Rect(x=766 y=477 w=258 h=27)]
2012-04-02 01:44:36,711:DEBUG:KeyboardGTK: At-spi focus event: name=, role=<enum ATSPI_ROLE_PASSWORD_TEXT of type Role>(password text), editable=True, states=1128274306, extents=Rect(x=766 y=477 w=258 h=27)]
2012-04-02 01:44:36,746:DEBUG:KeyboardGTK: begin opacity transition: 0.0 to 1.0

but it works bad with user textbox:

2012-04-02 01:44:23,373:DEBUG:KeyboardGTK: At-spi focus event: name=Terminal, role=<enum ATSPI_ROLE_TERMINAL of type Role>(terminal), editable=False, states=1124079872, extents=Rect(x=66 y=639 w=722 h=393)]
2012-04-02 01:44:23,395:DEBUG:KeyboardGTK: At-spi focus event: name=Terminal, role=<enum ATSPI_ROLE_TERMINAL of type Role>(terminal), editable=False, states=1124079872, extents=Rect(x=66 y=639 w=722 h=393)]
2012-04-02 01:44:23,601:DEBUG:KeyboardGTK: At-spi focus event: name=, role=<enum ATSPI_ROLE_PASSWORD_TEXT of type Role>(password text), editable=True, states=1128270210, extents=Rect(x=766 y=477 w=258 h=27)]
2012-04-02 01:44:23,627:DEBUG:KeyboardGTK: At-spi focus event: name=, role=<enum ATSPI_ROLE_PASSWORD_TEXT of type Role>(password text), editable=True, states=1128270210, extents=Rect(x=766 y=477 w=258 h=27)]
2012-04-02 01:44:23,680:DEBUG:KeyboardGTK: At-spi focus event: name=, role=<enum ATSPI_ROLE_PASSWORD_TEXT of type Role>(password text), editable=True, states=1128270210, extents=Rect(x=766 y=477 w=258 h=27)]
2012-04-02 01:44:23,797:DEBUG:KeyboardGTK: begin opacity transition: 0.0 to 1.0
2012-04-02 01:44:24,196:DEBUG:ConfigUtils: image «close.svg» aún no encontrada, intentando de nuevo en las rutas predeterminadas
2012-04-02 01:44:24,197:DEBUG:ConfigUtils: image «/usr/share/onboard/layouts/images/close.svg» encontrada.
2012-04-02 01:44:24,198:DEBUG:KeyGTK: loading image '/usr/share/onboard/layouts/images/close.svg'
2012-04-02 01:44:24,212:DEBUG:ConfigUtils: image «show-click.svg» aún no encontrada, intentando de nuevo en las rutas predeterminadas
2012-04-02 01:44:24,214:DEBUG:ConfigUtils: image «/usr/share/onboard/layouts/images/show-click.svg» encontrada.
2012-04-02 01:44:24,215:DEBUG:KeyGTK: loading image '/usr/share/onboard/layouts/images/show-click.svg'
2012-04-02 01:44:24,220:DEBUG:ConfigUtils: image «move.svg» aún no encontrada, intentando de nuevo en las rutas predeterminadas
2012-04-02 01:44:24,229:DEBUG:ConfigUtils: image «/usr/share/onboard/layouts/images/move.svg» encontrada.
2012-04-02 01:44:24,231:DEBUG:KeyGTK: loading image '...

[marmuta (marmuta)]

Thanks for the logs. I made onboard unhide for ATSPI_ROLE_COMBO_BOX a while ago. Could you try the snapshot of 0.97.1?
https://launchpad.net/~onboard/+archive/snapshots

If you come across broken text fields in any of the apps, please let me know which ones they are/where to find them. It may be best to add this to the list in the bug description too.

Concerning the second skype log, it seems the user textbox is invisible to at-spi. I've tried it here too and saw the same, no focus message being generated. I don't think I can do much to help there, unfortunately.

[Jeff Sereno (jsereno)]

I've just observed that OnBoard does not auto-show on any textbox within the Chrome or Chromium web browser (except for the URL text box), but appears to (so far) work perfectly within all text boxes in Firefox.

Also OnBoard does not auto-show for any text box inside Virtualbox's frontend (Obviously I don't expect it to work inside VM's themselves). Not that we need a situation for OnBoard to work when using the Virtualbox frontend, but any other apps under the same UI framework will probably be affected too. Actual text entry with manually shown OnBoard works fine - it's just the auto-show part that doesn't work.

I also echo that OnBoard fails to auto-show for the Unity Dash, but I can type within the Dash if I manually show OnBoard first before bringing up the Dash.

Versions of applications used:

- Ubuntu Precise 12.04.1
- Unity 5.16.0-0ubuntu1 (3D version - not using 2D)
- OnBoard 0.98.2-0ppa~precise~sysdef2
- Firefox 17.0+build2-0ubuntu0.12.04.1
- Chrome 23.0.1271.91-r167501 ("google-chrome-stable" package from Google's "http://dl.google.com/linux/chrome/deb/ stable main" repo)
- Chromium 20.0.1132.47~r144678-0ubuntu0.12.04.1
- Virtualbox 4.2.4-81684~Ubuntu~precise ("virtualbox-4.2" package from Oracle's "http://download.virtualbox.org/virtualbox/debian precise contrib" repo)

[marmuta (marmuta)]

Jeff, thanks for the details.

I can confirm that In Chromium auto-show only works in the url-bar. Apparently linux accessibility is still work in progress:
http://dev.chromium.org/developers/accessibility/linux-accessibility
There's also this open bug report for Orca:
https://code.google.com/p/chromium/issues/detail?id=24585
Onboard's needs are just a small subset of Orca's, so once that bug is closed Onboard's auto-show should be supported as well.

The qt-frontend of VirtualBox does well with auto-show on ubuntu 12.10. I've tested 4.1.18 from quantal's repos, as well as version 4.2 from your link. Both show/hide Onboard for all the text entries I tried.
In theory all qt applications should be supported. The bridge to AT-SPI for the qt toolkit is qt-at-spi, which is installed by default on quantal. Could you check if it is there on precise?

Auto-show with virtualized applications is unlikely to happen, I agree. VirtualBox would need to act as a proxy for the guest's AT-SPI calls for this to work.

Dash on quantal seems to show Onboard sometimes, though not reliably and not for long. It seems there is progress, but the bug is still open:
https://bugs.launchpad.net/ubuntu/+source/unity/+bug/960537

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Rüdiger Kupper (ruediger.kupper)]

It is most inconvenient that onboard does not auto-show when the dash is opened.

[mattgyver83 (mattgyver83)]

Confiriming auto-show issue with dash in Utopic with version 1.0.0-0ubuntu4. Settings confirmed to "Force on top" and "Auto-show when editing text" however opening the dash and editing never triggers the onboard to appear. Works everywhere else.

[mattgyver83 (mattgyver83)]

After a bit more testing I nailed down some more specifics of the issue I am noting including reproduction steps and I found a consistent method to "make it work" however it just continues to break. Hopefully helpful for identifying and resolving the issue.

Reproduction:
1. Start onboard with "Force on top" and "Auto-show while editing".
2. Open Dash.
3. Type in dash, onboard does not appear

Results:
Onboard opens but moving the mouse hides onboard it and never returns on search box selection.

Expected action:
Opening Dash and selecting search field should open onboard.

Potentialy helpfulfor troubeshooting to mimic "working" behavior sporadically:
1. Quit Onboard
2. Repeat reproduction steps 1-3
2a. Start onboard with "Force on top" and "Auto-show while editing".
2b. Open Dash.
2c. Type in dash, onboard does not appear
3. Open Gedit
4. Select inside gedit window, onboard appears
5. Click free area of Desktop, onboard hides.
6. Open Hud (left alt on physical keyboard), Onboard appears and is functional inside hud.
7. ESC hud
8. Click free area of desktop
8. Open Dash, onboard appears and is functional.

While this "works" its sporadic and the bad behavior returns after switching between other applications/normal use.

[tuxiano (tuxiano2)]

Hi,

Auto show still does not work with chrome and chromium on ubuntu 16.04 alpha.

Happy to see that it is working well with dash and gtk applications.

Regards

[Georgios Vasileiou (ge-vasiliou)]

Hello,
I can also confirm that with Uuntugnome 16.04 latest onboard version doesn't auto show in Chrome or chromium. With now days convertible '2 in 1' netbooks, luck of this feature becomes big problem.

[marmuta (marmuta)]

Georgios, there are hazy plans for input method support in the next release. That would possibly enable auto-show for a wider range of applications. I've only been experimenting with it, so far. If that eventually results in something useful I'll update this bug report.

[Georgios Vasileiou (ge-vasiliou)]

marmuta,
Thanks for your feedback.
For Chrome there are screen keyboard extensions that could do the job, but are not so nice as onboard.

Maybe builting a custom chrome extension that could possibly "call" onboard when required (address bar, text / input fields, etc) could do the trick and combine chrome with onboard instead of using third party extensions.

Is was also strange to me that pre-installed FireFox on Ubuntu supports onboard "out of the box".

Let us know when you have a new release of Onboard!