Must use hardware keyboard to perform search for applications in Unity

[Charlie Kravetz]

Binary package hint: unity

Unity as of 2011-03-21 does not allow an accessibile keyboard such as Onboard Keyboard or Dasher to be used with the Dash Search box.

To reproduce this issue:

1. Install Ubuntu Natty Narwhal
2. Open a terminal to start Onboard Keyboard.
3. click the Ubuntu Logo to open the search dialog
4. click back to the Onboard Keyboard application.
5. Observe that the search box disappears when the keyboard is clicked.

This same result can be reproduced using Dasher. Both are applications designed to allow the disabled user to type without a hardware keyboard. Unfortunately, they can not be used with Unity, since the search box requires the hardware keyboard to be used.

ProblemType: Bug
DistroRelease: Ubuntu 11.04
Package: unity 3.6.6-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.38-7.35-generic 2.6.38
Uname: Linux 2.6.38-7-generic i686
Architecture: i386
CompizPlugins: [core,bailer,detection,composite,opengl,decor,mousepoll,vpswitch,regex,animation,snap,expo,move,compiztoolbox,place,grid,imgpng,gnomecompat,wall,ezoom,workarounds,staticswitcher,resize,fade,unitymtgrabhandles,scale,session,unityshell]
Date: Mon Mar 21 17:04:05 2011
InstallationMedia: Ubuntu 11.04 "Natty Narwhal" - Alpha i386 (20110318)
ProcEnviron:
 LANGUAGE=en_US:en
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: unity
UpgradeStatus: No upgrade log present (probably fresh install)

[Mark Shuttleworth (sabdfl)]

Why is this critical, Brian?

[Nathan Haines (nhaines)]

Mark,

In my opinion, this is critical because this bug makes Unity (the premiere, default Ubuntu interface) non-accessible.

[Alan Bell (alanbell)]

This also makes search non-functional on an iPad type form factor device I think. Well you can still browse the big list and select things, but can't search for what you want. The policy is that a11y bugs basically get a bit of an importance boost, the lists are here
https://wiki.ubuntu.com/Bugs/Importance and actually, reviewing that, I think this is "A bug which impacts accessibility of a core application" which makes it High rather than "Critical: A bug which has a severe impact on a large portion of Ubuntu users" however, whatever level of importance the bug is, lets just get this one fixed.

[Mark Shuttleworth (sabdfl)]

We made very good progress on a11y in Natty, but will miss the goal of
perfect a11y. We'll nail it in Oneiric. That's OK, because we have the
Classic desktop fallback in Natty, but will not in Oneiric.

Mark

[Charlie Kravetz (charlie-tca)]

As the individual who marked this critical, I am changing the importance to "High", since this bug does not affect a large group, but rather a small group that can not use Unity because of this keyboard issue. This absolutely needs to be fixed, since those individuals who can not use a hardware keyboard are unable to use this important feature of Unity.

Mark: Thanks for bringing this to our attention. I can not change the upstream importance, which has the importance based on the Ubuntu task.

[Penelope Stowe (pendulum)]

This is also an issue for touch-screen devices. Is there a solution in the works for making it work with touch screens since I'd think that would be a market where otherwise Unity is a much better interface compared to classic GNOME.

[Grant Bowman (grantbow)]

The touch-screen device usability aspect of this is an april fool's joke on tablets that I ran across.

[Mark Shuttleworth (sabdfl)]

11.04 was designed for desktops, not tablets, there are a number of
profound changes in apps needed to get the libre desktop running nicely
on a tablet.

[cm-t arudy (cm-t)]

I add my +1 about touch devices:
I am using a tablet-pc.
In tablet mode I use apt://cellwriter because it allows me to use stylus and finger writing.
Cellwriter (in dock's cellwriter mode or not) is always under the dash (if it was in full screen) or dash lose focus and hides :
unity was designed to be the unified interface of ubuntu but it does not allow to use any search request in a no hardware keyboard device.
( else unity provide a better experience than an unmodified ubuntu's gnome2 )

@Alan Bell: please use «tablet» rather than «iPad»

[Jools Wills (jools)]

it also seems other accessibility stuff is broken. like "simulate second click"

https://bugs.launchpad.net/unity/+bug/762806

this makes unity non functional for any single click touchscreens, or for any users that may require this to help them use the desktop.

[nulled (elitescripts2000)]

11.04 and Unity I think are fucking awesome. SIMPLE, SLICK and just K.I.S.S. (Keep it simple stupid) Plus you may not be able to customize it much but that is the point. UNITY…

But, the proposal to remove the option to run UBUNTU CLASSIC is a TERRIBLE mistake.

Why?

Because on my shitty Dell celeron machine I can not run Unity! I do not have the graphics card for it. And to think Ubuntu will not support 5 year old computers is a FALLACY.

So I personally will be advocating for the KEEPING of the optional gnome option, because Linux runs great on slow machines too.

[Alan Bell (alanbell)]

Please, keep it civil and keep it related to the bug in hand. If you
have a patch to contribute or other information that adds value to the
bug then do comment, if you just want to say "me too" then click the me
too link at the top. If you just want to have a rant then please do so
on some other bit of the internet.
Thanks.

[Lynn (griffin-ld)]

As a tablet user, I find this to be a fairly critical bug. While, it's true one could click their way through the applications list, it's sometimes very difficult to click on the "See # more results" using a finger.

Thanks, let hope this gets fixed.

[Frederik Elwert (frederik-elwert)]

This is still present in Oneiric. Are there plans to fix this in this cycle?

The tablet vs. a11y discussion aside, having virtual keyboard support in Unity dash seems to be quite critical for a bunch of people.

[Grant Bowman (grantbow)]

dbarth or others, I believe this bug needs to be updated with an Oneiric importance higher than medium. There is no reasonable work around for tablet users. I do not see how to add an Oneiric status against https://bugs.launchpad.net/ubuntu/oneiric/+source/unity as was added for Natty.

[Alan Bell (alanbell)]

looks like on-screen keyboards can type into the unity dash search when using Ubuntu 2d, but still not when using Ubuntu 3d

[Alan Bell (alanbell)]

Jason Smith (DBO) may have an experimental patch for this I believe.

[Jean-Baptiste Lallement (jibel)]

Setting to 'High' "A bug which impacts accessibility of a core application"

[David Barth (dbarth)]

Sorry, that's out of scope for Oneiric.

We're still stabilizing a11y. OSK compatibility should be helped by the new IM support / ibus code, but we're not testing OSK support for Oneiric.

[Alan Bell (alanbell)]

So the tablet PC with touchscreen is not a supported form factor in Oneiric then?

[Neil J. Patel (njpatel)]

So according to Sam, our resident Xorg expert, the only way to fix this is to have Onboard set itself as a transient of the Dash window, allowing it to then correctly get stacked above the Dash (which, obviously, has the right hints set to stay above other apps).

I'm somewhat sure this will work as we already have Input Method support working (specifically IBus) which happily throws up a GtkWindow above the Dash and allows the user to click on it etc.

I also quickly converted Onboard's gtk-window to say it was type dock, and it stayed above the Dash and could be used to drive search just fine.

There are two ways we could do this, either I have Unity publicise the Dash's xid over D-Bus or as a xproperty on the root window, or Unity itself looks out for the Onboard window and makes it a transient when the Dash shows (and maybe unset's when it hides).

I'm prepared to have a go at the latter, if nothing else than for a test case, though it's obviously flawed in that it would be only useful for Onboard (which, this late in the cycle, might well be acceptable).

Thoughts?

[Charlie Kravetz (charlie-tca)]

My personal opinion here is that this is the second cycle without a usable onscreen keyboard in Unity. I think it should have at least Onboard working now. If accessibility is *really* important to Ubuntu, why is there no way to input without hardware? Do we really believe that a11y users are important? If so, then this should be a critical important item to make work, at least in a one app only mode for Oneiric. At the same time, the solution can not make Onboard unusable without Unity.

[Neil J. Patel (njpatel)]

@Charlie, we are trying to fix it for Oneiric hence I milestoned and assigned this bug to myself.

Sam, Jason and I discussed another possible method, which would be a patch to Onboard to make it correctly stack above the active application window and wouldn't add any soft-dep on Unity, nor would it require Unity to be changed. Going to try this first, if I can get my brain into Python mode ;)

[cm-t arudy (cm-t)]

Starting with Onboard is a good start.

But it must not reduce the choice of user to the software Onboard :
* first because each points of accessibility have specific needs (let's include stylus writing as accessibility)
* second because Ubuntu GNU/Linux is a *distribution*, means it provides packages for softwares, and you will understand that is useless if it can be installed but not useable.

Do you think unity, in the long term, will get the final patch, or we must start now open copies of this bug in all accessibility sofware having same issue than Onboard ?

Librement

[Jason Smith (jassmith)]

There is no *good* solution to this problem. Accessibility software will need some fashion to communicate that it is an accessibility window and thus should be above EVERYTHING even panel type windows. This is not currently something that there is a valid EHWM method of specifying. We will likely have to add a proposed expansion to the EHWM spec, implement it, and patch valid targets to match. We could also add a plugin to the window manager to detect windows that should be using the spec (by strict matching rules) and set the hint for them since X is cool like that.

The point is that this is not a "fix in unity" but a fix in the commincation protocol we use for windowing. This would be a problem regardless of X or Wayland too, so we just need to define a spec and fix it.

[marmuta (marmuta)]

@Neil, comment #24, could you elaborate what you discussed with Sam and Jason?

Francesco and me have been discussing this bug too and to us the window-type DOCK approach (or alternatively marking onboard "override-redirect") seems feasible. The main obstacle appears to be missing window decoration, but I have been working on making decoration optional anyway. There are "move", "hide" and "quit" buttons in an upstream branch already. What's missing is a replacement for window resizing.

There are more quirks though, loss of screen edge snapping and onboard can be positioned over the top panel. So if you have a better approach let us know please.

There is one other option. Onboard and other osks support the XEmbed extension. Gnome-screensaver and since recently lightdm use this to embed onboard (or other osks, cellwriter) into their windows. If unity supported that, onboard and others could already be made transients of dash.
That is more of a long term vision though and unity would probably have to be modified with on-screen keyboards in mind, e.g. having a "keyboard dock" at the botton of the screen or something.

[Launchpad Janitor (janitor)]

This bug was fixed in the package unity - 4.20.0-0ubuntu1

---------------
unity (4.20.0-0ubuntu1) oneiric; urgency=low

  * New upstream release + cherry-pick:
    - compiz crashed with SIGSEGV in PluginClassHandler<UnityWindow,
      CompWindow, 0>::get() (LP: #835259)
    - dash does not scroll down during keyboard navigation (LP: #852506)
    - compiz crashed with SIGSEGV in _XFreeEventCookies() (LP: #851472)
    - compiz crashed with SIGSEGV in g_object_unref() (LP: #847512)
    - compiz crashed with SIGSEGV in nux::WindowCompositor::MouseEventCycle()
      (LP: #831678)
    - compiz and X can disagree on the stacking order (LP: #845719)
    - compiz crashed with SIGSEGV in unity::dash::LensView::OnResultAdded()
      (LP: #855252)
    - CCSM crashes Unity (LP: #861643)
    - compiz crashed with SIGSEGV in PrivateWindow::addWindowStackChanges()
      (LP: #862115)
    - Touch - When dragging a window with a 3 touch, dragging to the top
      should show the maximize gesture preview. (LP: #750316)
    - compiz crashed with SIGABRT in raise() (LP: #750386)
    - Launcher - Spread should not affect the state of window (LP: #764673)
    - [UIFe] Add system setting icon to Launcher (LP: #764744)
    - compiz crashed with SIGABRT in raise() [Merely unchecking the Unity
      plugin is enough to crash Compiz...] (LP: #823386)
    - Launcher - does not hide if revealed from top 24px of the left edge of
      the screen (LP: #835989)
    - Clickable areas of previously active window remains on 'Show Desktop'
      (LP: #836325)
    - Cannot search for some apps in Dash (LP: #837075)
    - Dash - Search box cursor not responding correctly to focus (LP: #839704)
    - Multi-display: Application menu only showing on primary screen
      (LP: #845212)
    - Ellipsized values in size filter (LP: #851359)
    - (oneiric) dash result expansions don't display icons (LP: #854615)
    - run the wrong command if enter is hit before the view is refreshed
      (LP: #856205)
    - Java application windows cut-off/truncated/not displayed properly
      (LP: #857201)
    - compiz crashed with SIGSEGV in sigc::signal_base::impl() (LP: #831769)
    - Minimizing a window should switch focus to the windows underneath it
      (breaks restore) (LP: #859885)
    - Ratings filter allows ratings > 1.0 (LP: #861255)
    - Serious Unity problem stuck in spread when dragging icons back to the
      dash (LP: #862045)
    - Must use hardware keyboard to perform search for applications in Unity
      (LP: #739812)
    - Files lens should also search Downloads (LP: #748915)
    - The dash crosses the monitor boundary when resolution is 1152x864 or
      lesser width (LP: #765338)
    - Dash - dragging a file outside of the Desktop Dash should close the Dash
      (LP: #764641)
    - Text in Dash search text box overflows and doesn't crop with longer
      strings; RtL input starts in wrong place (LP: #834765)
    - Where the average background colour is either very light or dark, adjust
      this value to darken/lighten in order to preserve readability
      (LP: #837993)
    - Dash does not resize when I connect to an external display and mirror
      displays (LP: #83813...

[Bilal Akhtar (bilalakhtar)]

This bug seems to have re-appeared in Onboard 0.96.x, reported as bug #915250 .

[David López (david-lopez-upct)]

I'm affected with this bug (I posted bug #915250) and I'll explain why I think this bug is critical. Sorry for my English mistakes, I'm Spanish. I bought a tablet with ubuntu 11.10 preinstalled. The company which sells the tablet made his own oneiric iso image with some special additions for tablets. The default desktop in the iso is gnome-shell, and unity-2d was also preinstalled. unity doesn't.

I installed unity by myself, and I found it perfectly usable in the tablet (I made a lot of tweaks, for example I remove overlay-scrollbar, but unity worked fine, IMHO much better than gnome-shell unity-2d). The only problem I found in unity is this bug of onboard and dash.

I don't know anybody in the company and I don't know why they removed unity and include unity-2d and gnome-shell, but I wasn't surprised that it was for this bug. If so, if a bug makes a hardware vendor remove unity and include gnome-shell, I can imagine a bug more critical than this one.

[David López (david-lopez-upct)]

Sorry, I mean 'I can NOT imagine a bug more critical than this one'

[Bitterjug (bitterjug)]

According to post #28 this is fixed in Oneiric, but from reading the thread I can't tell what the fix is.
I've been trying to use my active digitizer laptop with Oneiric Unity 3d and Cellwriter but there is no obvious way to enter text into the dash with cellwriter. Please explain what the fix was, thanks.

[Francesco Fumanti (frafu)]

The fix only worked for a determined version of Onboard. You can enable the "Force to top" available in newer versions of Onboard to work around the problem until the issue is fixed again in dash.

Here is the new bug about the problem:
https://bugs.launchpad.net/ubuntu/+source/unity/+bug/915250
You might mark it as affecting you too.

[Alan Bell (alanbell)]

here are the lines of unity that you might want to go peek at to see what happened

ChangeLog:8858: make onboard stack above the dash like a boss
plugins/unityshell/src/unityshell.cpp:2611: /* stack any windows named "onboard" above us */
plugins/unityshell/src/unityshell.cpp:2614: if (w->resName() == "onboard")
plugins/unityshell/src/unityshell.cpp:2771: if (window->resName() == "onboard")

[Francesco Fumanti (frafu)]

@Alan Bell

So, our assumption that it does not work anymore because we changed the capitalisation to "Onboard" seems to have been correct. (We changed it to have a unique spelling for Onboard through all Onboard files; previously, there were several spellings in use.)

Would it not be possible to change the code in dash, so that the capitalisation of Onboard becomes irrelevant?

[Alan Bell (alanbell)]

yes, but this is swapping one ugly hack for another. Is there a nice way to identify an application as something that really should be allowed to overlay anything?

[Francesco Fumanti (frafu)]

@Alan

When looking for a clean solution, I wonder whether it would also make sense to simultaneously consider the current similar bug about not being able to enter the password into unity greeter.
https://bugs.launchpad.net/unity-greeter/+bug/878184

Unfortunately, I don't have the knowledge to answer this question. I hope people with the necessary knowledge are following this thread.

But should this discussion not happen in the bug thread that I cited in reply 33 of this thread? Or would it make sense to reopen this bug and mark the new bug as a duplicate of this bug?

[marmuta (marmuta)]

Alan, how about we add a custom X property to onboard's windows? Something like ON_SCREEN_KEYBOARD=1. Unity could check this instead and at least get rid of the hard coded "Onboard".

[Alan Bell (alanbell)]

that is really a question for the unity people not me, sounds like a good idea though.