New On-screen keyboard does not appear automatically when selecting some text fields

Bug #1760399 reported by Amr Ibrahim on 2018-04-01
152
This bug affects 25 people
Affects Status Importance Assigned to Milestone
OEM Priority Project
Undecided
Unassigned
gnome-settings-daemon (Ubuntu)
Undecided
Marco Trevisan (Treviño)
Bionic
Undecided
Marco Trevisan (Treviño)
Cosmic
Undecided
Marco Trevisan (Treviño)
gnome-shell (Ubuntu)
Undecided
Marco Trevisan (Treviño)
Bionic
Undecided
Marco Trevisan (Treviño)
Cosmic
Undecided
Marco Trevisan (Treviño)

Bug Description

[Impact]
on-screen keyboard cannot be reliably opened when running under xorg.

[Test Case]
1. Login into an xorg session
1. Open an application with text-entries (e.g. Firefox)
2. Try to enter text in a text entry using the new OSK

Expected results:
The on-screen keyboard should appear.

[Regression Potential]
The proposed fix requires a change in the way gnome-settings-daemon decides to enable or disable ibus. Please make sure that ibus correctly works when
using a keyboard layout that requires it (e.g. chinese).

[Original Report]
I have always used an on-screen keyboard as a virtual keyboard for entering text in a different layout than my physical keyboard. This time I cannot, since I cannot manually trigger the new OSK.

Steps:

1. Open Firefox
2. Try to enter text in a different layout using the new OSK
3. You cannot make the OSK appear

When enabled, the new OSK appears automatically in Activities, but I cannot trigger it outside GNOME apps. You have to enable the new OSK first from Universal Access in Settings.

I consider this a regression from 16.04 LTS, where On-board is easily triggered.

Workaround:

sudo apt install onboard

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: gnome-shell 3.28.0-0ubuntu2
ProcVersionSignature: Ubuntu 4.15.0-13.14-generic 4.15.10
Uname: Linux 4.15.0-13-generic x86_64
ApportVersion: 2.20.9-0ubuntu2
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Sun Apr 1 11:31:16 2018
DisplayManager: gdm3
InstallationDate: Installed on 2017-10-02 (180 days ago)
InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Beta amd64 (20170930)
SourcePackage: gnome-shell
UpgradeStatus: Upgraded to bionic on 2018-03-27 (4 days ago)

Related branches

Amr Ibrahim (amribrahim1987) wrote :
tags: added: osk
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in caribou (Ubuntu):
status: New → Confirmed
Changed in gnome-shell (Ubuntu):
status: New → Confirmed
François Tissandier (baloo) wrote :

Same problem. I can't use my Dell XPS 13 in tablet mode at all, since the keyboard isn't appearing on most apps, and I don't know how to trigger its appearance :-(

Workaround:

sudo apt install onboard

There is an extension to show a panel button to trigger it:
https://extensions.gnome.org/extension/1061/on-screen-keyboard-button/

Unfortunately, its panel button itself does not react to touch at least for me (on Cosmic, Gnome 3.30, Wayland), which kind of defeats the purpose

That said, in Gnome 3.30 the new OSK (from 2.28) automatically appears for me when I touch certain text entry fields, even when it is not enabled in Accessibility settings. This is IMHO the best solution. However, detection is either buggy or needs application support, as it works for me only in gnome-shell directly, e.g. the search field in Activities, but not anywhere else, not even first-class Gnome apps like Web or Videos

tags: added: cosmic
Mario Vukelic (mario-vukelic) wrote :

I notice that the release notes for Gnome 2.28 say "The new keyboard automatically activates when a text area is selected". If this worked, no activation button would be needed as far as I can tell, and I suppose this is why there is none. However as I said in my previous comment this is not the case for me

Mario Vukelic (mario-vukelic) wrote :

The Ubuntu help for 18.04 says that you need to activate it in the Accessibility settings, and "When you next have the opportunity to type, the on-screen keyboard will open at the bottom of the screen."
https://help.ubuntu.com/stable/ubuntu-help/keyboard-osk.html

Neither is true for me:
1. I don't need to enable it in Accessibility in order to trigger in Activities and Applications
2. If I do enable it, it doesn't trigger outside of these two menus anyway

tags: added: touch universalaccess
Mario Vukelic (mario-vukelic) wrote :

So after much hair pulling I contacted the author of the above mentioned Gnome extension and he set me on the right track (thank you):

1. Auto-summon of OSK only works with application/toolkit support. Text widgets in alien (non-Gnome or non-Gtk?) toolkits don't summon the OSK (Gnome, Firefox, Qt, whatever other toolkits; I don't know if this is even standardized by freedesktop.org.

2. Because of alien toolkits there is always a need to summon on demand. Since Gnome 3.28 in Ubuntu 18.04, you can summon on a touch screen by swiping up from the bottom of the screen. However, this ONLY works in a Wayland session, not in X. I can confirm that it does. None of this is described in the Gnome 3.28 release notes or the Ubuntu 18.04 help. These are clearly documentation bugs.

3. Judging by the confusion int his bug report and the many questions on Askubuntu,com, it seems that the swiping up eluded even those users who have access to the feature because they use Wayland. Quite clearly this is not sufficiently discoverable, which IMHO is a bug. (Of course this is not helped by the buggy docs)

4. The Gnome extension to summon on-demand (https://extensions.gnome.org/extension/1061/on-screen-keyboard-button/) does NOT work with touch in Ubuntu 18.04 (Gnome 3.28) according to the author, and neither does it in Cosmic (3.30). It works with the mouse but this is awkward to say the least. The author will hopefully be able to look into it but it may take some time.

5. I don't know why in Gnome 3.30 (Cosmic) the auto-summon only works for Activities and Applications menus search boxes, and not elsewhere in Gnome as far as I can find. Hard to believe that this is a bug in every Gnome application, maybe a Gtk bug?

Mario Vukelic (mario-vukelic) wrote :

Duh. In the above comment, it should have been:
"1. (**Chrome** [not Gnome], Firefox, Qt, whatever other toolkits...)

tags: removed: universalaccess
Mario Vukelic (mario-vukelic) wrote :

Sorry my earlier removal of the tag was unintentional, readding

tags: added: universalaccess
Will Cooke (willcooke) on 2018-10-15
tags: added: 2in1
Changed in oem-priority:
status: New → Confirmed
Carolynn (carobell) wrote :

Is there a manual way to have it come-up ? From comment #10 :
"Because of alien toolkits there is always a need to summon on demand."
Ido not have touch screen Ineed the keyboard because my space key does not work asitshould.
Still a very annoying bug.

summary: - New On-screen keyboard cannot be manually triggered to appear
+ New On-screen keyboard does not appear automatically when selecting some
+ text fields
Andrea Azzarone (azzar1) on 2019-01-22
no longer affects: caribou (Ubuntu)
Andrea Azzarone (azzar1) on 2019-01-22
description: updated
Andrea Azzarone (azzar1) on 2019-01-23
description: updated
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in gnome-settings-daemon (Ubuntu):
status: New → Confirmed
Changed in gnome-settings-daemon (Ubuntu):
status: Confirmed → Fix Released
Changed in gnome-shell (Ubuntu):
status: Confirmed → Fix Released
Iain Lane (laney) on 2019-01-25
Changed in gnome-settings-daemon (Ubuntu):
status: Fix Released → In Progress
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)
Changed in gnome-settings-daemon (Ubuntu Bionic):
status: New → In Progress
Changed in gnome-settings-daemon (Ubuntu Cosmic):
status: New → In Progress
Changed in gnome-shell (Ubuntu Bionic):
status: New → In Progress
Changed in gnome-shell (Ubuntu Cosmic):
status: New → In Progress
Changed in gnome-settings-daemon (Ubuntu Bionic):
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)
Changed in gnome-settings-daemon (Ubuntu Cosmic):
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)
Changed in gnome-shell (Ubuntu Bionic):
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)
Changed in gnome-shell (Ubuntu):
status: Fix Released → In Progress
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)
Changed in gnome-shell (Ubuntu Cosmic):
assignee: nobody → Marco Trevisan (Treviño) (3v1n0)

Gnome shell side of things has been fixed in 3.30.1

Changed in gnome-shell (Ubuntu):
status: In Progress → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package gnome-settings-daemon - 3.30.2-2ubuntu1

---------------
gnome-settings-daemon (3.30.2-2ubuntu1) disco; urgency=medium

  * Merge with Debian. Remaining changes:
    + debian/control.in:
      - Build-depend on accountsservice
    + debian/patches:
       - 45_suppress-printer-may-not-be-connected-notification.patch
       - 53_sync_input_sources_to_accountsservice.patch
       - 64_restore_terminal_keyboard_shortcut_schema.patch
       - correct_logout_action.patch
          display the logout action on ctrl-alt-del
       - ubuntu-lid-close-suspend.patch
          Reimplement support for setting lid close suspend actions
       - revert-wacom-migration.patch
       - revert-gsettings-removals.patch
          Revert gsettings keys that were dropped since 3.8, for u-s-d
       - revert-mediakeys-dbus-interface-drop.patch
       - ubuntu_ibus_configs.patch
       - git_binary_encoding.patch
       - ubuntu_calculator_snap.patch
    + debian/gnome-settings-daemon.install:
      - Install apport hook
  * Use ubuntu/disco branch, master has 3.31 prepared and I don't want to
    upload it yet

gnome-settings-daemon (3.30.2-2) unstable; urgency=medium

  * keyboard-Enable-ibus-for-OSK-purposes.patch: Cherry-pick from master.
    To fix enabling of the OSK in more circumstances. (LP: #1760399)
  * gbp.conf: Use debian/unstable branch now we've branched for exp

 -- Iain Lane <email address hidden> Fri, 25 Jan 2019 20:20:45 +0000

Changed in gnome-settings-daemon (Ubuntu):
status: In Progress → Fix Released
Łukasz Zemczak (sil2100) wrote :

Does this require some changes in gnome-shell as well? Since I only see a fix for this issue in gnome-settings-daemon for cosmic - are we still missing a gnome-shell upload for this bug there?

Both disco and cosmic side of things should have been fixed in disco 3.30.1-2ubuntu2 and cosmic 3.30.2-0ubuntu1.18.10.1

Iain Lane (laney) wrote :

Right, I think we just didn't include the bug # there since it comes as part of the upstream release.

Changed in gnome-shell (Ubuntu Cosmic):
status: In Progress → Fix Committed

Hello Amr, or anyone else affected,

Accepted gnome-settings-daemon into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gnome-settings-daemon/3.30.1.2-1ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in gnome-settings-daemon (Ubuntu Cosmic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-cosmic
Brian Murray (brian-murray) wrote :

Hello Amr, or anyone else affected,

Accepted gnome-settings-daemon into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/gnome-settings-daemon/3.28.1-0ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in gnome-settings-daemon (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Mario Vukelic (mario-vukelic) wrote :

I can confirm on Disco that this is definitely better, but not completely solved for me:

gnome-settings-daemon 3.30.2-2ubuntu1
Screen Keyboard is off in Universal Access settings

* Good in both X and Wayland:

1. Touch Activities on panel
2. Touch "type to search" box
-> OSK comes up.
3. Close OSK with ArrowDown icon
4. Touch "type to search" box again
-> OSK comes up again.
(Same good behavior for:
Applications on dock -> "Type to search"
GDM login screen -> password field)

* Good only in X, not good in Wayland:

1. Two applications side by side e.g. gnome-term and gedit
2. Switch windows by touching
-> In X, OSK comes up on focus change, but not in Wayland . So in X it is good so far but ...
3. Close OSK by ArrowDown icon
4. Touch _same_ window again (and hence no focus switch)
-> no OSK even in X

1. Start on an empty workspace: Firefox, nautilus, epiphany, and others:
2. Touch text entry fields
-> OSK comes up in X but not in Wayland

* Bad in both X and Wayland:

1. Go to empty workspace
2. Launch gnome-term, LibreOffice Writer or Calc, others:
-> no OSK. However in X (but not Wayland) ...
3. Switch focus by touching any menu on Gnome panel
4. Touch gnome-term again
-> Now OSK comes up

* Bad in X when OSK does not come up automatically:

When OSK does not appear automatically, there is still no way to summon it on demand.
(While in Wayland swiping up still works as workaround but remains difficult to discover)

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers