[Focal regression] On-screen keyboard (OSK) does not appear on touch under Xorg on convertible laptops in tablet mode when physical keyboard disabled

[Mario Vukelic]

[ Impact ]

OSK doesn't appear in X11 when disabling physical keyboard

[ Test case ]

1. Log into gnome-shell Xorg session ("Ubuntu")
2. Fold the display back, to switch to tablet mode. (On properly supported devices, this
   disables the physical keyboard)
3. Use touch screen to touch a (supported) text input field, such as
   gnome-shell > Activities > Search:
   - The OSK appear
4. Swipe from bottom of screen in order to summon OSK on demand:
   - occurred on Ubuntu 18.04 through 19.10)

[ Regression potential ]

The OSK shows when not needed in setup with no touch-screen or with touch screen but when not required.

---

Opening new bug as instructed by Sebastien Bacher (seb128) in bug #1866556.
Related upstream bugs:
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/2378
https://gitlab.gnome.org/GNOME/mutter/-/issues/1242

[Note: Please someone add a bug watch for https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/2378. I set one for the other and then Launchpad somehow did not let me again]

Reproduction with Ubuntu 20.04 on affected convertible devices like Dell XPS 9575 2-in-1:
1. Log into gnome-shell Xorg session ("Ubuntu")
2. Fold the display back, to switch to tablet mode. (On properly supported devices, this disables the physical keyboard)
3. Use touch screen to touch a (supported) text input field, such as gnome-shell > Activities > Search
-> EXPECTED RESULT (occurred on Ubuntu 18.04 through 19.10): The OSK appears
-> ACTUAL RESULT: nothing happens, you cannot enter text
4. Swipe from bottom of screen in order to summon OSK on demand:
-> EXPECTED RESULT (occurred on Ubuntu 18.04 through 19.10): The OSK appears
-> ACTUAL RESULT: nothing happens, you cannot enter text (e.g. into text fields of foreign toolkits that are expected not to trigger the OSK automatically)

Note that the EXPECTED RESULT still happens under Wayland in 20.04. Only the Xorg session is affected by the issue described in this bug.

Also note that Accessibility was NOT necessary to be enabled in 19.10 and before. (When enabling it in 20.04, the OSK does appear under Xorg, but this should not be necessary)

(On Wayland there is a different changed behavior with the OSK not appearing in the device's laptop mode, i.e. with the physical keyboard enabled. This seems intended and a different issue as per https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/872 and
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/2353 )

This bug report was created on Dell XPS 9575 2-in-1, where the bug reproduces.
I can confirm that the EXPECTED behavior occurred under Xorg in 19.10 on this machine as well as on the Asus Zenbook Flip UX561UD, and stopped with 20.04.
It probably also worked with 19.04 and possibly 18.10, but I ran mostly Wayland and so am not entirely sure. (I never ran 18.04 on these devices)

Other hardware reported to reproduce the issue (and in part are said to have been working in 19.10 and before, down to 18.04) include:

In bug #1866556;
Lenovo ThinkPad L390 Yoga
Asus ZenBook UX370UAF
Acer Switch 11 V sw5-173 (said to have worked with 18.04)
(Possibly HP ENVY x360 15-cp0 and Lenovo ThinkPad X201 Tablet, though the tablet mode seems unsupported on these devices and so the issue might be different)

In upstream bug:
Dell Inspiron 15 700 2-in-1

ProblemType: BugDistroRelease: Ubuntu 20.04
Package: gnome-shell 3.36.2-1ubuntu1~20.04.1
ProcVersionSignature: Ubuntu 5.4.0-31.35-generic 5.4.34
Uname: Linux 5.4.0-31-generic x86_64
ApportVersion: 2.20.11-0ubuntu27
Architecture: amd64
CasperMD5CheckResult: skip
CurrentDesktop: ubuntu:GNOME
Date: Mon May 25 19:59:40 2020
DisplayManager: gdm3
InstallationDate: Installed on 2018-09-13 (619 days ago)
InstallationMedia: Ubuntu 18.10 "Cosmic Cuttlefish" - Alpha amd64 (20180912)
RelatedPackageVersions: mutter-common 3.36.2-1ubuntu1~20.04.1SourcePackage: gnome-shell
UpgradeStatus: Upgraded to focal on 2020-01-04 (141 days ago)

[Mario Vukelic (kreuzsakra)]

sudo libinput list-devices > list-devices.txt:
was generated in laptop mode i.e. with the physical keyboard enabled.

[Mario Vukelic (kreuzsakra)]

Will be fixed in 3.36.3:
https://gitlab.gnome.org/GNOME/mutter/-/issues/1242#note_823253

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Launchpad Janitor (janitor)]

This bug was fixed in the package mutter - 3.36.3-1ubuntu1

---------------
mutter (3.36.3-1ubuntu1) groovy; urgency=medium

  * Merge with debian, including new stable upstream release
  * debian/control: BD on adwaita-icon-theme 3.36.1-2ubuntu3.
    This version includes all the window decoration icons that mutter uses
  * Remaining changes with debian:
    - debian/control:
      + Update VCS flags to point to ubuntu salsa branch
    - debian/gbp.conf: update branch to point to ubuntu/master
    - debian/patches/x11-Add-support-for-fractional-scaling-using-Randr.patch:
      + X11: Add support for fractional scaling using Randr

mutter (3.36.3-1) unstable; urgency=medium

  * New upstream release (LP: #1881971)
   - Broadcast clipboard/primary offers
   - Fix monitor screen cast on X11
   - Implement touch-mode detecation for the X11 backend (LP: #1880596)
   - Drop external keyboard detection from touch-mode heuristics
   - Fix leaked DMA buffers in screencasts
   - Fixed crashes

mutter (3.36.2+12+gb425f1153-1) experimental; urgency=medium

  * Team upload
  * New upstream snapshot from the gnome-3-36 branch
  * d/p/backend-x11-Reintroduce-XInitThreads.patch:
    Drop patch, applied upstream

 -- Marco Trevisan (Treviño) <email address hidden> Thu, 04 Jun 2020 21:46:56 +0200

[Timo Aaltonen (tjaalton)]

Hello Mario, or anyone else affected,

Accepted mutter into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mutter/3.36.3-0ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Mario Vukelic (kreuzsakra)]

Installed mutter 3.36.3-0ubuntu0.20.04.1 from -proposed.
This fixes the bug for me on the Dell XPS 9575 (could not test on the Asus Zenbook)

Tested:
1. Logged into Xorg session
2. In laptop mode (physical keyboard enabled):
- Touch Activities > Search: Success, OSK appears
- Touch Show Applications > Search: Success, OSK appears
- Open gnome-term > touch input area: Success, OSK appears
- Open gedit > touch test areas: Success, OSK appears
- Open Firefox > touch search box: Success, OSK appears
- Open GnuCash (gtk application) > touch: OSK appears, but is flakey. This is as I remember it always to be
- Swiped up from bottom of screen to summon OSK as needed: Success, OSK appears
- In all test cases: touched DownArrow button on OSK: Success, OSK disappears
3. Folded display to tablet mode (physical keyboard disabled):
- Same tests, same behavior
4. Log out and into Wayland session:
- Same tests and successes in tablet mode. In laptop mode, the OSK never appears as expected due to the design decision mentioned above*

* Still rather infuriating that you can touch a textfield but are not allowed to enter anything

[Mario Vukelic (kreuzsakra)]

And thank you!

[Mario Vukelic (kreuzsakra)]

Maybe I should add that I qualify the OSK appearing in laptop mode under X as "Success" because:
-This is how it behaved until 19.10 both under X and Wayland
-In the upstream bug it was mentioned that the laptop vs tablet mode detection would not work under X and so the new behavior under Wayland (no OSK in laptop mode) could not (yet) be duplicated

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for mutter has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package mutter - 3.36.3-0ubuntu0.20.04.1

---------------
mutter (3.36.3-0ubuntu0.20.04.1) focal; urgency=medium

  * New upstream release (LP: #1881971)
    - Broadcast clipboard/primary offers
    - Fix monitor screen cast on X11
    - Implement touch-mode detecation for the X11 backend (LP: #1880596)
    - Drop external keyboard detection from touch-mode heuristics
    - Fix leaked DMA buffers in screencasts
    - Fixed crashes
  * Refreshed patches
  * d/p/backend-x11-Reintroduce-XInitThreads.patch:
    - Dropped (merged upstream)

mutter (3.36.2-1ubuntu1~20.04.2) focal; urgency=medium

  [ Daniel van Vugt ]
  * d/p/backend-x11-Reintroduce-XInitThreads.patch:
    Fix a very common crash when running in X11 (LP: #1877075)

 -- Marco Trevisan (Treviño) <email address hidden> Fri, 05 Jun 2020 19:29:32 +0200

[albertvaka (albertvaka)]

I have this exact same problem (OSK doesn't pop up on X11 nor XWayland apps on Wayland) and I'm on 3.36.6. Has the fix been undone?

[albertvaka (albertvaka)]

I have to say that it works on a freshly installed Ubuntu 20.04, then breaks after I do apt-get upgrade.

[Mario Vukelic (kreuzsakra)]

@albertvaka

I realize it stopped working for me as well, 20.04 with mutter 3.36.6-1ubuntu0.20.04.2
Still same machine as in above posts, Dell XPS 9575 2-in-1

No idea if same problem reappeared or it's an entirely new one

[albertvaka (albertvaka)]

Given the comment above "In the event that you encounter a regression using the package from -updates please report a new bug" I've open a new bug so the right people get pinged:

https://bugs.launchpad.net/ubuntu/+source/mutter/+bug/1904237

[Andrés (forestrf)]

I'm unable to install Ubuntu because my ps2 keyboard doesn't work on the live cd, and the accesibility keyboard doesn't work on the installation app, so while I was able to copy and paste text from a text document to the text fields, because I can't copy/paste anything to a password field and the field itself doesn't open the on screen keyboard, I can't start the installation.

Given that the on screen keyboard is an accessibility feature, I don't know why it goes away on it's own without an option to keep it open. A keyboard is not only useful on a text field, also with shortcuts and more.

[Daniel van Vugt (vanvugt)]

Remember this bug is closed. All new comments should go in open bugs like bug 1904237.