Onboard keyboard shows black box on login

[Asvin]

The onboard keyboard does not show properly on login. Only a black box is shown where the keyboard is supposed to appear, but the keys are not displayed. After login onboard works as expected.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.4.0-15-generic 4.4.0-15.31 [modified: boot/vmlinuz-4.4.0-15-generic]
ProcVersionSignature: Ubuntu 4.4.0-15.31-generic 4.4.6
Uname: Linux 4.4.0-15-generic x86_64
ApportVersion: 2.20-0ubuntu3
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: asvin 1814 F.... pulseaudio
 /dev/snd/controlC1: asvin 1814 F.... pulseaudio
CurrentDesktop: Unity
Date: Mon Mar 28 09:42:30 2016
HibernationDevice: RESUME=UUID=f716048f-ce0e-4825-b3c4-563fa743242a
InstallationDate: Installed on 2016-03-23 (4 days ago)
InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Beta amd64 (20160323)
MachineType: Dell Inc. Venue 11 Pro 7140
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.4.0-15-generic.efi.signed root=UUID=a9cb1c53-fcf1-4691-a246-24b90dd908c6 ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-15-generic N/A
 linux-backports-modules-4.4.0-15-generic N/A
 linux-firmware 1.157
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 10/27/2015
dmi.bios.vendor: Dell Inc.
dmi.bios.version: A09
dmi.board.name: 0XMVMH
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 8
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvrA09:bd10/27/2015:svnDellInc.:pnVenue11Pro7140:pvr:rvnDellInc.:rn0XMVMH:rvrA00:cvnDellInc.:ct8:cvr:
dmi.product.name: Venue 11 Pro 7140
dmi.sys.vendor: Dell Inc.

[Brad Figg (brad-figg)]

This change was made by a bot.

[Joseph Salisbury (jsalisbury)]

Did this issue start happening after an update/upgrade? Was there a prior kernel version where you were not having this particular problem?

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.5 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.5-wily/

[Asvin (m-launch7ad-u)]

The problem occurred upon a fresh install. I just tested kernel 4.5.0-040500-generic with no success.

[Asvin (m-launch7ad-u)]

Corrected package information. This is a bug with the onboard package.

[marmuta (marmuta)]

This could be Onboard crashing on something. If so, there should be an error message + traceback in /var/log/lightdm/.

Does this print anything interesting?
sudo su
zgrep -i onboard /var/log/lightdm/*

[marmuta (marmuta)]

I've tried two old 16.04 partitions and a freshly installed one so far, but wasn't able to reproduce the problem yet. Any additional information you can provide is highly appreciated.

Asvin, X, please post the file "/var/log/lightdm/x-0-greeter.log". There's a chance we will find an error traceback of Onboard in there.
X, if you don't mind, what type of tablet is that?

[Asvin (m-launch7ad-u)]

It is a Dell 7140 (Dell Venue 11 Pro).

I will post the log-files as soon as possible (but don't have the tablet with me this week).

[X (pvh-ubuntu)]

Here is an archive containing all the files in the log directory.

The only purpose of this specific computer is to help test Ubuntu 16.04 on the Microsoft Surface Pro 3 tablet. Nothing on it is private.

[X (pvh-ubuntu)]

The tablet is a "Microsoft Surface Pro 3, 8GB RAM, 256GB SSD".

A bit of additional info:

1) I'm a fairly experienced developer ...perfectly happy to run code to test things and report results to you

2) The OS install is expendable. ...so if you'd like me to repeatedly wipe and re-install the OS, modify the kernel, etc ...that's cool as well.

3) Ubuntu is the only OS on this machine

[marmuta (marmuta)]

Thank you both. There's no obvious error or traceback in the logs. I'd still like to see Asvin's x-0-greeter.log, but at the moment it doesn't look like Onboard crashes outright. Actually, there's little hint it is being started at all, but this seems to be the normal case.

There is an undocumented (read awkward) way to get a log file from Onboard, even when running in unity-greeter (which hard-codes Onboard's command-line). That would be good to have.

Make /usr/lib/python3/dist-packages/Onboard/Config.py starting from line 290 look like this:
         # optionally log to file; everything, including stack traces
         if 1:
             options.debug = "debug"
             logfile = open("/tmp/onboard.log", "w")
             #sys.stdout = logfile
             sys.stderr = logfile

Edit it by hand or use this one-liner:
sudo vi /usr/lib/python3/dist-packages/Onboard/Config.py +"%s/if 0/if 1/" +"%s/sys.stdout = logfile/#sys.stdout = logfile/" +wq

Then reboot, make sure the black rectangle is still visible at the login screen (i.e. the keyboard is still enabled), login and post the file "/tmp/onboard.log" here. Note that Onboard will probably fail to start in the user session now, until you delete /tmp/onboard.log.

When done, undo the changes with:
sudo apt-get install --reinstall onboard

[X (pvh-ubuntu)]

Here is the log file.

It includes multiple reboots. The first batch of reboots is *PRE* today's apt-get dist-upgrade, the second batch is *POST* today's apt-get dist-upgrade.

BEHAVIOR HAS CHANGED BETWEEN UPDATES.

The onboard keyboard briefly flashes for about 500ms before disappearing.

[X (pvh-ubuntu)]

Here is a screen capture of the new behavior. Onboard now displays the keyboard for ~500ms before turning black.

[marmuta (marmuta)]

Thanks for the log. It's recreated on each reboot, so I only see the one from the latest boot. The contents are slightly different from what I get here, but I don't see anything really suspicious. Everything seems to go as planned, and the fact that you can briefly see the keyboard now only confirms that. I still have no good idea what would cause it to go black after half a second.

Is the keyboard window moved/resized or hidden for some reason? The log doesn't show this yet, unfortunately, but you could try this:
- reboot, wait for login screen
- switch to a different virtual console, e.g. CTRL+ALT+F1 and login
- type
  $ xauth add :0 $(sudo xauth -f /var/run/lightdm/root/\:0 list | grep -o MIT.*)
  $ xwininfo -display :0 -name "Onboard" | tee onboard-xwininfo.txt

and post onboard-xwininfo.txt. It helps if you add these commands to ~/.bash_history beforehand, then you won't actually have to type them out.

Is there a video mode switch happening after the 500ms delay? I've noticed with my test partition, that no matter what video options I set in xorg.conf.d or lightdm's display-setup-script, unity-greeter always reverts back to the monitor's defaults. If that happens on your system there should be an entry at the end of your /var/log/Xorg.0.log. Please attach that file too. Make a copy before you login, e.g. from the same virtual console you just used above.

One more thing to try, if you haven't done so already:
- boot to login
- turn keyboard off (via indicator)
- reboot to login
- turn keyboard on
Does that change anything?

[Asvin (m-launch7ad-u)]

> One more thing to try, if you haven't done so already:
> - boot to login
> - turn keyboard off (via indicator)
> - reboot to login
> - turn keyboard on
> Does that change anything?

This worked for me, however, the keyboard only covered the upper left quarter of the black box.

[Asvin (m-launch7ad-u)]

Here are my logfiles

[marmuta (marmuta)]

> the keyboard only covered the upper left quarter of the black box
Yes, that was the hint I needed, thanks. It's an issue with the window scaling factor. I wasn't aware of unity-settings-daemon changing it. That should explain the 500ms delay too. Onboard probably starts up with scaling-factor 1 and is later switched to 2.

[marmuta (marmuta)]

The scaling issue seems to be a Gtk problem. I've opened a bug report here
https://bugs.launchpad.net/ubuntu/+source/gtk+3.0/+bug/1572331
A workaround for that in Onboard wouldn't be pretty.

I have yet to figure out why the keyboard becomes all black, and not just half the size.

[marmuta (marmuta)]

The keyboard seems to go below the screen, that's why we see a black rectangle there. On my 1920x1200 screen, scaled to 960x600, it apparently ends up at y=750.

See attachment for the configure events. Case 1) is the black rectangle case. The values are all over the place. GdkEventConfigure doesn't agree with get_pos()/get_size() and neither are fully correct. At the moment I don't see how I could compensate for that in Onboard. AFAIKT the problem is with Gtk, not Onboard.

It might be possible for unity-greeter to help, however. If the greater delays Onboard's launch until after unity-settings-daemon is done with setting the scaling factor, we get to Case 2) and see at least the half/quarter sized keyboard. If/when bug #1572331 is fixed it would hopefully go back to normal size.

As the worst case, the DPI dependent scaling in u-s-d would have to be turned off until Gtk is ready for it.

[marmuta (marmuta)]

Ah, messed up the numbering. Here are the configure events again.

[marmuta (marmuta)]

I've added a patch to the upstream bug report of bug #1572331 that seems to solve the scaling issue. Waiting for feedback.

I then tested delaying Onboard's launch in unity-greeter:
https://code.launchpad.net/~marmuta/unity-greeter/delay-keyboard-until-settings-daemon-done

That indeed works. With both of these changes Onboard is fully visible and can be used. No further workarounds in Onboard itself necessary.

IMO it's up to unity-greeter now. What I believe is needed is a better fix that makes sure u-s-d is done initializing before the greeter UI and keyboard are created. That would not only help the keyboard but also prevent flashing the login screen in scale 1, then a second later in scale 2.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Alberto Salvia Novella (es20490446e)]

@ Joseph Salisbury

Since onboard is the only input method available for Ubuntu on touch devices, I am raising this bug report importance to "high".

[marmuta (marmuta)]

@DH, you can find partial workarounds in comment
https://bugs.launchpad.net/ubuntu/+source/unity-greeter/+bug/1562734/comments/15
and question
https://answers.launchpad.net/onboard/+question/404618

That's all I know of currently, unfortunately. For anything better we need cooperation from unity-greeter and Gtk, neither of which has been forthcoming so far.