unity 3d touch stops registering clicks

[mweyen]

SRU Criteria
============
[Impact]
Touchscreen input handling is broken in many ways in the Precise X server. The issues include crashes, events sent to the wrong window, broken grabs, memory leaks, and Compiz and toolkit pop-up menu breakage.

[Upstream Fixes]
http://cgit.freedesktop.org/~cndougla/xserver/log/?h=input-fixes

All commits were developed specifically for this bug. I just kept unraveling issues until things worked properly on my touchscreen. A high level overview of the impact of the fixes:

* Crash or events sent to the wrong windows due to inappropriate casting and memory accesses:
  - Check core event mask properly for pointer emulated touch events
  - Only set XI2 mask if pointer emulation is for XI2 client
  - Check other clients' core masks properly when adding touch listener
* Non-functional changes required for subsequent fixes:
  - Split out helper function TouchListenerAcceptReject()
  - Rename TouchEnsureSprite to TouchBuildSprite and event type checks
* Fixes for internal touch and pointer records remaining in an open state, leading to memory leaks and potentially input locking up:
  - Accept touch sequence for pointer listener after second event delivery
  - When deactivating an explicit pointer grab, reject all grabs on touches
  - Ensure touch is ended when last listener is rejected
* Fixes for event and server time issues, which lead to grab and ungrab requests being rejected or unhandled:
  - Fix copy/paste error from before git history in UpdateCurrentTimeIf()
  - Update currentTime in dispatch loop
  - Replay original touch begin event instead of generated begin event
* Don't send end event to accepting client if touch hasn't physically ended:
  - On touch accept, only process end event for owner if it has seen the end
* Don't prematurely end touches and deactivate implicit grabs if a touch end event is emitted in response to something other than the touch physically ending:
  - End a pointer emulated touch event only on a "real" end event
  - Don't deactivate implicit pointer grab on fake touch end event
* Fix toolkit pop-up menus so when you touch the menu, it opens, and then you drag to select an item before lifting the menu will receive the events:
  - When activating an explicit grab, update owning listener
* Fix dragging a Compiz window by touching the title bar using a touchscreen:
  - Create a new dix touch record for an emulated touch with no listeners
  - Update device state including when touch record does not exist

[Development Fix]
The Q series is not open yet, so there is no development fix.

[Stable Fix]
The upstream fixes have been squashed and rebased into one patch for the Precise xorg-server. The patch has been attached.

The touch code has been completely rewritten from the prototype implementation in previous Ubuntu releases, so these changes have not been nominated for them.

[Test Cases]
* Attempt to interact with a GTK menu on a touchscreen by touching to open the menu and dragging to select items without lifting the touch in between. The items should highlight, and the selected item should activate.
* Attempt to move a window in Unity 3D by dragging the title bar with a touchscreen touch.
* Interact with various windows in Unity to raise them. This exercises grabbing code paths. Ensure all input devices continue to function properly, no crashes result, and events appear to go to the appropriate windows.

[Regression Potential]
Many patches only affect touch pointer emulation, which only affects touchscreens. The code paths modified should have no regression potential for other usages. Some of the other patches affect touch handling in general. These could impact gestures for trackpads, but traditional trackpad pointing and clicking does not go through these code paths. The server time fixes that are not touch-specific are small and merely keep the server time more up to date than it currently is. There could be a regression potential here for input device grabbing, but it is unlikely.

Original Bug Report
===================
Hi,

similar to #949791 ...

that bug made things better (i can now move windows using the touch screen) but it still often stops to react to any clicks when using touch.

I just updated.

In unity 3d (not in unity 2d nor in gnome shell) if i touch the top bar without any program running it most of the time stops to react to any clicks (both touch and clicks of the touch pad buttons). I cannot reproduce it always with the first touch but usually latest with 2 or 3.

i am not sure what happens but after something around a minute it starts to work again ..... i tried the super key to open the dash and then it suddenly worked again. I'm sorry but i can't say exactly what happens.

this happened like 2 or 3 times in this session, so maybe you see something in the logs ubuntu-bug attaches.

If you need more info i'll do my best to provide it.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: unity 5.8.0-0ubuntu2
ProcVersionSignature: Ubuntu 3.2.0-22.35-generic 3.2.14
Uname: Linux 3.2.0-22-generic x86_64
.tmp.unity.support.test.0:

ApportVersion: 2.0-0ubuntu4
Architecture: amd64
CompizPlugins: [core,composite,opengl,compiztoolbox,decor,vpswitch,snap,mousepoll,resize,place,move,wall,grid,regex,imgpng,session,gnomecompat,animation,fade,unitymtgrabhandles,workarounds,scale,expo,ezoom,unityshell]
CompositorRunning: compiz
Date: Fri Apr 6 08:53:14 2012
DistUpgraded: 2012-03-11 11:16:56,683 DEBUG enabling apt cron job
DistroCodename: precise
DistroVariant: ubuntu
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64 (20111012)
MachineType: Acer Aspire 1825PTZ
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.2.0-22-generic root=UUID=3edf82f9-f9ff-489c-8b24-2ffb824a9cd1 ro quiet splash vt.handoff=7
SourcePackage: unity
UpgradeStatus: Upgraded to precise on 2012-03-11 (25 days ago)
dmi.bios.date: 04/23/2010
dmi.bios.vendor: INSYDE
dmi.bios.version: V1.3127
dmi.board.asset.tag: Base Board Asset Tag
dmi.board.name: JM12-MS-CAP
dmi.board.vendor: Acer
dmi.board.version: Base Board Version
dmi.chassis.type: 10
dmi.chassis.vendor: Chassis Manufacturer
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnINSYDE:bvrV1.3127:bd04/23/2010:svnAcer:pnAspire1825PTZ:pvrV1.3127:rvnAcer:rnJM12-MS-CAP:rvrBaseBoardVersion:cvnChassisManufacturer:ct10:cvrChassisVersion:
dmi.product.name: Aspire 1825PTZ
dmi.product.version: V1.3127
dmi.sys.vendor: Acer
version.compiz: compiz 1:0.9.7.4-0ubuntu2
version.ia32-libs: ia32-libs N/A
version.libdrm2: libdrm2 2.4.32-1ubuntu1
version.libgl1-mesa-dri: libgl1-mesa-dri 8.0.2-0ubuntu3
version.libgl1-mesa-dri-experimental: libgl1-mesa-dri-experimental N/A
version.libgl1-mesa-glx: libgl1-mesa-glx 8.0.2-0ubuntu3
version.xserver-xorg-core: xserver-xorg-core 2:1.11.4-0ubuntu10
version.xserver-xorg-input-evdev: xserver-xorg-input-evdev 1:2.7.0-0ubuntu1
version.xserver-xorg-video-ati: xserver-xorg-video-ati 1:6.14.99~git20111219.aacbd629-0ubuntu2
version.xserver-xorg-video-intel: xserver-xorg-video-intel 2:2.17.0-1ubuntu4
version.xserver-xorg-video-nouveau: xserver-xorg-video-nouveau 1:0.0.16+git20111201+b5534a1-1build2

[Chase Douglas (chasedouglas)]

There are many bugs in the touchscreen pointer emulation code. I have developed many fixes for them and sent them on to <email address hidden>. I also created one big patch for the precise xserver. I have uploaded it to ppa:chasedouglas/jupiter as xorg-server 1.11.4-0ubuntu11~jupiter5. Once it has built, please test it out.

Note that there appears to be a bug in gnome panel where it will no longer show the menu after tapping it. I have traced the server and found that it still does send events and receive replies from the panel, so I believe this is a bug in the panel itself rather than the server. The server is still working properly if you can interact with other objects like the desktop.

[mweyen (mweyen)]

this changed things and also improved but whenever i open an item in the top right and try to click on one of the menus lightdm seems to crash and i am back at the login screen.
even on the login screen i have the same behaviour: trying to enable the screen keyboad -> crash, anything else: crash too

[Chase Douglas (chasedouglas)]

Ahh, I missed that. I can reproduce here. Hopefully I can fix it quickly.

Thanks!

[Chase Douglas (chasedouglas)]

I have uploaded a new version of the xserver and it is now built and available. Please retest.

Thanks!

[mweyen (mweyen)]

Thanks. That version solves the described problems.

while touching around it works quite nicely now (even though i still had something hang again, but wasn't able to reproduce it).
But this jupiter6 version has a new problem i noticed compared to the current precise version:

i cannot move around in the menu anymore using touch. (trying to explain ...)

Example: on the login screen (but happens everywhere it seems) i want to enable the screen keyboard.

i touch the icon, then touch the now open menu and leave my finger on it, because i know i do not always hit it exactly (more precisely i mostly hit the increase contrast thingy) so i leave my finger on it and move towards the menu item i desire and do not lift the finger away from the touchscreen until i see the correct item highlighted.

with the version from your ppa this does not work. there it directly starts whatever i hit, no matter if the finger is still on the screen or not. This works with the version currently in precise.

[Henri Sundelin (henri-sundelin)]

I also tried jupiter6 version - it almost works. (MT grab handles come out etc as it should). However I crashes now and then, backtrace attached.

[Chase Douglas (chasedouglas)]

Hi Henri,

Your backtrace looks like a general corruption of some sort. We have been seeing these in the precise X server, but we have not been able to track down the issue. If you can find a way to reproduce the issue please let us know.

THanks!

[Henri Sundelin (henri-sundelin)]

Hmm,

This happens quite often if you:
- triple tap MT handles
- then move windows around

Seems you get this only with multitouch though. Is it possible this has something to do kernel multitouch drivers?
(I'm running 3.3.1, seems to have best hid-multitouch driver so far)

I checked that second & third freezes gave similar bt:

#0 0x00007f12598531bb in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f12597d7cb1 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007f12597d5a37 in malloc () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007f125b6e0ee7 in XIChangeDeviceProperty (dev=0x7f125d24ee90,
    property=<optimized out>, type=19, format=8, mode=<optimized out>,
    len=<optimized out>, value=0x7fffa4e319df, sendevent=1)
    at ../../Xi/xiproperty.c:754
#4 0x00007f125b600a8c in DisableDevice (dev=0x7f125d24ee90,
    sendevent=1 '\001') at ../../dix/devices.c:505
#5 0x00007f125b600d38 in RemoveDevice (dev=0x7f125d24ee90, sendevent=1 '\001')
    at ../../dix/devices.c:1092
#6 0x00007f125b658d72 in DeleteInputDeviceRequest (pDev=0x7f125d24ee90)
    at ../../../../hw/xfree86/common/xf86Xinput.c:1007
#7 0x00007f125b5fcd5c in CloseDeviceList (listHead=0x7f125b9bf308)
    at ../../dix/devices.c:997
#8 0x00007f125b5fd301 in CloseDownDevices () at ../../dix/devices.c:1027
#9 0x00007f125b74c4b7 in SigAbortServer (signo=11) at ../../os/log.c:419
.. etc

On 16 Apr 2012, at 21:36, Chase Douglas wrote:

> Hi Henri,
>
> Your backtrace looks like a general corruption of some sort. We have
> been seeing these in the precise X server, but we have not been able to
> track down the issue. If you can find a way to reproduce the issue
> please let us know.
>
> THanks!
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/974887
>
> Title:
> unity 3d touch stops registering clicks
>
> Status in “xorg-server” package in Ubuntu:
> In Progress
>
> Bug description:
> Hi,
>
> similar to #949791 ...
>
> that bug made things better (i can now move windows using the touch
> screen) but it still often stops to react to any clicks when using
> touch.
>
> I just updated.
>
> In unity 3d (not in unity 2d nor in gnome shell) if i touch the top
> bar without any program running it most of the time stops to react to
> any clicks (both touch and clicks of the touch pad buttons). I cannot
> reproduce it always with the first touch but usually latest with 2 or
> 3.
>
> i am not sure what happens but after something around a minute it
> starts to work again ..... i tried the super key to open the dash and
> then it suddenly worked again. I'm sorry but i can't say exactly what
> happens.
>
> this happened like 2 or 3 times in this session, so maybe you see
> something in the logs ubuntu-bug attaches.
>
> If you need more info i'll do my best to provide it.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 12.04
> Package: unity 5.8.0-0ubuntu2
> ProcVersionSignature: Ubuntu 3.2.0-22.35-generic 3.2.14
> Uname: Linux 3.2.0-22-generic x86_64
> .tmp.unity.support.test.0:
>
> ApportVersion: 2.0-0ubuntu4
> Architecture: amd64
> CompizPlugins: [core,composite...

[Chase Douglas (chasedouglas)]

Hi mweyen,

I believe the issue you are seeing is a bug in gtk. It is using GTK 3, which now has some touch support. I have tracked the X event handling through the X server and found that it is sending events correctly. Please file a separate bug for this issue against gtk+3.0.

Please confirm that there are no other issues with the jupiter6 X server. I would like to begin preparing an SRU for the fixes for this bug.

Thanks!

[mweyen (mweyen)]

hi,

except for that described menu issue i can confirm that it works for me.

i just retried purging the jupiter6 version and i definitely don't have the described menu issue with the 'original' precise X server.

I feel a bit bad/weird reporting a bug against another package that only appears with a package version from a PPA ... but if you want, i'll do that tomorrow.

[Chase Douglas (chasedouglas)]

Hi mweyen,

Good point about reporting a bug against a PPA. Lets wait until this bug is fixed at least in precise-proposed.

Thanks!

[Chase Douglas (chasedouglas)]

I have found the causes of both Henri's and mweyen's issues, along with multiple other bug fixes along the way. They have all be proposed upstream on the xorg-devel mailing list and uploaded to the jupiter ppa as 1.11.4-0ubuntu11~jupter7. Please retest.

[Henri Sundelin (henri-sundelin)]

Tested jupiter7, works properly now.

Thanks,

[mweyen (mweyen)]

works here as well

Thanks !

[Chase Douglas (chasedouglas)]

xorg-server has been accepted into precise-proposed. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[mweyen (mweyen)]

retested and works for me

[Launchpad Janitor (janitor)]

This bug was fixed in the package xorg-server - 2:1.11.4-0ubuntu10.1

---------------
xorg-server (2:1.11.4-0ubuntu10.1) precise-proposed; urgency=low

  [ Bryce Harrington ]
  * Enable 227_null_ptr_midispcur.patch to apply

  [ Chase Douglas ]
  * Update to xserver 1.12.1 for the input stack
  * Drop patches merged upstream in 1.12.1:
    - 501_touch_accept_end.patch
    - 502_indirect_touch_window_set.patch
    - 503_fix_mouse_warp.patch
    - 504_implement_passive_touch_ungrab.patch
  * Fix patch 506_touchscreen_pointer_emulation_checks.patch after upstream
    review
  * Fix various touchscreen issues (LP: #974887)
    - Add temporary patch 506_touchscreen_fixes.patch, which is a combination of
      multiple upstream patches
 -- Chase Douglas <email address hidden> Thu, 19 Apr 2012 12:34:54 -0700

[Martin von Gagern (gagern)]

One aspect of those patches, the removal of TOUCH_END events, breaks emulated pointer core mouse event state for me. See bug #1015183 for discussion.