SwipeArea lets touch events through before a drag is detected

[Olivier Tilloy]

I recently build a branch of lp:webbrowser-app with the latest code drop for DirectionalDragArea from lp:unity8. The branch is there: lp:~osomon/webbrowser-app/update-dda.

The DDA is used to detect a bottom edge drag over a WebView component, and display the tabs view accordingly. This works well, but often when a drag is detected the WebView under it gets a long press event, thus triggering the selection mechanism that it implements.

According to Daniel with whom I discussed the issue prior to filing this bug:

> Yes, the recognition parameters (and algorithm) were changed a bit.
> So it's perfectly possible that what you consider a long press (don't
> know how many milliseconds of pressed are needed for that) can happen
> before the DirectionalDragArea reaches a decision on whether the gesture
> is a directional drag gesture or not.

The threshold duration for a long press to be detected by the WebView is 500ms.

I’ve rebuilt my branch with DIRECTIONALDRAGAREA_DEBUG set to 1, and here is what I’m seeing when observing the issue:

[DDA()] 10035 TouchBegin (id:8, state:pressed, scenePos:(268.503,917.002))
[DDA()] WaitingForTouch -> Undecided
[DDA()] Unowned 10072 TouchUpdate (id:8, state:moved, scenePos:(268.503,917.002))
[DDA()] Sill within composition window. Let's wait more.
[DDA()] Unowned 10089 TouchUpdate (id:8, state:moved, scenePos:(268.503,917.002))
[DDA()] Sill within composition window. Let's wait more.
[DDA()] Unowned 10106 TouchUpdate (id:8, state:moved, scenePos:(268.503,917.002))
[DDA()] movedFarEnoughAlongGestureAxis: scalarProjection=0, distanceThreshold=15.748
[DDA()] Didn't move far enough yet. Let's wait more.
[DDA()] Unowned 10123 TouchUpdate (id:8, state:moved, scenePos:(267.006,911.008))
[DDA()] movedFarEnoughAlongGestureAxis: scalarProjection=2.05677, distanceThreshold=15.748
[DDA()] Didn't move far enough yet. Let's wait more.
[DDA()] Unowned 10139 TouchUpdate (id:8, state:moved, scenePos:(267.505,891.706))
[DDA()] movedFarEnoughAlongGestureAxis: scalarProjection=21.3591, distanceThreshold=15.748
[DDA()] grabbing touch
[DDA()] Undecided -> Recognized
[DDA()] 10150 TouchUpdate (id:8, state:moved, scenePos:(267.505,891.706))
[DDA()] 10186 TouchUpdate (id:8, state:moved, scenePos:(268.503,845.866))
[DDA()] 10215 TouchUpdate (id:8, state:moved, scenePos:(270.853,799.68))
[DDA()] 10238 TouchUpdate (id:8, state:moved, scenePos:(271.996,774.151))
[DDA()] 10248 TouchUpdate (id:8, state:moved, scenePos:(272.212,747.697))
[DDA()] 10260 TouchUpdate (id:8, state:moved, scenePos:(274.196,721.618))
[DDA()] 10288 TouchUpdate (id:8, state:moved, scenePos:(275.736,663.346))
[DDA()] 10345 TouchUpdate (id:8, state:moved, scenePos:(278.203,582.836))
[DDA()] 10465 TouchUpdate (id:8, state:moved, scenePos:(280.481,432.403))
[DDA()] 10483 TouchUpdate (id:8, state:moved, scenePos:(281.486,382.402))
[DDA()] 10501 TouchUpdate (id:8, state:moved, scenePos:(282.613,359.847))
[DDA()] 10522 TouchUpdate (id:8, state:moved, scenePos:(283.838,335.342))
[DDA()] 10532 TouchUpdate (id:8, state:moved, scenePos:(284.972,311.133))
[DDA()] 10545 TouchUpdate (id:8, state:moved, scenePos:(286.124,288.236))
[DDA()] 10590 TouchUpdate (id:8, state:moved, scenePos:(287.307,265.747))
[DDA()] 10600 TouchUpdate (id:8, state:moved, scenePos:(288.466,228.588))
[DDA()] 10601 TouchEnd (id:8, state:released, scenePos:(288.466,228.588))
[DDA()] Recognized -> WaitingForTouch

Let me know if I can provide more useful information.

[Daniel d'Andrada (dandrader)]

The log tells that 104 milliseconds have passed between the moment you landed your finger on the screen (10035) and the moment DirectionalDragArea recognized the gesture and grabbed the touch (10139). So much earlier than the 500 ms deadline for a longpress event to occur.

Could you please add a debug print (eg qDebug) at the moment your code notices a long press event? That way we would be able to see at what point in the log it happened.

[Daniel d'Andrada (dandrader)]

Also some pointers on how I can reproduce it locally would be nice (such as what test page to run if any, where in the code to look, any special build instructions, etc).

[Olivier Tilloy (osomon)]

Just updated my update-dda branch with the latest code drop from lp:unity8, and I’m still seeing this issue. I’m also seeing another related issue: sometimes the bottom-edge drag doesn’t trigger a long press event (i.e. no selection is shown), however it seems to leave the webview’s touch detection in an inconsistent state, where scrolling up and down with one finger results in zooming in/out, as if there was still a pending touch point that hadn’t been released.

Adding an oxide task.

[Olivier Tilloy (osomon)]

I’ve now updated my branch to use the public SwipeArea component from the UITK, and I’m still seeing an issue similar to the one originally reported. I can reproduce it with a minimal standalone example:

import QtQuick 2.4
import Ubuntu.Components 1.3
import com.canonical.Oxide 1.11
WebView {
  url: "https://en.wikipedia.org/wiki/Ubuntu_(operating_system)"
  SwipeArea {
    direction: SwipeArea.Upwards
    onDraggingChanged: console.log("dragging:", dragging)
    onDistanceChanged: console.log("distance:", distance)
    anchors {
      left: parent.left
      right: parent.right
      bottom: parent.bottom
    }
    height: units.gu(2)
  }
}

I instrumented oxide to print a log statement every time a touch event is detected, and when starting a drag even from the bottom edge, the following happens:

qml: distance: 0
virtual void OxideQQuickWebView::touchEvent(QTouchEvent*) QTouchEvent(TouchBegin states: QFlags(0x1), 1 points: (TouchPoint(0 QRectF(599.979,1848.5 2.9974x2.9974) pressed)))
virtual void OxideQQuickWebView::touchEvent(QTouchEvent*) QTouchEvent(TouchUpdate states: QFlags(0x2), 1 points: (TouchPoint(0 QRectF(613.967,1676.09 2.9974x1.99827) moved)))
virtual void OxideQQuickWebView::touchEvent(QTouchEvent*) QTouchEvent(TouchUpdate states: QFlags(0x2), 1 points: (TouchPoint(0 QRectF(618.213,1623.12 4.99566x2.9974) moved)))
virtual void OxideQQuickWebView::touchEvent(QTouchEvent*) QTouchEvent(TouchUpdate states: QFlags(0x2), 1 points: (TouchPoint(0 QRectF(620.961,1507.47 2.9974x1.99827) moved)))
virtual void OxideQQuickWebView::touchEvent(QTouchEvent*) QTouchEvent(TouchUpdate states: QFlags(0x2), 1 points: (TouchPoint(0 QRectF(620.961,1492.02 2.9974x1.99827) moved)))
qml: dragging: true
qml: distance: 148.6371093749999
[…]
qml: distance: 513.631586442578
qml: dragging: false

It appears the webview is getting a TouchBegin event, followed by a number of TouchUpdate events, before the SwipeArea even detects a drag. When it does, events stop being sent to the webview, which as a consequence never gets a TouchEnd event to match the first TouchBegin event, leaving it in an inconsistent state.

[Daniel d'Andrada (dandrader)]

Simply sending it a TouchEnd is not the way to go as it would tell WebView that the touch sequence belongs to it and that it should commit whatever UI changes this touch sequence caused to it.

If SwipeArea grabbed the touch, WebView should undo whatever changes that were caused by that touch.

To me it looks like WebView doesn't handle a touch point being grabbed from it and is thus left in a bad state.

[Olivier Tilloy (osomon)]

Ok, there’s probably something we can do in oxide to undo changes when a touch is grabbed. However, I’m still not sure I understand why the WebView is getting the touch events in the first place: the SwipeArea is above the WebView, shouldn’t it intercept the touch events before they reach the WebView?

[Daniel d'Andrada (dandrader)]

No, that's one of the corner stones of SwipeArea (and gesture detection in general): doing gesture recognition without adding latency to touch events reaching items behind it.

It's like a Flickable, it lets buttons in its content item react immediately to touch presses but cancels it (grabs the mouse/touch) as soon as it detects a flick being performed.

[Olivier Tilloy (osomon)]

OK, thanks for clarifying. I’m tentatively closing the ubuntu-ui-toolkit task then, let’s see if we can fix that in oxide.

[Launchpad Janitor (janitor)]

This bug was fixed in the package webbrowser-app - 0.23+16.04.20160509.3-0ubuntu1

---------------
webbrowser-app (0.23+16.04.20160509.3-0ubuntu1) xenial; urgency=medium

  [ CI Train Bot ]
  * Resync trunk.

  [ Olivier Tilloy ]
  * Fine-tune the custom memory-pressure handler, from data gathered on
    several devices. (LP: #1576639)
  * Update translation template.

 -- Olivier Tilloy <email address hidden> Mon, 09 May 2016 17:56:03 +0000