gestures are not rotated by evdev coordinate transformation matrix

[Timo Kluck]

When I rotate my screen 180 degrees and touch input along with it by using

xinput set-prop $ID "Coordinate Transformation Matrix" -1.0 0.0 1.0 0.0 -1.0 1.0 0.0 0.0 1.0

gestures are not rotated. That is, the rotation:

 * does take effect in QT4's Fingerpaint example (/usr/lib/qt4/examples/touch/fingerpaint/fingerpaint)
 * does not take effect for ginn
 * does not take effect for libgrip (i'm running several patched applications).

The effect is most apparent when I have two Evince windows side-by-side (Compiz grid) try to two-finger-drag one document up. Then the other document scrolls down.

This is still happening with today's update in natty-proposed (#774938). I attached my package versions.

[Timo Kluck (tkluck)]

I investigated this a bit more.

It seems that the coordinate matrix is applied by the x server (in dix/getevents.c), well *after* the x evdev driver is done with the event. The gesture recognition, however, happens in utouch-grail, which is loaded by the x evdev driver, and it happens *before / in parallel* with the evdev driver handling the event. After that, the gesture event follows a codepath quite distinct from the one for touch events, and the coordinate matrix is never applied.

It seems to me quite difficult to apply the coordinate matrix to the gesture events themselves; it would be a nonlinear transformation on the rotation angle, for example. That means that we should apply the coordinate matrix to the touch data, before utouch-grail takes a look at it.

I'm marking this as affecting utouch-grail since that seems to be to most likely place where to fix this. I'm probably not knowledgeable enough to produce a patch.

[Chase Douglas (chasedouglas)]

Hi Timo,

Yes, you're completely correct about what is going on :). The reason we haven't addressed this yet is because we are in the process of rearchitecting the utouch stack. The rearchitecture will place the gesture recognizer on the X client side, after the coordinate transformation has been applied in the server. Thus, this bug will be fixed automatically when we're finished (hopefully in time for P).

Unfortunately, that means the bug will exist until then. If you are interested, please feel free to propose a patch to fix this.

Thanks!

[Timo Kluck (tkluck)]

The quick-n-dirty way to patch this is probably to extend the grail api to allow setting a coordinate transform matrix. Then the evdev driver should watch for changes on its coordinate transform property and forward these changes to grail.

It's probably even easier to be able to pass a callback to grail which obtains the matrix. That makes for an ugly api (but that's okay if it's going to be changed anyway), but saves me from finding out how to watch for property changes (that's probably easy but I don't know how to do it).

I wouldn't mind investing some of my spare time on a patch to make this work in Oneiric. I would need some guidance though. And by what time should it be done to be able to make it into Oneiric?

[Chase Douglas (chasedouglas)]

Hmmm... I should have thought about this when I wrote my previous comment. Here's what it would take to get it into Oneiric:

1. Develop the patches against xserver-xorg-input-evdev and utouch-grail
2. Get the patches reviewed and tested (I would be happy to help out here :)
3. Get a feature freeze exception due to the API change (and the relatively large amount of new code)
4. Get the package uploaded to Oneiric

Unfortunately, step 3 is going to be near impossible. Feature freeze was last a week or two ago, and the farther you get from the freeze, the less likely you'll be granted an exception. Since you will need to develop the patches and test them first, it's unlikely it will be reasonable and done fast enough to warrant an exception. This would be a case where it would be best to consult a member of the release team, who arbitrate exceptions, before starting on the task.

Of course, we would be happy to take any patches and put them in our PPA for others to use, but it's up to you if you want to go that route.

[Timo Kluck (tkluck)]

Thanks for your explanation, Chase.

I'll see how far I can get with a patch this weekend. Getting it into a PPA should be fine; anyone who experiences this bug has most likely put some effort in getting things to work (disabling the x wacom driver, for instance), so they'll probably find their way to a PPA, too. And it's worth the effort for everyone who uses a convertible laptop/tablet (like me).

[Timo Kluck (tkluck)]

Here's two patches that fix this. They are minimal code changes and I hid the new ugly api call behind a GRAIL_DEPRECATED_API #define.

This was harder than I thought, mostly because grail keeps on passing a const struct utouch_frame* around. I couldn't just modify it (because it is owned by the utouch library) so I memcpy-d it. A problem is that the utouch frames contain references to each-other, which means that the transformed frames point to untransformed previous frames. In practice, however, it works for me.

[Chase Douglas (chasedouglas)]

Hi Timo,

Great work! I think the evdev patch is good, but I have some review comments about the grail patch. Please push it to a branch based off lp:utouch-grail and create a merge proposal. It will be easier to review the changes this way.

Thanks!

[Timo Kluck (tkluck)]

Hi Chase,

Thanks for taking a look! I just did what you asked.

[Chase Douglas (chasedouglas)]

utouch-frame and utouch-grail have been extended to provide coordinate transformation. Please test out the packages in the ppa:utouch-team/daily archive. If they work, we can release new versions of the software and upload them to ppa:utouch-team/utouch. Then we can add a patch to xserver-xorg-input-evdev and upload it there too.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "utouch grail patch (applies against r164)" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

[Bryce Harrington (bryce)]

Timo and Chase, how is it going with this bug, did Chase's changes resolve it? Are they included in Precise now? If so, we can probably close out this bug.

[Timo Kluck (tkluck)]

Off the top of my head: I think that the changes to utouch-grail and utouch-frame have been committed to trunk, so they are probably in precise too. I don't think the patch to xserver-xorg-input-evdev (taking advantage of it) ever got applied.

[Chase Douglas (chasedouglas)]

This should be resolved now that the utouch stack is on the client side of X. I am marking this as fix released.