Unity8 pointer does not stay in sync with Qemu VM tablet input

Bug #1670670 reported by Pete Woods on 2017-03-07
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical System Image
High
Michał Sawicz
Mir
Invalid
Undecided
Unassigned
mir (Ubuntu)
High
Andreas Pokorny
qtmir (Ubuntu)
High
Unassigned
unity8 (Ubuntu)
High
Unassigned

Bug Description

Each time you login, or come back from the lock screen, you need to "recalibrate" the pointer by carefully moving the pointer into each corner.

Contrast this with unity7, where the absolute position of the virtual tablet pointer is tracked, and sync is never lost.

Michał Sawicz (saviq) wrote :

I think this might be due to us relying on relative (mouse/trackballs/trackpoints) instead of absolute (graphics tablets) input data.

Michał Sawicz (saviq) wrote :

Or, possibly, the lack of hardware cursor for nested servers still?

Changed in unity8 (Ubuntu):
status: New → Incomplete
kevin gunn (kgunn72) on 2017-03-07
Changed in canonical-devices-system-image:
assignee: nobody → Stephen M. Webb (bregma)
Changed in mir (Ubuntu):
assignee: nobody → Andreas Pokorny (andreas-pokorny)
Changed in canonical-devices-system-image:
importance: Undecided → High
Changed in mir (Ubuntu):
importance: Undecided → High
Daniel van Vugt (vanvugt) wrote :

Yes, anpok was talking about this problem in a meeting yesterday.

The issue is that U8 doesn't yet understand that some devices are absolute and not relative. As such U8 tracking it's own cursor position won't work.

Changed in mir (Ubuntu):
status: New → Invalid
Changed in canonical-devices-system-image:
assignee: Stephen M. Webb (bregma) → Michał Sawicz (saviq)
tags: added: input unity8-desktop
tags: added: vm
Changed in qtmir (Ubuntu):
importance: Undecided → High
Changed in unity8 (Ubuntu):
importance: Undecided → High
Changed in qtmir (Ubuntu):
status: New → Confirmed
Changed in unity8 (Ubuntu):
status: Incomplete → Confirmed
Changed in canonical-devices-system-image:
status: New → Confirmed
Daniel van Vugt (vanvugt) wrote :

I think in the long run to solve this bug (and to reduce cursor lag, and to eliminate the rendering load of U8 cursor movement) we will need to switch to using native Mir input coordinates as well as the hardware cursor.

AFAIK the only thing stopping us from using Mir's native input coordinates was the inability to implement barriers/edge resistance in places. But I guess we could add an API for that (also discussed in the same meeting yesterday).

Pete Woods (pete-woods) wrote :

I'm not totally convinced that this bug is about relative pointer movement. For example, if I leave the pointer at one corner of the virtual screen, then move it outside and all the way around the edge, then move it back inside the virtual screen at the opposite corner, the virtual pointer moves more or less to the bottom right.

Also, given that the pointers gradually calibrate and move more closely in sync with each other, it feels like there's some "estimate" for the mapping between the size of the graphics tablet and the screen that is improving as you increase the range of coordinates that U8 has sampled.

Gerry Boland (gerboland) wrote :

> AFAIK the only thing stopping us from using Mir's native input coordinates
> was the inability to implement barriers/edge resistance in places.

We had other use-cases where unity8 having control over mouse position was useful. One example coming to mind is phone docked to an external display, we wanted to limit the cursor to the external display only. And we had a few other pie-in-the-sky design ideas which would be much easier if unity8 had cursor control.

I still don't see why Mir can't just let Unity8 (a nested) position the cursor (like it could, if it were a host server)!

And absolute vs relative. If we're doing it wrong, we can fix it.

Gerry Boland (gerboland) on 2017-03-08
Changed in mir (Ubuntu):
status: Invalid → Confirmed
tags: added: cursor
Changed in mir:
status: New → Invalid
Changed in mir (Ubuntu):
status: Confirmed → Invalid
Daniel van Vugt (vanvugt) wrote :

Invalid for Mir because the enhancement Gerry mentioned is covered in bug 1600220 instead.

If Unity8 used the absolute pointer coordinates that USC gives it then we wouldn't have a problem here. If Unity8 doesn't want to use those all the time then we will need to work out a compromise that makes events from Qemu's mouse/tablet hybrid more obviously absolute (like touches are) so that clients (which U8 is one) know that relative motion from that device is useless.

In Qemu you can move the cursor off the right edge of the window, move it around the Qemu window and make it appear on some other edge. It would be messy at least to try and communicate that using relative motion data. If we really want to do that then sure this is a Mir bug, but I don't think we want to do that.

Useful links:
https://bugs.launchpad.net/mir/+bugs?field.tag=wacom
https://bugs.launchpad.net/mir/+bugs?field.tag=cursor

> We had other use-cases where unity8 having control over mouse position
> was useful. One example coming to mind is phone docked to an external
> display, we wanted to limit the cursor to the external display only. And
> we had a few other pie-in-the-sky design ideas which would be much
> easier if unity8 had cursor control.

Exactly this, btw, is still in play with our multi-monitor designs.

Daniel van Vugt (vanvugt) wrote :

Actually the Mir hack I describe in comment #7 would not work because it assumes the velocity and acceleration of USC and Unity8 cursors is identical.

We'll just need to find a way to ensure Unity8 knows some devices are absolute (which is required for pen tablets in general - see https://bugs.launchpad.net/mir/+bugs?field.tag=wacom).

Gerry Boland (gerboland) wrote :

Looking into QtMir, we're using relative

Gerry Boland (gerboland) wrote :

Looking into QtMir, we're only using the relative mouse motion in our calculation of the cursor position. We need that to implement the push-against-screen-edge behaviour for a mouse.

We've not considered tablet input devices at all. I see Mir's associates a device with each input event, so it possible for us to change behaviour depending on the input device. Overall I don't knom enough to speak intelligently about what we need to do, needs further investigation on our side.

Daniel van Vugt (vanvugt) wrote :

Whatever the solution, we need to be aware that a stylus can jump all over the screen but also move like a normal mouse cursor if it's in close proximity to (even not touching) the screen. The shell has no say in the movement then because the cursor needs to match the real-world stylus position, even when moving like a mouse.

I think that's the kind of device Qemu is emulating but also suspect Qemu's fake tablet is more complicated than that -- a mouse rather than a tablet, which emits absolute coordinates that can suddenly appear from any side of the virtual screen.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers