Touchpad Gestures Need Visible Feedback

Bug #672766 reported by Rising Eagle
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Open Input Framework
Undecided
Unassigned

Bug Description

Right now, the only feedback from the touchpad is when the cursor moves (or screen scrolls, or whatever). We get no feedback on what our fingers are doing on the touchpad surface itself or what the touchpad is experiencing. When we write, we see the lettering as it is appearing immediately on the paper as the pen is forming the letter or pattern. Or when we fingerpaint, we see bold, curvy strokes of color trailing behind our moving finger(s). We need that same feedback with the touchpad.

1) When the touchpad detects pressure/presence/fingertouch, a transparent shadow (or more aesthetic graphic representation thereof) of the pressure points/spots/fingerprints should appear on the computer screen with transparency reduced (or darkness/opacity increased) proportional to pressure.

2) The touchpad surface area should map directly onto the computer screen (though the user could adjust the size of the representational area on the screen).

3) The touchpad mapping will be completely invisible UNLESS the touchpad feels pressure or is actively being used. When the touchpad first detects activity, a faint transparent boundary appears on the computer screen representing the touchpad's entire surface area (whose size on the screen is configured by the user) and pressure shadows appear where fingers or pressures are applied on the touchpad surface. The touchpad graphics will then phase out after some brief delay once activity stops. The graphics will always be transparent so that they will never block out the work content beneath it and so that visual annoyance is minimized.

4) Drag trails can be drawn so that gestures can clearly be seen on the screen. A good example is using the broad paint brush in photoshop. Multitouch gestures will have different colors assigned to each separate finger or pressure trail. Drag trails will fade away slowly after gesture is complete, or will fade quickly if the next gesture begins before fade out is done. A cool effect would be that fade out is progressive so the beginning of a gesture starts fading earlier than later parts of the gesture (like the way a dust trail dissipates behind a motor bike speeding through the desert or the way a con trail dilutes and spreads after a fighter jet flys by and then fades from the sky later on; we could even show the drag trails "drifting with the wind" as the con trails do in the sky as they fade). Timing and effects themes could be configurable by the user (this is a whole new domain for creating themes of various kinds and they can even be connected with various desktop themes or application skins that are out there already).

5) We can present visible feedback cues for action triggers such as left button, right button, double/triple clicks and taps, scrolling, dwells or lingerings, and for disabled access behaviours/features.

6) We can have the various differently defined areas within the touchpad mapping shown and possibly labled along with the overall boundary in order to provide better finger position awareness (fpa). Examples are the scroll region to the right or across the bottom edge (I have often tried to scroll, but landed my fingers outside the scroll region without knowing it). Another example is the grid pattern overlaid upon the touchpad gesture region which is used to measure gesture motion.

7) An immediate benefit is the feedback we get when accidentally brushing or grazing the touchpad while typing. We will see that an action has occurred and immediately stop typing. I often have the cursor move accidentally to another part of the screen because of an accidental brushing of pressure on the touchpad and yet I continue typing until I finally notice something wrong. Then I have to find where the cursor went to, delete the extra characters, then go back and take up where I left off.

8) The same software concept can be used to create a gesture tutorial or even a game with visible trails that the user trys to trace as practice for learning the gesture lexicon.

description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
Rising Eagle (borntobesteve) wrote :

How this graphic depiction works in cooperation with the visible cursor is an open question that must be considered in order to provide an optimal user experience.

description: updated
Revision history for this message
Duncan McGreggor (oubiwann) wrote :

There is already a blueprint for this here:
  https://blueprints.launchpad.net/ubuntu/+spec/multimedia-dx-n-basic-visual-touch-feedback

The session was held recently at UDS and the notes are available on gobby.ubuntu.com (requires gobby 0.5) in the "multimedia" section. I would recommend creating a brainstorm with the information you have provided in this ticket. Checkout the site here:
  http://brainstorm.ubuntu.com/
Once you do so, be sure to link your brainstorm to the multitouch wiki page:
  https://wiki.ubuntu.com/Multitouch#Brainstorming

Please note:
Bug tickets on launchpad represent a unit of work (e.g., bugs and tasks), and this ticket you have created assumes a fairly large number of development tasks. As such, this ticket will eventually be marked as invalid, only because the scope is too big. You have some great ideas, and I'l like to see them get the attention they deserve. There's a good chance that most or all of them will get covered the the course of addressing this (and next) cycle's design and UX work on touch, but it's better to be safe than sorry :-)

Revision history for this message
Rising Eagle (borntobesteve) wrote :

Thx for the advice Duncan. It's done:

http://brainstorm.ubuntu.com/idea/26418

Revision history for this message
Chase Douglas (chasedouglas) wrote :

I am marking this bug as invalid due to the number of tasks involved. The bug reporter has found other ways of providing his input.

Changed in utouch:
status: New → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers