A Gesture Language

Unknowingly we’ve acquired a lot of terminology regarding the use of a computer mouse. Apart from clicking, right-clicking and double-clicking, we’ve learned about dragging and dropping. Pushing two mouse buttons at the same time never reached much acceptance, but it is called a ‘chord-click”.
 
Personally, I’ve made frequently use of pens. Even though a Wacom tablet never seem to be it for me, I’ve been playing with Tablet PC’s from 2004 onwards. Even now, I regularly write with a pen using the Letter Recognizer input mode on my mobile phone to enter text and even to write articles. A stylus has a separate set of commands not fit for a computer mouse. With a pen you don’t click but you tap. Tab and Hold is the usual way to start the same function that a right mouse-click triggers. Flicks are quick movements with the pen in a certain direction. Since Windows Vista customizable commands are executed with these pen flicks. Gestures are also possible with a pen. Special software like StrokeIt is available to customize these pen gestures.  But… a stylus is out and fingers are in…
 
The essence of Touch interactions are fingers. Yet fingers are thick. A cursor beneath a fingertip cannot be seen and the object you place your finger on must be large enough and must have enough distance from other objects if you want to hit it very well. An interaction designer must find a way to show that an object is activated, even while a finger is obscuring that object. That advantage of a finger is that you always have one with you. What’s more: you’ve got another nine of them. This leads to a series of gestures, that all have a specific meaning in the touch interface of an application. A certain terminology is associated with this. A problem is that this terminology is not yet finalized. In spite of the fact that programmers can start implementing gestures in Windows 7, Surface and Silverlight, there is hardly a standard for naming and visualizing gestures. There is no exact basic set with an exact meaning. Design software templates and icons showing little hands are coming out now. But they all use different terminology:
 
Tap and double-tab: Push and let go
Slide, Swipe or Push: Sliding or pushing an object with a finger. 
Flick: Push, move quickly in one direction, let go.
Rotate: Moving two contact points in opposite directions in an ellipse. 
Touch-and-turn: Drag an object with only one contact point, so it moves but also rotates around that point.
Spin: Rotate an object quickly around its axis using one finger.
Pull apart or Stretch: Move two opposite contact points away from each other with the fingers of one or two hands.
Push together or Shrink: Move two opposite contact points towards each other with the fingers of one or two hands.
Twist: Turn an object with one or two fingers.
Pinch: Move two contact points towards each other by pinching the fingers of one hand.
Squeeze: Bringing three of more fingers of one hand together.
Spread: Moving two contact points apart by spreading the fingers of one hand.
Pin turn: Turning an object with three of more fingers around its one finger in the center
 
Don’t pin me down on the exact terminology. It is not clear which gestures we need for a coherent touch interface. On CodePlex there is a project with a much more ambitious goal to find a place for more creative gestures. Erase (from left to right), Create (a plus sign with one finger), Split (a diagonal line with one finger), Draw (holding a pen with three fingers), Clone (double-tab with two fingers), Global Rotate (rotate an entire application with five fingers of one hand) and Global Perspective Rotate, also known as Tilt (Tilt an application met five fingers of one hand using a finger of the other hand as a pivot point.
 
Despite this creative use of gestures, the basic set will likely not be much larger than that for a computer mouse. But which gestures are minimally necessary to control a computer interface? What extra gestures are necessary to get some work done in an application? It will take some more time before we have gathered all knowledge and experience necessary to use any computer with a touch interface in an intuitive manner…

About the Author

Antoni Dol

Antoni Dol is Senior Designer at Macaw since 1996. He graduated as illustrator at the Gerrit Rietveld Academy of Arts in Amsterdam, studied advertising at the Rotterdam Art Academy and was freelance illustrator for three years. From 1995 onwards, he grew as a web designer with the internet en from 1996 as a designer at Macaw. Antoni is experienced in Windows Presentation Foundation since October 2006 and designed applications in WPF and Silverlight. Antoni works with XAML in WPF and Silverlight projects on a daily basis. He wrote three books in Dutch about XAML, Silverlight 3 and Interaction Design.

#1 .net development on 8.14.2011 at 9:01 PM

many web browers have supported mouse geastures!

#2 text message marketing on 12.19.2011 at 11:41 AM

Great article. I often write with a pen using the Letter Recognizer input mode on my mobile phone. It took me awhile to adjust to the "tap" vs. the click of a mouse. Thanks so much for all the tips.

Leave a Comment