Device helps computers read hand gestures

US researchers have unveiled a device that allows computers to accurately read human hand gestures. The scientists, from the University at Buffalo's (UB) Virtual Reality Lab, said that their 'Fingertip Digitizer' is designed to transfer common hand gestures such as pointing and tapping in the air to the virtual world.

The component, which users wear on the tip of their index fingers, can be used to direct the actions of an electronic device, much like a mouse directs the actions of a personal computer, but, according to its creators, with greater precision.

The Fingertip Digitizer can transfer to personal computers very precise information about the physical characteristics of an object when a user taps, scratches, squeezes, strokes or glides a finger over the surface of the object.

"The gesture-recognition function of this device, in particular, has great potential for a wide range of applications, from personal computing to medical diagnostics to computer games," said Young-Seok Kim, who received his doctoral degree in mechanical engineering from UB in May. Kim created the Fingertip Digitizer with Thenkurussi Kesavadas, director of UB's Virtual Reality Lab and associate professor of mechanical and aerospace engineering in the UB School of Engineering and Applied Sciences.

According to Kesavadas, the Fingertip Digitizer will help bridge the gap between what a person knows and what a computer knows.

"With this device a computer, cell phone or computer game could read human intention more naturally," he explained. "Eventually the Fingertip Digitizer may be used as a high-end substitute for a mouse, a keyboard or a joystick."

The Fingertip Digitizer is described as a major enhancement in haptic technology, an emerging field focused on bringing a sense of touch to technological devices, according to Kim and Kesavadas.

The Fingertip Digitizer's design, the researchers explain, is modelled after the biomechanical properties of a finger, which means it can more accurately and intuitively sense the physical properties of an object. To sense touch and movement, the device uses a force sensor, an accelerometer and a motion tracker – all contained in a thimble-sized device that fits on a user's finger.

Kesavadas explained that a real-time, multi-rate data acquisition system used with the Fingertip Digitizer reads the force feedback exerted by an object as it is touched by the user. To read hand gestures, the system tracks the acceleration and location of the fingertip device as the finger moves and gestures.

A touch screen is not required. With the device attached to the fingertip, the user would simply gesture in the air as he looks at a computer screen where a software program or computer game may be running. In this way, the user can direct the opening or moving of an electronic file, for example. Using the device as a computer-game accessory, the user could imitate the squeezing of a trigger or the stroking of pool cue, for example, pointed out Kim and Kesavadas.

A provisional patent application has been filed on the device.