JOURNAL OF MULTIMEDIA (JMM)
ISSN : 1796-2048
Volume : 1    Issue : 7    Date : November/December 2006

Employing the Hand as an Interface Device
Afshin Sepehri, Yaser Yacoob and Larry S. Davis
Page(s): 18-29
Full Text:
PDF (3,427 KB)


Abstract
We propose algorithms and applications for using the hand as an interface device in virtual and
physical spaces. In virtual drawing, by tracking the hand in 3-D and estimating a virtual plane in
space, the intended drawing of user is recognized. In a virtual marble game, the instantaneous
orientation of the hand is simulated to render a graphical scene of the game board. Real-time visual
feedback allows the user to navigate a virtual ball in a maze. In 3-D model construction, the system
tracks the hand motion in space while the user is traversing edges of a physical object. The object
is then rendered virtually by the computer. These applications involve estimating the 3-D absolute
position and/or orientation of the hand in space. We propose parametric modelling of the central
region of the hand to extract this information. A stereo camera is used to first build a preliminary
disparity map of the hand. Then, the best fitting plane to the disparity points is computed using
robust estimation. The 3-D hand plane is calculated based on the disparity plane and the position
and orientation parameters of the hand. Tracking the hand region over a sequence of frames and
coping with noise using robust modelling of the hand motion enables estimating the trajectory of
the hand in space. The algorithms are real-time and experiments are presented to demonstrate the
proposed applications of using the hand as an interface device.

Index Terms
Virtual Drawing, Virtual  Game, 3-D Model Construction, Parametric Hand Tracking, Disparity
Modelling, Motion Modelling, Stereopsis