Hand gesture modeling and recognition involving changing shapes and trajectories, using a Predictive EigenTracker (Patwardhan & Roy – 2006)

Current Mood: studious

Blogs I Commented On:

• Brandon Paulson
• Pankaj Rajan
• Josh Johnston

Summary:
This is a hand gesture recognition paper that takes an eigenspace-based modeling approach, which takes into account trajectory and shape information. Recognition involves eigenspace projection and probability computation using the Mahalanobis distance. The paper is based on a previous system called EigenTracker, which can track moving objects that undergo appearance changes. The authors augment that system called Predictive EigenTracker in three ways: a particle filtering-based predictive framework, on the fly tracking of unknown views, and a combination of skin color and motion cues for hand tracking. Their gesture model framework accounts for both shape and temporal trajectory of the moving hand by constructing an eigenspace of suitably scaled shapes from a large number of training instances corresponding to the same shape. The framework also incorporates selecting a vocabulary to maximize recognition accuracy by computing the Mahalanobis distance of a query gesture from all gestures with some k shape-trajectory coefficient pair. This gives a probability of the given gesture being in some set of gestures. To test their system, the authors use a representative set of eight gestures for controlling a software audio player.

Discussion:
The paper was not an easy read for me, but from what I did read, it was an interesting system involving an approach we haven’t been exposed to yet. I do have some doubts, because I don’t know how this system is any different from using typical vision-based recognition specifically for hands. Concerning their experiment, I can’t gauge what the results are since they aren’t straightforward. Furthermore, the environment seems too controlled due to a white background and black, long-sleeved shirt in use for hand tracking.