Television control by hand gestures (Freeman & Weissman – 1994)

Current Mood: studious

Blogs I Commented On:

• Josh Peschel
• Katie Dahmen
• Josh Johnston

Summary:
The goal of this paper is to use hand gestures to operate a television set remote, and the authors do so by creating a user interface for new users to master instantly. The problems claimed is the lack of a vocabulary to do so along with the image processing problem of identifying and hand gestures quickly and reliably in a complex and unpredictable visual environment. The solution involves imposing constraints on the user and the environment by exploiting the visual feedback from the television. Users only memorize one gesture: holding an open hand to the television. The computer tracks the hand, echoes its position with a hand icon, and then lets the users operate on-screen controls. The hand recognition method uses normalized correlation, where the position of the maximum correlation is at the user’s hand. Local orientation is use, as opposed to pixel intensities, for robustness against lighting variations by using filters in the image processing stage. Background removal is used to avoid analyzing stationary objects like furniture by first linearly combining the current image with a running average image, then subtracting the two images to detect image positions where change was above some pre-set threshold. Positions above the change threshold were finally processed to gain efficiency and resolve false positives.

Discussion:
This is an ancient paper dating back to 14 years ago. Even though its primitive in contribution to today’s standard, I believe it was pretty innovative for its time in various concepts introduced in the paper. One example was its desire of trying to create a vocabulary for gesture recognition (though I found contributions on that were highly underwhelming), and another was the use of background removal to do visual detection of the hand for recognition purposes. The authors cheat by requiring an open hand to do detection, and their application doesn’t appear intuitive to perform the task, but I think the ideas in the paper were ahead of its time.