3D Visual Detection of Correct NGT Sign Production (Lichtenauer, et al – 2007)

Current Mood: studious

Blogs I Commented On:

• Josh Peschel
• Josh Johnston
• Pankaj Rajan

Summary:
This paper focuses on hand tracking based on skin detection for the domain of Dutch sign language (NGT). The method involves the user producing a sign, the hands and heads being visually tracked, and then their features measured. A classifier is used to evaluate the measured features. For tracking, the hands and head are tracked by detecting skin color blocks assigned to the head and both hands from the previous positions, and then combining them with template tracking during occlusions between and hands and the head. An operator is then employed to click a square around face and head/hair/neck. Skin color is modeled by a 2D Gaussian perpendicular to the main direction of the distribution of positive skin samples in RGB space. For classification, the authors used fifty properties to the 2D/3D location and movement of the hands, measured at each frame. First, a reference sign is selected for each classifier and its time signal warped onto that sign using Dynamic Time Warping (DTW), solely for synchronization. Then the classifier records the properties under the assumption that the features are independent. Base classifiers are split for single features and then combined by the summation of their results, where a feature is selected for classification if less than 20% of the negative examples have a feature value within some 50% winsorization interval of the positive set. Signs are classified if they exceed some threshold value, which is determined by evaluating the positive training examples and using the median of the resulting values. Their approach gives 8% true positives against 5% false positives.

Discussion:
If I could summarize this paper in one sentence, it would be that it’s a very complicated hand gesture recognition system which relies on skin color. Given the amount of space dedicated to their technique which focuses on skin color to aid in recognition, I think it's made partially moot that requires manually selecting body parts beforehand as opposed to making it automatic. Add to the fact that it also requires some sort of ideal environment doesn't really motivate me to use their technique over some other vision-based technique. Then there's their results. It's not that they're good or bad. It's just...I still don't know how well their system performs even after seeing their results.