Wiizards: 3D Gesture Recognition for Game Play Input (Kratz, et al – 2007)

Current Mood: studious

Blogs I Commented On:

• Aaron Wolin
• Brandon Paulson
• Katie Dahmen

Summary:
This is another applications paper that uses the Wiimote. Their application is called Wiizards, a multiplayer game which uses HMMs for gesture recognition. The game itself is a two player zero-sum game, where the player tries to damage to the other while limiting damage to one’s self. Gestures dictate spell casting, and players are more successful in the game if they use a variety of them. The three main components for the game is the Wiimote, gesture recognizer, and the game. Observations for the model are accelerometer data from the Wiimote, and then normalized using calibration information. Each gesture, which is an observation vector, is a collection of oberservations trained on the Baum-Welch algorithm, and also has a separate model associated with them for recognition. As calculated by the Viterbi Algorithm, the probability of a gesture given a model is the distribution of the observations and hidden states. To train the models, data was collected from 7 users. Each user were shown the gestures and performed them 40 times, and an HMM was created from the user data.

Discussion:
There are relatively few gesture recognition papers that cater to the Wiimote, since it’s a new device, and given the number of Wiimote papers on the topic matter, this would be considered a pretty good paper. It’s interesting that they built a nice application to demonstrate their recognizer using an HMM approach, but the system does have some kinks since the paper states about 50% accuracy by users whom hadn’t used the system before.

TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning (Lieberman & Breazeal – 2007)

Current Mood: studious

Blogs I Commented On:

• Josh Peschel
• Pankaj Rajan
• Aaron Wolin

Summary:
This is a hardware application paper with the goal of creating a robotic wearable suit to analyze target movement and provide real-time corrective vibrotactile feedback to a student’s body over multiple joints in order to quickly develop new motor skills. Their system consists of optical tracking (for motion capture using markers on the wearable device), tactile actuators (for proportional feedback at the joints), feedback software (for determining the vibrotactile signals), and customized hardware for output control. Their system was tested by having users copying a series of images on a video screen while wearing the suit. Their user study generally gave positive feedback on their system.

Discussion:
I didn’t know how to comment on this paper directly since its applications didn’t really relate to the core aspect of the course. Judged independently from the purpose of the class, I felt it was a wonderful system that also had a sufficient user study applied to it. I could see some merits related to our class if it concentrated more on the hand.

A Spatio-temporal Extension to Isomap Nonlinear Dimension Reduction (Jenkins & Matari – 2004)

Current Mood: studious

Blogs I Commented On:

• Josh Johnston
• Pankaj Rajan
• Brandon Paulson

Summary:
The focus of this paper appears to be in efficiently uncovering the structure of motion using unsupervised learning for dimension reduction. The authors use a spatio-temporal Isomap approach for both continuous and segmented input data with sequential temporal ordering, where continous ST-Isomap is suited for uncovering spatio-temporal manifolds of data, and segmented ST-Isomap is for uncovering spatio-temporal clusters in segmented data. Their technique tries to address temporal relationships of proximal disambiguation and distal correspondence in order to uncover the spatio-temporal structure. Their example of the two relationships is two low waving motions of different directions, and also a low and high motion of the same direction. In the former, the two motions fall in proximal disambiguation, and in the latter, the two motions fall in distal correspondence. Their ST-Isomap approach extends an Isomap approach by having temporal windowing to provide a temporal history for each data point, hard spatio-temporal correspondences between proximal data pairs, and distance between data airs with spatio temporal relationships to accentuate their similarity.

Discussion:
I honestly had no idea what this paper was talking about most of the time. Most of it was the lingering feeling that I couldn’t find an aspect of this paper that would be relevant to the topics we are doing in the class. But I think it’s safe to say that this a nice paper to refer to if one wishes to use unsupervised learning in hand motion.

Articulated Hand Tracking by PCA-ICA Approach (Kato, et al – 2006)

Current Mood: studious

Blogs I Commented On:

• Josh Johnston
• Pankaj Rajan
• Aaron Wolin

Summary:
This paper focuses on hand tracking using a PCA-ICA approach. To do so, the authors first model the human with OpenGL as spheres, columns, and a rectangular parallelepiped. Hand motion data is capture with a data glove by capturing all combinations of open and closed fingers so that angles for 20 joints were measured. These measurements were divided into 100 instances to obtain a 2000-dimensional hand motion row vector. PCA is then used to find a smaller set of variables with less redundancy, measured by correlations between data elements using Singular Value Decomposition. From their approach, the authors first use PCA to reduce dimensionality, and then perform ICA on the low-dimensional PCA subspace to extract feature vectors. For ICA, the authors use a neural learning algorithm to maximize the joint entropy by using stochastic gradient ascent. The ICA-based model thus can represent a hand pose by five independent parameters corresponding to a particular finger at a particular time instant. From the PCA-ISA approach, PCA basis vectors represent global hand motion with mostly unfeasible hand motions, where ICA basis vectors represent particular finger motion. Particle filtering is then used for tracking hands by first generating samples where the hand pose is determined by five parameters (corresponding to each finger) from the ICA-based model, and then by using an observation model for employing edge and silhouette information to evaluate their hypothesis.

Discussion:
If I pretend what the paper was talking about then I will say that I found it intriguing that they combined the strengths of PCA and ICA to come up with what appears to be a viable hand tracking system, in that PCA’s limitations were overcome by ICA to model the hand for tracking purposes. It’s kind of hard to judge the merits of this paper though based on scant results, but the images provided at the end of the paper in less-than-ideal environments. I wish it had working actual results though (online video link is dead).

The 3D Tractus: A Three-Dimensional Drawing Board (Lapides, et al – 2006)

Current Mood: studious

Blogs I Commented On:

• Pankaj Rajan
• Aaron Wolin
• Brandon Paulson

Summary:
This paper discusses the 3D Tractus, a drawing board-like device which can be raised and lowered to provide sketches in 3D. The device employs a counterweight system for easy vertical motion, four vertical aluminum bars for support, a tablet for the actual sketching, and a string potentiometer as a height senseor. For the software, a pen-based device handles input, and users have three visual software components to work with: a 3D sketch overview window, a drawing pad window, and a menu bar to access less common features. Dynamic line width is used to provide depth cues, and a traditional image editor-like eraser is used for deleting entire strokes.

Discussion:
The device discussed in this paper is an interesting concept for emulating 3D sketching using a standard tablet. There are obviously some limitations in providing true 3D sketching as it can only provide depth cues though a 2D window as opposed to a VR-based solution to visualize those same depths. I would imagine the usability would feel a bit distracting by relying on the other arm to navigate the drawing area in order to perform 3D sketching. Some improvements on the system I would suggest are a button to automate vertical movement, and an inclined surface to provide easier drawing. I can imagine some people whom would enjoy using this system over traditional devices.

A Hidden Markov Model Based Sensor Fusion Approach for Recognizing Continuous Human Grasping Sequences (Bernardin, et al – 2005)

Current Mood: studious

Blogs I Commented On:

• Pankaj Rajan
• Aaron Wolin
• Brandon Paulson

Summary:
This paper discusses a system based on HMMs for recognizing hand grasps. Classification follows grasp types from Kamakura’s grasp taxonomy, separating grasps into 14 different classes by purpose, hand shape, and contact points with grasped objects. Each HMM is assigned to a different grasp type, and recognition is performed using the Viterbi algorithm. The focus of their system differs from existing ones in that it distinguishes between the purpose of grasps, as opposed to the object shape or number of fingers. Their glove-based device is equipped with flexible capacitive sensors to measure sensor readings for grasping. Noise and unwanted motion was filtered out in a garbage model with ergodic topology, and a ‘task’ grammar was used to reduce the search space. Their only assumption was that a grasp motion is followed up by a release motion. Their system is able to achieve 90% for multiple users.

Discussion:
The recognition system discussed in this paper is a lot different from the other types of systems discussed in prior papers because none of the papers tackled the problem of recognizing grasping. I liked the paper because it was different and covers an area overlooked in the hand gesture recognition domain. The use of grasp as a feature is very intriguing, and I believe that incorporating it in a recognition system would make such a system more powerful. Reminds me of the hand tension paper, now that I think about it. It does seem like extracting grasping data is a non-trivial affair.

Temporal Classification: Extending the Classification Paradigm to Multivariate Time Series (Kadous – 2002)

Current Mood: studious

Blogs I Commented On:

• Pankaj Rajan
• Aaron Wolin
• Brandon Paulson

Summary:
The core idea behind this thesis work is taking advantage of multivariate time series in order to aid in hand gesture recognition accuracy. In particular, the author focuses on the sub-events that a human might detect as part of a sign within some sign language, or Australian Sign Language (Auslan) for this paper. The author goes on by saying that metafeatures can parameterize them by capturing their properties such as temporal characteristics within a parameter space, a 2-D space of time and height, for feature construction. The temporal classification system, which I’m guessing is called TClass, uses synthetic events found within this space for feature construction, and applying several metafeatures into training instances constructs synthetic features. This is done in order for the TClass to mix temporal and non-temporal features not found in other temporal classification systems, as claimed by the author. A motivation is to produce a temporal classifier which produces comprehensible yet accurate descriptions. The system was implemented on Auslan, a language where signs consists of a mixture of handshapes, location, orientation, movement, and expression. Data was collected on the Nintendo Powerglove and Flock. The first input device was very noisy and far inferior to the second device. Several machine learning techniques were tested in conjunction and in comparison with TClass. Some observations concerning the Flock data itself was that TClass didn’t perform well with the HMM, smoothing of data didn’t improve results, and that TClass can handle tons of data. Accuracy rates for the Flock data are stated to be at 98% accuracy on voting, which sounds like ensemble averaging.

Discussion:
For me, some of the results were a bit confusing for me to give a fair critique of what I thought of the performance the author’s system. This would warrant reading the rest of the thesis, but on areas which were clear, I thought it was a pretty good approach. The author also did a nice job in collection tons of data to build his system based on the more accurate Flock device, despite it not having multiple users like the Nintendo data. It seems like a sound approach with nice accuracy results, but comparisons to other temporal classifiers showing improvement would have made it better. I don’t think I saw them in the sections we were supposed to read.