[13] A Survey of POMDP Applications (Cassandra – 2004)

Current Mood: studious

Blogs I Commented On:

• Kevin Wei
• Josh Johnston
• Pankaj Rajan

Summary:
The paper gives a brief introduction of an certain type of Markov decision process model (MDP) called partial observable MDP (POMDP), and also a survey of applications that benefit from the use of POMDP). In POMDP models, there consists the following: a fine state set of states S (all possible, unobservable states that the process can be in), actions A (representing all available control choices at a point in time), observations Z (all possible observations that the process can emit), along with a state transition, observation, and immediate reward function tau (encodes the uncertainty in the process state evolution), o (relates the observations to the true process state), and r (gives the immediate utility for an action in each process state). The goal is to derive a control policy that will yield the highest utility over a number of decision steps.

The majority of the paper goes into the survey of applications, but none of them address haptics directly. The end of the paper then goes into two types of limitations from POMDPs: theoretical and real. For theoretical limitations, POMDPs do not easily handle problems that have certain characteristics, and the model is itself data intensive. For the real limitations, the first problem involves representing states as a set of attributes, which causes small concepts to have large state spaces since it requires enumerating every attribute value combination. The second problem is that the optimal policy for a general POMDP is intractable.

Discussion:
While it would have been preferably to have seen the paper focus specifically and in more detail of a particular application that used POMDPs to judge it merits, it appears to have made a decent case on the utility of using POMDPs as a potentially useful technique for the types of things that can be done in haptics. In addition, I think this paper gave an okay summary of POMDPs, assuming that the reader already had prior knowledge of MDPs.