Inferring Human Intent from Video by Sampling Hierarchical Plans

Steven Holtzen,Yibiao Zhao,Tao Gao,Joshua B. Tenenbaum,Song-Chun Zhu
DOI: https://doi.org/10.1109/iros.2016.7759242
2016-01-01
Abstract:This paper presents a method which allows robots to infer a human's hierarchical intent from partially observed RGBD videos by imagining how the human will behave in the future. This capability is critical for creating robots which can interact socially or collaboratively with humans. We represent intent as a novel hierarchical, compositional, and probabilistic And-Or graph structure which describes a relationship between actions and plans. We infer human intent by reverse-engineering a human's decision-making and action planning processes under a Bayesian probabilistic programming framework. We present experiments from a 3D environment which demonstrate that the inferred human intent (1) matches well with human judgment, and (2) provides useful contextual cues for object tracking and action recognition.
What problem does this paper attempt to address?