Markerless motion capture of interacting characters using multi-view image segmentation

Yebin Liu,C. Stoll,J. Gall,H-P Seidel,C. Theobalt
DOI: https://doi.org/10.1109/CVPR.2011.5995424
2011-01-01
Computer Vision and Pattern Recognition
Abstract:We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video. Due to ambiguities in feature-to-person assignments and frequent occlusions, it is not feasible to directly apply single-person capture approaches to the multi-person case. We therefore propose a combined image segmentation and tracking approach to overcome these difficulties. A new probabilistic shape and appearance model is employed to segment the input images and to assign each pixel uniquely to one person. Thereafter, a single-person markerless motion and surface capture approach can be applied to each individual, either one-by-one or in parallel, even under strong occlusions. We demonstrate the performance of our approach on several challenging multi-person motions, including dance and martial arts, and also provide a reference dataset for multi-person motion capture with ground truth.
What problem does this paper attempt to address?