Abstract:Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model personalization a different problem and use manual or semi-automatic model initialization, which greatly reduces applicability. In this paper, we propose a fully automatic algorithm that jointly creates a rigged actor model commonly used for animation - skeleton, volumetric shape, appearance, and optionally a body surface - and estimates the actor's motion from multi-view video input only. The approach is rigorously designed to work on footage of general outdoor scenes recorded with very few cameras and without background subtraction. Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy. For reconstruction, 3D body shape is approximated as Gaussian density field. For pose and shape estimation, we minimize a new edge-based alignment energy inspired by volume raycasting in an absorbing medium. We further propose a new statistical human body model that represents the body surface, volumetric Gaussian density, as well as variability in skeleton shape. Given any multi-view sequence, our method jointly optimizes the pose and shape parameters of this model fully automatically in a spatiotemporal way.

Markerless motion capture of interacting characters using multi-view image segmentation

Markerless motion capture of multiple characters using multiview image segmentation

Shape and Pose Estimation for Closely Interacting Persons Using Multi-view Images.

Full-body Motion Capture for Multiple Closely Interacting Persons.

Markerless Shape and Motion Capture From Multiview Video Sequences

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Multi-view Human Motion Capture with an Improved Deformation Skin Model

Towards Accurate Markerless Human Shape and Pose Estimation over Time

3D hypothesis clustering for cross-view matching in multi-person motion capture

Performance Capture Of Interacting Characters With Handheld Kinect

Markerless Body Motion Capturing for 3D Character Animation based on Multi-view Cameras

Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras

New multi-view human motion capture framework

Human Motion Estimation from a Reduced Marker Set.

Human motion segmentation using collaborative representations of 3D skeletal sequences.

Synergetic Reconstruction from 2D Pose and 3D Motion for Wide-Space Multi-Person Video Motion Capture in the Wild

Outdoor Markerless Motion Capture with Sparse Handheld Video Cameras

Two-camera-based Human Motion Capture

Human Performance Capture Using Multiple Handheld Kinects

4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras

General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues