Abstract:Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model personalization a different problem and use manual or semi-automatic model initialization, which greatly reduces applicability. In this paper, we propose a fully automatic algorithm that jointly creates a rigged actor model commonly used for animation - skeleton, volumetric shape, appearance, and optionally a body surface - and estimates the actor's motion from multi-view video input only. The approach is rigorously designed to work on footage of general outdoor scenes recorded with very few cameras and without background subtraction. Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy. For reconstruction, 3D body shape is approximated as Gaussian density field. For pose and shape estimation, we minimize a new edge-based alignment energy inspired by volume raycasting in an absorbing medium. We further propose a new statistical human body model that represents the body surface, volumetric Gaussian density, as well as variability in skeleton shape. Given any multi-view sequence, our method jointly optimizes the pose and shape parameters of this model fully automatically in a spatiotemporal way.

Multi-view Human Motion Capture with an Improved Deformation Skin Model

Dynamic Human Body Reconstruction and Motion Tracking with Low-Cost Depth Cameras

New multi-view human motion capture framework

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Markerless motion capture of multiple characters using multiview image segmentation

Model-Based Markerless Human Body Motion Capture using Multiple Cameras

Markerless Human Body Motion Capture Using Multiple Cameras

Human model adaptation for multiview markerless motion capture

Markerless Shape and Motion Capture From Multiview Video Sequences

MODEL-BASEDMARKERLESS HUMAN BODY MOTION CAPTURE USINGMULTIPLE CAMERAS

Multi-View Articulated Human Body Tracking With Textured Deformable Mesh Model

A Robust Multi‐View System for High‐Fidelity Human Body Shape Reconstruction

Markerless Human Body Motion Capture Using Markov Random Field and Dynamic Graph Cuts

Markerless Body Motion Capturing for 3D Character Animation based on Multi-view Cameras

Markerless motion capture of interacting characters using multi-view image segmentation

Markerless 3D Human Motion Tracking for Monocular Video Sequences

Optimization and Soft Constraints for Human Shape and Pose Estimation Based on a 3D Morphable Model

Towards Accurate Markerless Human Shape and Pose Estimation over Time

Model-based markerless human body motion capture using active contour

General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

A new hierarchical method for markerless human pose estimation