Abstract:We describe a method for articulated human detection and human pose estimation in static images based on a new representation of deformable part models. Rather than modeling articulation using a family of warped (rotated and foreshortened) templates, we use a mixture of small, nonoriented parts. We describe a general, flexible mixture model that jointly captures spatial relations between part locations and co-occurrence relations between part mixtures, augmenting standard pictorial structure models that encode just spatial relations. Our models have several notable properties: 1) They efficiently model articulation by sharing computation across similar warps, 2) they efficiently model an exponentially large set of global mixtures through composition of local mixtures, and 3) they capture the dependency of global geometry on local appearance (parts look different at different locations). When relations are tree structured, our models can be efficiently optimized with dynamic programming. We learn all parameters, including local appearances, spatial relations, and co-occurrence relations (which encode local rigidity) with a structured SVM solver. Because our model is efficient enough to be used as a detector that searches over scales and image locations, we introduce novel criteria for evaluating pose estimation and human detection, both separately and jointly. We show that currently used evaluation criteria may conflate these two issues. Most previous approaches model limbs with rigid and articulated templates that are trained independently of each other, while we present an extensive diagnostic evaluation that suggests that flexible structure and joint training are crucial for strong performance. We present experimental results on standard benchmarks that suggest our approach is the state-of-the-art system for pose estimation, improving past work on the challenging Parse and Buffy datasets while being orders of magnitude faster.

Articulated motion estimation from a monocular image sequence using spherical tangent bundles

A Stochastic-Geometrical Framework for Object Pose Estimation Based on Mixture Models Avoiding the Correspondence Problem

Two-View Multibody Structure from Motion

Unsupervised Articulated Skeleton Extraction from Point Set Sequences Captured by a Single Depth Camera

Motion Representations for Articulated Animation

3D Articulated Skeleton Extraction Using a Single Consumer-Grade Depth Camera.

Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation

Robust Camera Motion Estimation in Video Sequences

Exploiting polar symmetry in designing equivariant observers for vision-based motion estimation

Motion Estimation in Computer Vision: Optimization on Stiefel Manifolds

Trajectory Space: A Dual Representation for Nonrigid Structure from Motion

Learning Articulated Motions From Visual Demonstration

Learning to Search on Manifolds for 3D Pose Estimation of Articulated Objects

Articulated Human Detection with Flexible Mixtures of Parts

Articulated motion reconstruction from feature points

Learning Part Motion of Articulated Objects Using Spatially Continuous Neural Implicit Representations

ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild.

Structure from Motion for Panorama-Style Videos

A Riemannian Take on Human Motion Analysis and Retargeting

Robust Algebraic Segmentation of Mixed Rigid-Body and Planar Motions from Two Views