3D Human Pose Estimation = 2D Pose Estimation + Matching

Ching-Hang Chen,Deva Ramanan

DOI: https://doi.org/10.48550/arXiv.1612.06524

2017-04-11

Abstract:We explore 3D human pose estimation from a single RGB image. While many approaches try to directly predict 3D pose from image measurements, we explore a simple architecture that reasons through intermediate 2D pose predictions. Our approach is based on two key observations (1) Deep neural nets have revolutionized 2D pose estimation, producing accurate 2D predictions even for poses with self occlusions. (2) Big-data sets of 3D mocap data are now readily available, making it tempting to lift predicted 2D poses to 3D through simple memorization (e.g., nearest neighbors). The resulting architecture is trivial to implement with off-the-shelf 2D pose estimation systems and 3D mocap libraries. Importantly, we demonstrate that such methods outperform almost all state-of-the-art 3D pose estimation systems, most of which directly try to regress 3D pose from 2D measurements.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to estimate 3D human poses from a single RGB image. Specifically, the paper proposes a method to infer 3D human poses through intermediate 2D pose prediction in order to overcome the challenges of predicting 3D poses directly from image measurements. This method is based on two key observations:

1. Deep neural networks have revolutionized 2D pose estimation and can produce accurate 2D predictions even in the case of self - occlusion.
2. A large number of 3D motion - capture datasets are now available, which makes it possible to lift the predicted 2D poses to 3D through simple memorization methods such as the nearest - neighbor algorithm.

By combining the existing 2D pose estimation system and the 3D motion - capture library, the paper constructs a simple and effective architecture for estimating 3D human poses from a single RGB image. The experimental results show that this method outperforms the existing 3D pose estimation systems in most cases, which usually attempt to regress 3D poses directly from 2D measurements.