Abstract:This work addresses the problem of estimating the full body 3D human pose and shape from a single color image. This is a task where iterative optimization-based solutions have typically prevailed, while Convolutional Networks (ConvNets) have suffered because of the lack of training data and their low resolution 3D predictions. Our work aims to bridge this gap and proposes an efficient and effective direct prediction method based on ConvNets. Central part to our approach is the incorporation of a parametric statistical body shape model (SMPL) within our end-to-end framework. This allows us to get very detailed 3D mesh results, while requiring estimation only of a small number of parameters, making it friendly for direct network prediction. Interestingly, we demonstrate that these parameters can be predicted reliably only from 2D keypoints and masks. These are typical outputs of generic 2D human analysis ConvNets, allowing us to relax the massive requirement that images with 3D shape ground truth are available for training. Simultaneously, by maintaining differentiability, at training time we generate the 3D mesh from the estimated parameters and optimize explicitly for the surface using a 3D per-vertex loss. Finally, a differentiable renderer is employed to project the 3D mesh to the image, which enables further refinement of the network, by optimizing for the consistency of the projection with 2D annotations (i.e., 2D keypoints or masks). The proposed approach outperforms previous baselines on this task and offers an attractive solution for direct prediction of3D shape from a single color image.

3D Human Pose Estimation from RGB+D Images with Convolutional Neural Networks

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

A Joint Relationship Aware Neural Network for Single-Image 3D Human Pose Estimation

3D Human Pose Estimation by Depth Map

3D Human Pose Estimation with 2D Human Pose and Depthmap

3D Human Pose Estimation = 2D Pose Estimation + Matching

Context-Aware Network for 3D Human Pose Estimation from Monocular RGB Image.

Attention Residual Network with 3D convolutional neural network for 3D Human Pose Estimation.

Rgb-D Fusion for Point-Cloud-Based 3d Human Pose Estimation

3D Human Pose Estimation Based on Volumetric Joint Coordinates

Improved Modular Convolution Neural Network for Human Pose Estimation

Learning to Estimate 3D Human Pose and Shape from a Single Color Image

3D Human Body Shape and Pose Estimation from Depth Image.

Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose

Position Constrained Network for 3D Human Pose Estimation

An RGB-D Based Approach for Human Pose Estimation

3D Human Pose Estimation Via Spatio-Temporal Matching from Monocular RGB Images.

Monocular 3D Human Pose Estimation by Predicting Depth on Joints

3D Human Pose Estimation with Cross-Modality Training and Multi-Scale Local Refinement

3D Human Pose Estimation Using RGBD Camera

3D-UGCN: A Unified Graph Convolutional Network for Robust 3D Human Pose Estimation from Monocular RGB Images