Abstract:Reconstructing 3D human shape and pose from monocular images is challenging despite the promising results achieved by the most recent learning-based methods. The commonly occurred misalignment comes from the facts that the mapping from images to the model space is highly non-linear and the rotation-based pose representation of body models is prone to result in the drift of joint positions. In this work, we investigate learning 3D human shape and pose from dense correspondences of body parts and propose a Decompose-and-aggregate Network (DaNet) to address these issues. DaNet adopts the dense correspondence maps, which densely build a bridge between 2D pixels and 3D vertices, as intermediate representations to facilitate the learning of 2D-to-3D mapping. The prediction modules of DaNet are decomposed into one global stream and multiple local streams to enable global and fine-grained perceptions for the shape and pose predictions, respectively. Messages from local streams are further aggregated to enhance the robust prediction of the rotation-based poses, where a position-aided rotation feature refinement strategy is proposed to exploit spatial relationships between body joints. Moreover, a Part-based Dropout (PartDrop) strategy is introduced to drop out dense information from intermediate representations during training, encouraging the network to focus on more complementary body parts as well as neighboring position features. The efficacy of the proposed method is validated on both indoor and real-world datasets including Human3.6M, UP3D, COCO, and 3DPW, showing that our method could significantly improve the reconstruction performance in comparison with previous state-of-the-art methods. Our code is publicly available at <a class="link-external link-https" href="https://hongwenzhang.github.io/dense2mesh" rel="external noopener nofollow">this https URL</a> .

Hierarchical Neural Network for 3d Human Pose and Shape Estimation with Deep Shape Optimizer As Supervisor

Efficient Multi-person Hierarchical 3D Pose Estimation for Autonomous Driving

Unsupervised Universal Hierarchical Multi-Person 3D Pose Estimation for Natural Scenes

Detailed Human Shape Estimation From A Single Image By Hierarchical Mesh Deformation

Hierarchical Graph Networks for 3D Human Pose Estimation

Learning 3D Human Shape and Pose from Dense Body Parts

DaNet: Decompose-and-aggregate Network for 3D Human Shape and Pose Estimation

Learnable human mesh triangulation for 3D human pose and shape estimation

3D Human Pose Estimation Via Human Structure-Aware Fully Connected Network

Semi-Dynamic Hypergraph Neural Network for 3D Pose Estimation

HDPose: Post-Hierarchical Diffusion with Conditioning for 3D Human Pose Estimation

Ssman: self-supervised masked adaptive network for 3D human pose estimation

PoseNet3D: Learning Temporally Consistent 3D Human Pose via Knowledge Distillation

3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training

Estimating 3D body mesh without SMPL annotations via alternating successive convex approximation

Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video

Monocular 3D Human Pose Estimation by Predicting Depth on Joints

HiMoReNet: A Hierarchical Model for Human Motion Refinement.

Geometry-Driven Self-Supervised Method for 3D Human Pose Estimation

Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation.

Towards Accurate Markerless Human Shape and Pose Estimation over Time