Abstract:Reconstructing 3D human shape and pose from monocular images is challenging despite the promising results achieved by the most recent learning-based methods. The commonly occurred misalignment comes from the facts that the mapping from images to the model space is highly non-linear and the rotation-based pose representation of body models is prone to result in the drift of joint positions. In this work, we investigate learning 3D human shape and pose from dense correspondences of body parts and propose a Decompose-and-aggregate Network (DaNet) to address these issues. DaNet adopts the dense correspondence maps, which densely build a bridge between 2D pixels and 3D vertices, as intermediate representations to facilitate the learning of 2D-to-3D mapping. The prediction modules of DaNet are decomposed into one global stream and multiple local streams to enable global and fine-grained perceptions for the shape and pose predictions, respectively. Messages from local streams are further aggregated to enhance the robust prediction of the rotation-based poses, where a position-aided rotation feature refinement strategy is proposed to exploit spatial relationships between body joints. Moreover, a Part-based Dropout (PartDrop) strategy is introduced to drop out dense information from intermediate representations during training, encouraging the network to focus on more complementary body parts as well as neighboring position features. The efficacy of the proposed method is validated on both indoor and real-world datasets including Human3.6M, UP3D, COCO, and 3DPW, showing that our method could significantly improve the reconstruction performance in comparison with previous state-of-the-art methods. Our code is publicly available at <a class="link-external link-https" href="https://hongwenzhang.github.io/dense2mesh" rel="external noopener nofollow">this https URL</a> .

DL-PoseNet: A Differential Lightweight Network for Pose Regression over SE(3)

FDN: Feature Decoupling Network for Head Pose Estimation.

Unseen Object Pose Estimation via Registration

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

PDQ-Net: Deep probabilistic dual quaternion network for absolute pose regression on SE(3).

GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation

NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation

DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-scale Consistency

SEMPose: A Single End-to-end Network for Multi-object Pose Estimation

REG-Net: Improving 6DoF Object Pose Estimation with 2D Keypoint Long-Short-Range-Aware Registration

Learning 3D Human Shape and Pose from Dense Body Parts

DSC-HRNet: a lightweight teaching pose estimation model with depthwise separable convolution and deep high-resolution representation learning in computer-aided education

EPro-PnP: Generalized End-to-End Probabilistic Perspective-N-Points for Monocular Object Pose Estimation

SLBNet: Shallow and Lightweight Bilateral Network for Pose Estimation

LSDNet: lightweight stochastic depth network for human pose estimation

Pose Estimation Using Convolutional Neural Network with Synthesis Depth Data

Direct-PoseNet: Absolute Pose Regression with Photometric Consistency

PoseLifter: Absolute 3D human pose lifting network from a single noisy 2D human pose

REDE: End-to-End Object 6D Pose Robust Estimation Using Differentiable Outliers Elimination

BiCo-Net: Regress Globally, Match Locally for Robust 6D Pose Estimation