Abstract:Reconstructing 3D human shape and pose from monocular images is challenging despite the promising results achieved by the most recent learning-based methods. The commonly occurred misalignment comes from the facts that the mapping from images to the model space is highly non-linear and the rotation-based pose representation of body models is prone to result in the drift of joint positions. In this work, we investigate learning 3D human shape and pose from dense correspondences of body parts and propose a Decompose-and-aggregate Network (DaNet) to address these issues. DaNet adopts the dense correspondence maps, which densely build a bridge between 2D pixels and 3D vertices, as intermediate representations to facilitate the learning of 2D-to-3D mapping. The prediction modules of DaNet are decomposed into one global stream and multiple local streams to enable global and fine-grained perceptions for the shape and pose predictions, respectively. Messages from local streams are further aggregated to enhance the robust prediction of the rotation-based poses, where a position-aided rotation feature refinement strategy is proposed to exploit spatial relationships between body joints. Moreover, a Part-based Dropout (PartDrop) strategy is introduced to drop out dense information from intermediate representations during training, encouraging the network to focus on more complementary body parts as well as neighboring position features. The efficacy of the proposed method is validated on both indoor and real-world datasets including Human3.6M, UP3D, COCO, and 3DPW, showing that our method could significantly improve the reconstruction performance in comparison with previous state-of-the-art methods. Our code is publicly available at <a class="link-external link-https" href="https://hongwenzhang.github.io/dense2mesh" rel="external noopener nofollow">this https URL</a> .

LEAPSE: Learning Environment Affordances for 3D Human Pose and Shape Estimation

LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

LEAP: Liberate Sparse-view 3D Modeling from Camera Poses

Embodied Scene-aware Human Pose Estimation

Geometric Pose Affordance: 3D Human Pose with Scene Constraints

Live Stream Temporally Embedded 3D Human Body Pose and Shape Estimation

Unsupervised Universal Hierarchical Multi-Person 3D Pose Estimation for Natural Scenes

Resolving 3D Human Pose Ambiguities with 3D Scene Constraints

Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments

Lifting by Image -- Leveraging Image Cues for Accurate 3D Human Pose Estimation

LASOR: Learning Accurate 3D Human Pose and Shape Via Synthetic Occlusion-Aware Data and Neural Mesh Rendering

3D Human Pose and Shape Estimation with Dense Correspondence from a Single Depth Image

Learning 3D Human Shape and Pose from Dense Body Parts

Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation

PONet: Robust 3D Human Pose Estimation via Learning Orientations Only

Language-Conditioned Affordance-Pose Detection in 3D Point Clouds

Sequential 3D Human Pose and Shape Estimation from Point Clouds

Robust Estimation of 3D Human Poses from a Single Image

3D Human Pose Estimation via Intuitive Physics

Learning Temporal-Spatial Contextual Adaptation for Three-Dimensional Human Pose Estimation

Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation