Abstract:Reconstructing 3D human shape and pose from a monocular image is challenging despite the promising results achieved by most recent learning based methods. The commonly occurred misalignment comes from the facts that the mapping from image to model space is highly non-linear and the rotation-based pose representation of the body model is prone to result in drift of joint positions. In this work, we present the Decompose-and-aggregate Network (DaNet) to address these issues. DaNet includes three new designs, namely UVI guided learning, decomposition for fine-grained perception, and aggregation for robust prediction. First, we adopt the UVI maps, which densely build a bridge between 2D pixels and 3D vertexes, as an intermediate representation to facilitate the learning of image-to-model mapping. Second, we decompose the prediction task into one global stream and multiple local streams so that the network not only provides global perception for the camera and shape prediction, but also has detailed perception for part pose prediction. Lastly, we aggregate the message from local streams to enhance the robustness of part pose prediction, where a position-aided rotation feature refinement strategy is proposed to exploit the spatial relationship between body parts. Such a refinement strategy is more efficient since the correlations between position features are stronger than that in the original rotation feature space. The effectiveness of our method is validated on the Human3.6M and UP-3D datasets. Experimental results show that the proposed method significantly improves the reconstruction performance in comparison with previous state-of-the-art methods. Our code is publicly available at https://github.com/HongwenZhang/DaNet-3DHumanReconstrution .

Skeleton-Based Spatio-Temporal U-Network for 3D Human Pose Estimation in Video

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

3D Human pose estimation from video via multi-scale multi-level spatial temporal features

3D-UGCN: A Unified Graph Convolutional Network for Robust 3D Human Pose Estimation from Monocular RGB Images

Live Stream Temporally Embedded 3D Human Body Pose and Shape Estimation

3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training

Spatiotemporal Neural Network for Video-Based Pose Estimation

Hierarchical Graph Networks for 3D Human Pose Estimation

Unsupervised Universal Hierarchical Multi-Person 3D Pose Estimation for Natural Scenes

ST-UNet: A Spatio-Temporal U-Network for Graph-structured Time Series Modeling

Simplified-attention Enhanced Graph Convolutional Network for 3D human pose estimation

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

STGFormer: Spatio-Temporal GraphFormer for 3D Human Pose Estimation in Video

Graph Stacked Hourglass Networks for 3D Human Pose Estimation

Graph U-Shaped Network with Mapping-Aware Local Enhancement for Single-Frame 3D Human Pose Estimation

Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

Conditional Directed Graph Convolution for 3D Human Pose Estimation

Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representation

DaNet: Decompose-and-aggregate Network for 3D Human Shape and Pose Estimation

Skeleton Cluster Tracking for robust multi-view multi-person 3D human pose estimation

Spatial-Temporal Graph U-Net for Skeleton-Based Human Motion Infilling