Abstract:In this paper we present a novel method to estimate 3D human pose and shape from monocular videos. This task requires directly recovering pixel-alignment 3D human pose and body shape from monocular images or videos, which is challenging due to its inherent ambiguity. To improve precision, existing methods highly rely on the initialized mean pose and shape as prior estimates and parameter regression with an iterative error feedback manner. In addition, video-based approaches model the overall change over the image-level features to temporally enhance the single-frame feature, but fail to capture the rotational motion at the joint level, and cannot guarantee local temporal consistency. To address these issues, we propose a novel Transformer-based model with a design of independent tokens. First, we introduce three types of tokens independent of the image feature: \textit{joint rotation tokens, shape token, and camera token}. By progressively interacting with image features through Transformer layers, these tokens learn to encode the prior knowledge of human 3D joint rotations, body shape, and position information from large-scale data, and are updated to estimate SMPL parameters conditioned on a given image. Second, benefiting from the proposed token-based representation, we further use a temporal model to focus on capturing the rotational temporal information of each joint, which is empirically conducive to preventing large jitters in local parts. Despite being conceptually simple, the proposed method attains superior performances on the 3DPW and Human3.6M datasets. Using ResNet-50 and Transformer architectures, it obtains 42.0 mm error on the PA-MPJPE metric of the challenging 3DPW, outperforming state-of-the-art counterparts by a large margin. Code will be publicly available at <a class="link-external link-https" href="https://github.com/yangsenius/INT_HMR_Model" rel="external noopener nofollow">this https URL</a>

Parallel‐branch Network for 3D Human Pose and Shape Estimation in Video

Live Stream Temporally Embedded 3D Human Body Pose and Shape Estimation

3D Human pose estimation from video via multi-scale multi-level spatial temporal features

Towards Accurate Markerless Human Shape and Pose Estimation over Time

LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

Multi-Branch High-Dimensional Guided Transformer-Based 3D Human Posture Estimation

SPGformer: Serial-Parallel Hybrid GCN-Transformer with Graph-Oriented Encoder for 2D-to-3d Human Pose Estimation

SPGformer: Serial–Parallel Hybrid GCN-Transformer With Graph-Oriented Encoder for 2-D-to-3-D Human Pose Estimation

Hierarchical Neural Network for 3d Human Pose and Shape Estimation with Deep Shape Optimizer As Supervisor

Learnable human mesh triangulation for 3D human pose and shape estimation

3D Human Pose Estimation Via Human Structure-Aware Fully Connected Network

Prior-knowledge-based Self-Attention Network for 3D Human Pose Estimation

Towards Locality Similarity Preserving to 3D Human Pose Estimation.

Estimating 3D body mesh without SMPL annotations via alternating successive convex approximation

3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training

Vertex Position Estimation with Spatial–temporal Transformer for 3D Human Reconstruction

Joint Multi-Scale Transformers and Pose Equivalence Constraints for 3D Human Pose Estimation

Capturing the motion of every joint: 3D human pose and shape estimation with independent tokens

Skeleton-Aware 3d Human Shape Reconstruction From Point Clouds

Skeleton Cluster Tracking for robust multi-view multi-person 3D human pose estimation

Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video