Abstract:In this paper we present a novel method to estimate 3D human pose and shape from monocular videos. This task requires directly recovering pixel-alignment 3D human pose and body shape from monocular images or videos, which is challenging due to its inherent ambiguity. To improve precision, existing methods highly rely on the initialized mean pose and shape as prior estimates and parameter regression with an iterative error feedback manner. In addition, video-based approaches model the overall change over the image-level features to temporally enhance the single-frame feature, but fail to capture the rotational motion at the joint level, and cannot guarantee local temporal consistency. To address these issues, we propose a novel Transformer-based model with a design of independent tokens. First, we introduce three types of tokens independent of the image feature: \textit{joint rotation tokens, shape token, and camera token}. By progressively interacting with image features through Transformer layers, these tokens learn to encode the prior knowledge of human 3D joint rotations, body shape, and position information from large-scale data, and are updated to estimate SMPL parameters conditioned on a given image. Second, benefiting from the proposed token-based representation, we further use a temporal model to focus on capturing the rotational temporal information of each joint, which is empirically conducive to preventing large jitters in local parts. Despite being conceptually simple, the proposed method attains superior performances on the 3DPW and Human3.6M datasets. Using ResNet-50 and Transformer architectures, it obtains 42.0 mm error on the PA-MPJPE metric of the challenging 3DPW, outperforming state-of-the-art counterparts by a large margin. Code will be publicly available at <a class="link-external link-https" href="https://github.com/yangsenius/INT_HMR_Model" rel="external noopener nofollow">this https URL</a>

TokenPose: Learning Keypoint Tokens for Human Pose Estimation

Human Pose as Compositional Tokens

Improved TokenPose with Sparsity

TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation

LMFormer: Lightweight and Multi-Feature Perspective Via Transformer for Human Pose Estimation

GTPT: Group-based Token Pruning Transformer for Efficient Human Pose Estimation

Multi-person pose estimation using atrous convolution

A Multi-Channel Parallel Keypoint Fusion Framework for Human Pose Estimation

Multi-Human Pose Estimation by Deep Learning-Based Sequential Approach for Human Keypoint Position and Human Body Detection

Capturing the motion of every joint: 3D human pose and shape estimation with independent tokens

Pose-native Network Architecture Search for Multi-person Human Pose Estimation

SSpose: Self-supervised Spatial-aware Model for Human Pose Estimation

Learning Delicate Local Representations for Multi-person Pose Estimation

Adaptive Positive Sample Selection and Dynamic Soft Label Assignment for Keypoint Detection

Simple and Lightweight Human Pose Estimation

Distilling Token-Pruned Pose Transformer for 2D Human Pose Estimation

DSPose: Dual-Space-Driven Keypoint Topology Modeling for Human Pose Estimation

Human Pose Estimation Based on Lightweight Multi-Scale Coordinate Attention

SD-Pose: facilitating space-decoupled human pose estimation via adaptive pose perception guidance

PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimation

KSL-POSE: A Real-Time 2D Human Pose Estimation Method Based on Modified YOLOv8-Pose Framework