Abstract:Although existing video-based 3D human mesh recovery methods have made significant progress, simultaneously estimating human pose and shape from low-resolution image features limits their performance. These image features lack sufficient spatial information about the human body and contain various noises (e.g., background, lighting, and clothing), which often results in inaccurate pose and inconsistent motion. Inspired by the rapid advance in human pose estimation, we discover that compared to image features, skeletons inherently contain accurate human pose and motion. Therefore, we propose a novel semiAnalytical Regressor using disenTangled Skeletal representations for human mesh recovery from videos, called ARTS. Specifically, a skeleton estimation and disentanglement module is proposed to estimate the 3D skeletons from a video and decouple them into disentangled skeletal representations (i.e., joint position, bone length, and human motion). Then, to fully utilize these representations, we introduce a semi-analytical regressor to estimate the parameters of the human mesh model. The regressor consists of three modules: Temporal Inverse Kinematics (TIK), Bone-guided Shape Fitting (BSF), and Motion-Centric Refinement (MCR). TIK utilizes joint position to estimate initial pose parameters and BSF leverages bone length to regress bone-aligned shape parameters. Finally, MCR combines human motion representation with image features to refine the initial human model parameters. Extensive experiments demonstrate that our ARTS surpasses existing state-of-the-art video-based methods in both per-frame accuracy and temporal consistency on popular benchmarks: 3DPW, MPI-INF-3DHP, and Human3.6M. Code is available at <a class="link-external link-https" href="https://github.com/TangTao-PKU/ARTS" rel="external noopener nofollow">this https URL</a>.

Adaptive Human Silhouette Reconstruction Based on the Exploration of Temporal Information

Dynamic Spatio-Temporal Modeling for Example-Based Human Silhouette Recovery

Temporal Constrained Feasible Subspace Learning for Human Pose Forecasting

Dynamic Human Body Reconstruction and Motion Tracking with Low-Cost Depth Cameras

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

Parametric Human Shape Reconstruction Via Bidirectional Silhouette Guidance

A Spatial-temporal 3D Human Pose Reconstruction Framework

3D Human Pose and Shape Reconstruction from Videos Via Confidence-Aware Temporal Feature Aggregation

Real-time tracking and pose-estimation of walking human from silhouette images

SCAPE-based Human Performance Reconstruction

Reconstruction of 3D Human Motion Using Multiple View L1 Trackers

3D Human Reconstruction from A Single Depth Image

Silhouette Representation and Matching for 3d Pose Discrimination - A Comparative Study

Learning Temporal-Spatial Contextual Adaptation for Three-Dimensional Human Pose Estimation

Towards robust 3d reconstruction of human motion from monocular video

Spatio-temporal Tendency Reasoning for Human Body Pose and Shape Estimation from Videos

Learning Pose Controllable Human Reconstruction with Dynamic Implicit Fields from a Single Image

ARTS: Semi-Analytical Regressor using Disentangled Skeletal Representations for Human Mesh Recovery from Videos

ProGraph: Temporally-alignable Probability Guided Graph Topological Modeling for 3D Human Reconstruction

A 3D 2-subiteration thinning algorithm for human pose estimation