Abstract:Human mesh recovery (HMR) is crucial in many computer vision applications; from health to arts and entertainment. HMR from monocular images has predominantly been addressed by deterministic methods that output a single prediction for a given 2D image. However, HMR from a single image is an ill-posed problem due to depth ambiguity and occlusions. Probabilistic methods have attempted to address this by generating and fusing multiple plausible 3D reconstructions, but their performance has often lagged behind deterministic approaches. In this paper, we introduce GenHMR, a novel generative framework that reformulates monocular HMR as an image-conditioned generative task, explicitly modeling and mitigating uncertainties in the 2D-to-3D mapping process. GenHMR comprises two key components: (1) a pose tokenizer to convert 3D human poses into a sequence of discrete tokens in a latent space, and (2) an image-conditional masked transformer to learn the probabilistic distributions of the pose tokens, conditioned on the input image prompt along with randomly masked token sequence. During inference, the model samples from the learned conditional distribution to iteratively decode high-confidence pose tokens, thereby reducing 3D reconstruction uncertainties. To further refine the reconstruction, a 2D pose-guided refinement technique is proposed to directly fine-tune the decoded pose tokens in the latent space, which forces the projected 3D body mesh to align with the 2D pose clues. Experiments on benchmark datasets demonstrate that GenHMR significantly outperforms state-of-the-art methods. Project website can be found at <a class="link-external link-https" href="https://m-usamasaleem.github.io/publication/GenHMR/GenHMR.html" rel="external noopener nofollow">this https URL</a>

VoteHMR: Occlusion-Aware Voting Network for Robust 3D Human Mesh Recovery from Partial Point Clouds

Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution.

In-Hand 3D Object Reconstruction from a Monocular RGB Video

MH‐HMR: Human mesh recovery from monocular images via multi‐hypothesis learning

Visibility-Aware Human Mesh Recovery Via Balancing Dense Correspondence and Probability Model

MHPro: Multi-hypothesis Probabilistic Modeling for Human Mesh Recovery

Self-supervised 3D Human Mesh Recovery from Noisy Point Clouds

LiDAR-HMR: 3D Human Mesh Recovery from LiDAR

CenterHMR: Multi-Person Center-based Human Mesh Recovery

Human Mesh Recovery from Arbitrary Multi-view Images

End-to-end Recovery of Human Shape and Pose

Recovering 3D Human Mesh from Monocular Images: A Survey

Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representation

MMHMR: Generative Masked Modeling for Hand Mesh Recovery

GenHMR: Generative Human Mesh Recovery

GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras

Implicit 3D Human Mesh Recovery using Consistency with Pose and Shape from Unseen-view

Simultaneously Recovering Multi-Person Meshes and Multi-View Cameras with Human Semantics

Distribution and Depth-Aware Transformers for 3D Human Mesh Recovery

Skeleton-Aware 3d Human Shape Reconstruction From Point Clouds

Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Images