Abstract:Human pose and shape estimation (HPS) has attracted increasing attention in recent years. While most existing studies focus on HPS from 2D images or videos with inherent depth ambiguity, there are surging need to investigate HPS from 3D point clouds as depth sensors have been frequently employed in commercial devices. However, real-world sensory 3D points are usually noisy and incomplete, and also human bodies could have different poses of high diversity. To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture. Specifically, each stage of PointHPS performs a series of downsampling and upsampling operations to extract and collate both local and global cues, which are further enhanced by two novel modules: 1) Cross-stage Feature Fusion (CFF) for multi-scale feature propagation that allows information to flow effectively through the stages, and 2) Intermediate Feature Enhancement (IFE) for body-aware feature aggregation that improves feature quality after each stage. To facilitate a comprehensive study under various scenarios, we conduct our experiments on two large-scale benchmarks, comprising i) a dataset that features diverse subjects and actions captured by real commercial sensors in a laboratory environment, and ii) controlled synthetic data generated with realistic considerations such as clothed humans in crowded outdoor scenes. Extensive experiments demonstrate that PointHPS, with its powerful point feature extraction and processing scheme, outperforms State-of-the-Art methods by significant margins across the board. Homepage: <a class="link-external link-https" href="https://caizhongang.github.io/projects/PointHPS/" rel="external noopener nofollow">this https URL</a>.

3D Human Posture Segmentation by Spectral Clustering with Surface Normal Constraint

Human Posture Segmentation by Constrained Spectral Clustering

Healthy Human Sitting Posture Estimation in RGB-D Scenes Using Object Context

Based on network probabilistic graph human pose segmentation algorithm

Mpose: Environment- and Subject-Agnostic 3D Skeleton Posture Reconstruction Leveraging a Single Mmwave Device

Symmetry-Aware Semantic Segmentation of a 3D Human Body Model

A Semi-supervised Clustering-based Segmentation Algorithm of 3D Reconstructed Human Body Parts

Sequential 3D Human Pose and Shape Estimation from Point Clouds

Real-Time Human Segmentation Using Pose Skeleton Map

Spectral 3D Mesh Segmentation with a Novel Single Segmentation Field.

Surface Extraction and Boundary Detection Based on DBSCAN Clustering in 3D Point Clouds

Unsupervised Universal Hierarchical Multi-Person 3D Pose Estimation for Natural Scenes

Skeleton Cluster Tracking for robust multi-view multi-person 3D human pose estimation

Biview learning for human posture segmentation from 3D points cloud

A Hierarchical Algorithm for Human Posture Recognition Based on Spatial and Frequency Domain Features

Automatic segmentation of 3D point cloud using spectral clustering

PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds

Resolving 3D Human Pose Ambiguities with 3D Scene Constraints

Multi-Branch High-Dimensional Guided Transformer-Based 3D Human Posture Estimation

Self-supervised 3D Human Mesh Recovery from Noisy Point Clouds

Topological features detection and automatic segmentation of point clouds from human body surface