Abstract:Human pose and shape estimation (HPS) has attracted increasing attention in recent years. While most existing studies focus on HPS from 2D images or videos with inherent depth ambiguity, there are surging need to investigate HPS from 3D point clouds as depth sensors have been frequently employed in commercial devices. However, real-world sensory 3D points are usually noisy and incomplete, and also human bodies could have different poses of high diversity. To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture. Specifically, each stage of PointHPS performs a series of downsampling and upsampling operations to extract and collate both local and global cues, which are further enhanced by two novel modules: 1) Cross-stage Feature Fusion (CFF) for multi-scale feature propagation that allows information to flow effectively through the stages, and 2) Intermediate Feature Enhancement (IFE) for body-aware feature aggregation that improves feature quality after each stage. To facilitate a comprehensive study under various scenarios, we conduct our experiments on two large-scale benchmarks, comprising i) a dataset that features diverse subjects and actions captured by real commercial sensors in a laboratory environment, and ii) controlled synthetic data generated with realistic considerations such as clothed humans in crowded outdoor scenes. Extensive experiments demonstrate that PointHPS, with its powerful point feature extraction and processing scheme, outperforms State-of-the-Art methods by significant margins across the board. Homepage: <a class="link-external link-https" href="https://caizhongang.github.io/projects/PointHPS/" rel="external noopener nofollow">this https URL</a>.

Topological features detection and automatic segmentation of point clouds from human body surface

3D Human Landmark Detection and Segmentation with Semantic Knowledge

Automatic Measurement of Scanned Human Body in Fixed Posture

High-precision Human Body Acquisition Via Multi-View Binocular Stereopsis

Surface Extraction and Boundary Detection Based on DBSCAN Clustering in 3D Point Clouds

Human Articulated Body Recognition Method in High-Resolution Monitoring Images

Automatic Location and Semantic Labeling of Landmarks on 3D Human Body Models

FAST SEGMENTATION OF HUMAN BODY COMPONENTS BASED ON FACE DETECTION AND KEY POINT RECOGNITION

Automatic landmark annotation and dense correspondence registration for 3D human facial images

Sequential 3D Human Pose and Shape Estimation from Point Clouds

Automatic Registration of Panoramic Images and Point Clouds in Urban Large Scenes Based on Line Features

Research on filtering and measurement algorithms based on human point cloud data

Segmentation for Point Cloud of Auto Body Surface Reconstruction

Convolutional Neural Network Powered Identification of the Location and Orientation of Human Body Via Human Form Point Cloud

PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds

Study on the rotary human body scanning system based on multiple Kinects

AUTOMATIC REGISTRATION OF TERRESTRIAL POINT CLOUD USING PANORAMIC REFLECTANCE IMAGES

3D Human Posture Segmentation by Spectral Clustering with Surface Normal Constraint

Automatic Feature-Based Point Cloud Alignment and Inspection

Human Posture Segmentation by Constrained Spectral Clustering

3D Landmark Detection on Human Point Clouds: A Benchmark and A Dual Cascade Point Transformer Framework