Abstract:Reconstructing a 3D human body mesh from a monocular image is a challenging inverse problem because of occlusion and complicated human articulations. Recent deep learning-based methods have made significant progress in single-image human reconstruction. Most of these works are either model-based methods, which estimate the sparse parameters of a statistical human body model, or model-free methods, which directly recover the vertices' coordinates of a predefined mesh. However, model-based methods always suffer detail losses due to the limited parameter space, and model-free methods are hard to directly recover satisfactory results from images due to the use of a shared global feature for all vertices and the domain gap between 2D regular images and 3D irregular meshes. To resolve these issues, we propose a hybrid model, which combines the advantages of both model-based approach and model-free approach to estimate a 3D human mesh in a coarse-to-fine manner. Initially, we utilize a convolutional neural network (CNN) to estimate the parameters of a Skinned Multi-Person Linear Model (SMPL), which allows us to generate a coarse human mesh. After that, the vertex coordinates of the coarse human mesh are further refined by a graph convolutional neural network (GCN). Unlike previous GCN-based methods, whose vertex coordinates are recovered from a shared global feature, we propose a LOcal CorRespondence-Aware (LOCRA) module to extract local special features for each vertex. To make the local features related to the human pose, we also add a keypoint-related loss to supervise the training process of the LOCRA module. Experiments demonstrate that our hybrid model with the LOCRA module outperforms existing methods on multiple public benchmarks. Our code will be publicly available.

Convolutional Neural Network Powered Identification of the Location and Orientation of Human Body Via Human Form Point Cloud

Modelling Human Body Pose for Action Recognition Using Deep Neural Networks

View Invariant Human Body Detection and Pose Estimation from Multiple Depth Sensors

Classroom Student Posture Recognition Based on an Improved High-Resolution Network.

A Local Correspondence-aware Hybrid CNN-GCN Model for Single-image Human Body Reconstruction

Motion Capture for Sporting Events Based on Graph Convolutional Neural Networks and Single Target Pose Estimation Algorithms

3D PostureNet: A unified framework for skeleton-based posture recognition

Sequential 3D Human Pose and Shape Estimation from Point Clouds

A Portable System for ToF Camera Based Human Body Detection and Pose Estimation

Center point to pose: Multiple views 3D human pose estimation for multi-person

Self-supervised 3D Patient Modeling with Multi-modal Attentive Fusion

Human Pose Estimation Based on Parallel Atrous Convolution and Body Structure Constraints

Human Body Orientation Estimation using Convolutional Neural Network

Locally Connected Network for Monocular 3D Human Pose Estimation

A Hybrid Approach for Cross-modality Pose Estimation Between Image and Point Cloud

PointSkelCNN: Deep Learning-Based 3D Human Skeleton Extraction from Point Clouds

Pose estimation of non-cooperative spacecraft based on Convelutional Neural Network

3D Joints Estimation of the Human Body in Single-Frame Point Cloud

Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision

MonoCap: Monocular Human Motion Capture Using a CNN Coupled with a Geometric Prior

A novel GCN-based point cloud classification model robust to pose variances