Abstract:Reconstructing a 3D human body mesh from a monocular image is a challenging inverse problem because of occlusion and complicated human articulations. Recent deep learning-based methods have made significant progress in single-image human reconstruction. Most of these works are either model-based methods, which estimate the sparse parameters of a statistical human body model, or model-free methods, which directly recover the vertices' coordinates of a predefined mesh. However, model-based methods always suffer detail losses due to the limited parameter space, and model-free methods are hard to directly recover satisfactory results from images due to the use of a shared global feature for all vertices and the domain gap between 2D regular images and 3D irregular meshes. To resolve these issues, we propose a hybrid model, which combines the advantages of both model-based approach and model-free approach to estimate a 3D human mesh in a coarse-to-fine manner. Initially, we utilize a convolutional neural network (CNN) to estimate the parameters of a Skinned Multi-Person Linear Model (SMPL), which allows us to generate a coarse human mesh. After that, the vertex coordinates of the coarse human mesh are further refined by a graph convolutional neural network (GCN). Unlike previous GCN-based methods, whose vertex coordinates are recovered from a shared global feature, we propose a LOcal CorRespondence-Aware (LOCRA) module to extract local special features for each vertex. To make the local features related to the human pose, we also add a keypoint-related loss to supervise the training process of the LOCRA module. Experiments demonstrate that our hybrid model with the LOCRA module outperforms existing methods on multiple public benchmarks. Our code will be publicly available.

Matchable Image Retrieval by Learning from Surface Reconstruction

Domain-Specific Modeling and Semantic Alignment for Image-Based 3d Model Retrieval

Source-enhanced Prototypical Alignment for Single Image 3D Model Retrieval

3D Point Cloud Generation Reconstruction from Single Image Based on Image Retrieval

3D Reconstruction and Semantic Segmentation Method Combining PointNet and 3D-Lmnet from Single Image

3D-Isrnet

Mismatched: Evaluating the Limits of Image Matching Approaches and Benchmarks

Monocular Image-Based 3-D Model Retrieval: A Benchmark

Robust Image Matching with Cascaded Outliers Removal

A Local Correspondence-aware Hybrid CNN-GCN Model for Single-image Human Body Reconstruction

Multiple Discrimination and Pairwise CNN for view-based 3D object retrieval

Where to Focus: Query Adaptive Matching for Instance Retrieval Using Convolutional Feature Maps.

Feature Reconstruction by Laplacian Eigenmaps for Efficient Instance Search.

Category-Sensitive Incremental Learning for Image-Based 3D Shape Reconstruction

A Point Matching Strategy of 3D Loss Function for Single RGB Images Deep Mesh Reconstruction

A Coarse-to-Fine Transformer-Based Network for 3D Reconstruction from Non-Overlapping Multi-View Images

LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image

Deep Learning Using Isotroping, Laplacing, Eigenvalues Interpolative Binding, and Convolved Determinants with Normed Mapping for Large-Scale Image Retrieval

Shape-aware Speckle Matching Network for Cross-Domain 3D Reconstruction

Efficient match pair selection for matching large-scale oblique UAV images using spatial priors

A Matching Optimization Algorithm about Low-Altitude Remote Sensing Images Based on Geometrical Constraint and Convolutional Neural Network