Abstract:Single-view 3D object reconstruction is a fundamental and challenging computer vision task that aims at recovering 3D shapes from single-view RGB images. Most existing deep learning based reconstruction methods are trained and evaluated on the same categories, and they cannot work well when handling objects from novel categories that are not seen during training. Focusing on this issue, this paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories and encourage models to reconstruct objects literally. Specifically, we propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction. Firstly, we factorize the complicated image-to-mesh mapping into two simpler mappings, i.e., image-to-point mapping and point-to-mesh mapping, while the latter is mainly a geometric problem and less dependent on object categories. Secondly, we devise a local feature sampling strategy in 2D and 3D feature spaces to capture the local geometry shared across objects to enhance model generalization. Thirdly, apart from the traditional point-to-point supervision, we introduce a multi-view silhouette loss to supervise the surface generation process, which provides additional regularization and further relieves the overfitting problem. The experimental results show that our method significantly outperforms the existing works on the ShapeNet and Pix3D under different scenarios and various metrics, especially for novel objects. The project link is <a class="link-external link-https" href="https://github.com/Wi-sc/GenMesh" rel="external noopener nofollow">this https URL</a>.

Research on 3D Reconstruction of Furniture Based on Differentiable Renderer

Multiview Textured Mesh Recovery by Differentiable Rendering

Monocular Human Pose and Shape Reconstruction Using Part Differentiable Rendering

MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Model

Online Global Non-rigid Registration for 3D Object Reconstruction Using Consumer-level Depth Cameras

Three-Dimensional Reconstruction of Indoor Scenes Based on Implicit Neural Representation

Single-view 3D Mesh Reconstruction for Seen and Unseen Categories

Differentiable Rendering Approach to Mesh Optimization for Digital Human Reconstruction.

High-Quality Textured 3D Shape Reconstruction with Cascaded Fully Convolutional Networks

Deformable Model-Driven Neural Rendering for High-Fidelity 3D Reconstruction of Human Heads Under Low-View Settings

Self-Supervised 3D Mesh Reconstruction from Single Images

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization

A General Differentiable Mesh Renderer for Image-Based 3D Reasoning

2L3: Lifting Imperfect Generated 2D Images into Accurate 3D

CD$^2$: Fine-grained 3D Mesh Reconstruction With Twice Chamfer Distance

An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering

A Review of Deep Learning-Powered Mesh Reconstruction Methods

$R^2$-Mesh: Reinforcement Learning Powered Mesh Reconstruction via Geometry and Appearance Refinement

Improving Neural Indoor Surface Reconstruction with Mask-Guided Adaptive Consistency Constraints

Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction

Deep Hybrid Self-Prior for Full 3D Mesh Generation