Abstract:High-quality surface reconstruction and texture reconstruction of large-scale scene play a pivotal role in the domains of ancient architecture, cultural heritage preservation, and 3D urban modeling. Given that point cloud data has emerged as a crucial medium for representing three-dimensional spatial information, its utilization for surface and texture reconstruction becomes indispensable. In this research, we propose a novel framework for reconstructing surface and texture from unstructured colorful point clouds without normal information to restore large-scale real-world scenes. Specifically, we first introduce an automatic virtual viewpoint selection method to generate virtual views by rendering the point cloud from multiple viewpoints. Subsequently, we construct a two-step network to facilitate accurate visibility prediction and texture inpainting for each virtual view. Then, the visibility information from multiple perspectives is integrated to solve an optimization problem incorporating visibility constraints, resulting in the generation of a 3D mesh. Subsequently, texture information from various perspectives is integrated, filtering techniques are applied to determine the optimal perspective for texturing, and a texture atlas is generated. Precise texture mapping is then performed, ultimately leading to the production of a comprehensive textured model. In contrast to alternative learning-based methodologies, our framework exclusively learns from two-dimensional images, encompassing the prediction of both visible and invisible points as well as the execution of image inpainting tasks. This approach exhibits exceptional versatility in managing large-scale point clouds while effectively leveraging the color and intensity attributes of the data for precise texture reconstruction. The experimental results demonstrate that our approach achieves a significant improvement of 2.06% in F-scores for outdoor surface reconstruction compared to the current state-of-the-art learning-based methods, while also outperforming them in texture reconstruction.

Multiview Textured Mesh Recovery by Differentiable Rendering

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Differentiable Rendering Approach to Mesh Optimization for Digital Human Reconstruction.

Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction

Multi-View Stereo Representation Revist: Region-Aware MVSNet

Efficient Multi-View Inverse Rendering Using a Hybrid Differentiable Rendering Method

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

TexPainter: Generative Mesh Texturing with Multi-view Consistency

Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering

Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering

ColorMesh: Surface and Texture Reconstruction of Large-Scale Scenes from Unstructured Colorful Point Clouds with Adaptive Automatic Viewpoint Selection

A General Differentiable Mesh Renderer for Image-Based 3D Reasoning

Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering

Multi-view self-supervised learning for 3D facial texture reconstruction from single image

MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction

Explicit 3D Reconstruction from Images with Dynamic Graph Learning and Rendering-Guided Diffusion

Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture

A Progressive Quadric Graph Convolutional Network for 3D Human Mesh Recovery

Learning Topology Uniformed Face Mesh by Volume Rendering for Multi-view Reconstruction

An Optimization Framework to Enforce Multi-View Consistency for Texturing 3D Meshes

NIST: Learning Neural Implicit Surfaces and Textures for Multi-view Reconstruction