Abstract:We propose a novel explicit dense 3D reconstruction approach that processes a set of images of a scene with sensor poses and calibrations and estimates a photo-real digital model. One of the key innovations is that the underlying volumetric representation is completely explicit in contrast to neural network-based (implicit) alternatives. We encode scenes explicitly using clear and understandable mappings of optimization variables to scene geometry and their outgoing surface radiance. We represent them using hierarchical volumetric fields stored in a sparse voxel octree. Robustly reconstructing such a volumetric scene model with millions of unknown variables from registered scene images only is a highly non-convex and complex optimization problem. To this end, we employ stochastic gradient descent (Adam) which is steered by an inverse differentiable renderer. We demonstrate that our method can reconstruct models of high quality that are comparable to state-of-the-art implicit methods. Importantly, we do not use a sequential reconstruction pipeline where individual steps suffer from incomplete or unreliable information from previous stages, but start our optimizations from uniformed initial solutions with scene geometry and radiance that is far off from the ground truth. We show that our method is general and practical. It does not require a highly controlled lab setup for capturing, but allows for reconstructing scenes with a vast variety of objects, including challenging ones, such as outdoor plants or furry toys. Finally, our reconstructed scene models are versatile thanks to their explicit design. They can be edited interactively which is computationally too costly for implicit alternatives.

Generalized Scene Reconstruction

RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction

A New Era of Indoor Scene Reconstruction: A Survey

G3R: Gradient Guided Generalizable Reconstruction

GaRField++: Reinforced Gaussian Radiance Fields for Large-Scale 3D Scene Reconstruction

Recent Trends in 3D Reconstruction of General Non-Rigid Scenes

Scene reconstruction techniques for autonomous driving: a review of 3D Gaussian splatting

Radiometric Scene Decomposition: Scene Reflectance, Illumination, and Geometry from RGB-D Images

Temporally Coherent General Dynamic Scene Reconstruction

ERF: Explicit Radiance Field Reconstruction From Scratch

Gated Fields: Learning Scene Reconstruction from Gated Videos

Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View

Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

Gaussian Scenes: Pose-Free Sparse-View Scene Reconstruction using Depth-Enhanced Diffusion Priors

PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction

Image-based 3D scene reconstruction and exploration in augmented reality

A Refined 3D Gaussian Representation for High-Quality Dynamic Scene Reconstruction

Evaluating Modern Approaches in 3D Scene Reconstruction: NeRF vs Gaussian-Based Methods

High-resolution 3D reconstruction for complex color scenes with structured light

GS-IR: 3D Gaussian Splatting for Inverse Rendering

Optimizing 3D Gaussian Splatting for Sparse Viewpoint Scene Reconstruction