Abstract:Neural Radiance Field (NeRF) significantly degrades when only a limited number of views are available. To complement the lack of 3D information, depth-based models, such as DSNeRF and MonoSDF, explicitly assume the availability of accurate depth maps of multiple views. They linearly scale the accurate depth maps as supervision to guide the predicted depth of few-shot NeRFs. However, accurate depth maps are difficult and expensive to capture due to wide-range depth distances in the wild. In this work, we present a new Sparse-view NeRF (SparseNeRF) framework that exploits depth priors from real-world inaccurate observations. The inaccurate depth observations are either from pre-trained depth models or coarse depth maps of consumer-level depth sensors. Since coarse depth maps are not strictly scaled to the ground-truth depth maps, we propose a simple yet effective constraint, a local depth ranking method, on NeRFs such that the expected depth ranking of the NeRF is consistent with that of the coarse depth maps in local patches. To preserve the spatial continuity of the estimated depth of NeRF, we further propose a spatial continuity constraint to encourage the consistency of the expected depth continuity of NeRF with coarse depth maps. Surprisingly, with simple depth ranking constraints, SparseNeRF outperforms all state-of-the-art few-shot NeRF methods (including depth-based models) on standard LLFF and DTU datasets. Moreover, we collect a new dataset NVS-RGBD that contains real-world depth maps from Azure Kinect, ZED 2, and iPhone 13 Pro. Extensive experiments on NVS-RGBD dataset also validate the superiority and generalizability of SparseNeRF. Project page is available at https://sparsenerf.github.io/.

DSSMNeRF: Depth Self-supervised MVS NeRF

Depth-supervised NeRF: Fewer Views and Faster Training for Free

RDNeRF: Relative Depth Guided NeRF for Dense Free View Synthesis

Multi-View Stereo and Depth Priors Guided NeRF for View Synthesis.

HG3-NeRF: Hierarchical Geometric, Semantic, and Photometric Guided Neural Radiance Fields for Sparse View Inputs

DaRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation

StructNeRF: Neural Radiance Fields for Indoor Scenes With Structural Hints

Improving Neural Radiance Fields with Depth-aware Optimization for Novel View Synthesis

Enhancing View Synthesis with Depth-Guided Neural Radiance Fields and Improved Depth Completion

MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

DSEM-NeRF: Multimodal feature fusion and global-local attention for enhanced 3D scene reconstruction

DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features

SSNeRF: Sparse View Semi-supervised Neural Radiance Fields with Augmentation

NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes

NeRF-SDP: Efficient Generalizable Neural Radiance Field with Scene Depth Perception.

BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenes

SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis

AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization

MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

S$^3$-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint

SC-NeRF: Self-Correcting Neural Radiance Field with Sparse Views