Abstract:Neural radiance fields (NeRF) shows powerful performance in novel view synthesis and 3D geometry reconstruction, but it suffers from critical performance degradation when the number of known viewpoints is drastically reduced. Existing works attempt to overcome this problem by employing external priors, but their success is limited to certain types of scenes or datasets. Employing monocular depth estimation (MDE) networks, pretrained on large-scale RGB-D datasets, with powerful generalization capability would be a key to solving this problem: however, using MDE in conjunction with NeRF comes with a new set of challenges due to various ambiguity problems exhibited by monocular depths. In this light, we propose a novel framework, dubbed DäRF, that achieves robust NeRF reconstruction with a handful of real-world images by combining the strengths of NeRF and monocular depth estimation through online complementary training. Our framework imposes the MDE network's powerful geometry prior to NeRF representation at both seen and unseen viewpoints to enhance its robustness and coherence. In addition, we overcome the ambiguity problems of monocular depths through patch-wise scale-shift fitting and geometry distillation, which adapts the MDE network to produce depths aligned accurately with NeRF geometry. Experiments show our framework achieves state-of-the-art results both quantitatively and qualitatively, demonstrating consistent and reliable performance in both indoor and outdoor real-world datasets. Project page is available at <a class="link-external link-https" href="https://ku-cvlab.github.io/DaRF/" rel="external noopener nofollow">this https URL</a>.

HDPNERF: Hybrid Depth Priors for Neural Radiance Fields from Sparse Input Views

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

Enhancing Neural Radiance Fields with Depth and Normal Completion Priors from Sparse Views

HG3-NeRF: Hierarchical Geometric, Semantic, and Photometric Guided Neural Radiance Fields for Sparse View Inputs

Depth-Guided Optimization of Neural Radiance Fields for Indoor Multi-View Stereo

Dense Depth Priors for Neural Radiance Fields from Sparse Input Views

Enhancing View Synthesis with Depth-Guided Neural Radiance Fields and Improved Depth Completion

SN 2 eRF: A Framework for Neural Radiance Fields given Sparse and Noisy Poses

Depth-supervised NeRF: Fewer Views and Faster Training for Free

MonoPatchNeRF: Improving Neural Radiance Fields with Patch-based Monocular Guidance

Digging into Depth Priors for Outdoor Neural Radiance Fields

Single-view Neural Radiance Fields with Depth Teacher

Prior-Driven NeRF: Prior Guided Rendering

Depth assisted novel view synthesis using few images

AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization

SG-NeRF: Sparse-Input Generalized Neural Radiance Fields for Novel View Synthesis

MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields

Improved Neural Radiance Fields Using Pseudo-depth and Fusion

DaRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation

TD-NeRF: Novel Truncated Depth Prior for Joint Camera Pose and Neural Radiance Field Optimization