Abstract:Neural radiance fields (NeRF) shows powerful performance in novel view synthesis and 3D geometry reconstruction, but it suffers from critical performance degradation when the number of known viewpoints is drastically reduced. Existing works attempt to overcome this problem by employing external priors, but their success is limited to certain types of scenes or datasets. Employing monocular depth estimation (MDE) networks, pretrained on large-scale RGB-D datasets, with powerful generalization capability would be a key to solving this problem: however, using MDE in conjunction with NeRF comes with a new set of challenges due to various ambiguity problems exhibited by monocular depths. In this light, we propose a novel framework, dubbed DäRF, that achieves robust NeRF reconstruction with a handful of real-world images by combining the strengths of NeRF and monocular depth estimation through online complementary training. Our framework imposes the MDE network's powerful geometry prior to NeRF representation at both seen and unseen viewpoints to enhance its robustness and coherence. In addition, we overcome the ambiguity problems of monocular depths through patch-wise scale-shift fitting and geometry distillation, which adapts the MDE network to produce depths aligned accurately with NeRF geometry. Experiments show our framework achieves state-of-the-art results both quantitatively and qualitatively, demonstrating consistent and reliable performance in both indoor and outdoor real-world datasets. Project page is available at <a class="link-external link-https" href="https://ku-cvlab.github.io/DaRF/" rel="external noopener nofollow">this https URL</a>.

FDC-NeRF: Learning Pose-Free Neural Radiance Fields with Flow-Depth Consistency.

Depth-supervised NeRF: Fewer Views and Faster Training for Free

Structure-aware neural radiance fields without posed camera

TD-NeRF: Novel Truncated Depth Prior for Joint Camera Pose and Neural Radiance Field Optimization

Pose-Free Neural Radiance Fields via Implicit Pose Regularization

AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization

CT-NeRF: Incremental Optimizing Neural Radiance Field and Poses with Complex Trajectory

UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields

IL-NeRF: Incremental Learning for Neural Radiance Fields with Camera Pose Alignment

SN 2 eRF: A Framework for Neural Radiance Fields given Sparse and Noisy Poses

DaRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation

SPARF: Neural Radiance Fields from Sparse and Noisy Poses

Enhancing Neural Radiance Fields with Depth and Normal Completion Priors from Sparse Views

Depth-guided NeRF Training via Earth Mover's Distance

CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Poses

CLA-NeRF: Category-Level Articulated Neural Radiance Field

FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editing

F$^{2}$-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories

KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter

PI-NeRF: A Partial-Invertible Neural Radiance Fields for Pose Estimation

FBINeRF: Feature-Based Integrated Recurrent Network for Pinhole and Fisheye Neural Radiance Fields