Abstract:Neural Radiance Fields (NeRF) are an advanced technology that creates highly realistic images by learning about scenes through a neural network model. However, NeRF often encounters issues when there are not enough images to work with, leading to problems in accurately rendering views. The main issue is that NeRF lacks sufficient structural details to guide the rendering process accurately. To address this, we proposed a Depth and Normal Dense Completion Priors for NeRF (CP\_NeRF) framework. This framework enhances view rendering by adding depth and normal dense completion priors to the NeRF optimization process. Before optimizing NeRF, we obtain sparse depth maps using the Structure from Motion (SfM) technique used to get camera poses. Based on the sparse depth maps and a normal estimator, we generate sparse normal maps for training a normal completion prior with precise standard deviations. During optimization, we apply depth and normal completion priors to transform sparse data into dense depth and normal maps with their standard deviations. We use these dense maps to guide ray sampling, assist distance sampling and construct a normal loss function for better training accuracy. To improve the rendering of NeRF's normal outputs, we incorporate an optical centre position embedder that helps synthesize more accurate normals through volume rendering. Additionally, we employ a normal patch matching technique to choose accurate rendered normal maps, ensuring more precise supervision for the model. Our method is superior to leading techniques in rendering detailed indoor scenes, even with limited input views.

Generalizable Neural Radiance Field with Hierarchical Geometry Constraint.

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

VDN-NeRF: Resolving Shape-Radiance Ambiguity Via View-Dependence Normalization

MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

Cascaded and Generalizable Neural Radiance Fields for Fast View Synthesis

Geometry-aware Reconstruction and Fusion-refined Rendering for Generalizable Neural Radiance Fields

HG3-NeRF: Hierarchical Geometric, Semantic, and Photometric Guided Neural Radiance Fields for Sparse View Inputs

Multi-View Stereo and Depth Priors Guided NeRF for View Synthesis.

SG-NeRF: Sparse-Input Generalized Neural Radiance Fields for Novel View Synthesis

NeRF-SDP: Efficient Generalizable Neural Radiance Field with Scene Depth Perception.

Improved Neural Radiance Fields Using Pseudo-depth and Fusion

GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Images

MM-NeRF: Large-Scale Scene Representation with Multi-Resolution Hash Grid and Multi-View Priors Features

GSNeRF: Generalizable Semantic Neural Radiance Fields with Enhanced 3D Scene Understanding

Enhancing Neural Radiance Fields with Depth and Normal Completion Priors from Sparse Views

Efficient Neural Radiance Fields with Learned Depth-Guided Sampling

Mask-Based Modeling for Neural Radiance Fields

GARF:Geometry-Aware Generalized Neural Radiance Field

HDPNERF: Hybrid Depth Priors for Neural Radiance Fields from Sparse Input Views

MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenes