Abstract:Neural radiance fields (NeRF) synthesize realistic novel views by estimating point attributes (density and color), followed by the volume rendering method. However, accurately predicting the arbitrary point attributes poses a challenge for the single NeRF-based model. Such limitation directly impacts the quality of novel view synthesis. To address this problem, a collaborative strategy with multiple NeRF-based models is proposed. This strategy is the first to introduce a multi-model cascaded architecture into NeRF for achieving high-quality novel view synthesis. Its purpose is to utilize a cascading architecture in space for the progressive enhancement of point attribute accuracy. The cascading architecture includes point adjustment and snapshots fusion. Specifically, point adjustment leverages a pretrained NeRF-based model to predict the initial density and color of each point in space. This step affords an initial rendering of target scene. Then, these initial density and color of points are directly transferred to the subsequent NeRF-based model. This process guides the subsequent NeRF-based model to focus on the refinement of initial point attributes and synthesize more realistic novel views. Finally, snapshots fusion fuses outputs (referred as snapshots) from multiple parallel subsequent NeRF-based models to synthesize the ultimate high-quality novel views. The proposed strategy is tested with a range of established NeRF-based methods, such as NeRF, Instant-NGP, and TensoRF. Experimental data for this research are sourced from the realistic 360 synthetic dataset and the LLFF dataset. Results indicate that the proposed collaborative strategy with established NeRF-based methods can improve the quality of novel view synthesis, surpassing the corresponding single model. Our project page is available at https://github.com/ZhenyangLiu/Collaborative-Neural-Radiance-Fields-for-Novel-View-Synthesis.

NeRFE: Free View Synthesis for Event Data

NeRFVS: Neural Radiance Fields for Free View Synthesis Via Geometry Scaffolds

Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Stream

Structure-aware neural radiance fields without posed camera

Improving NeRF Quality by Progressive Camera Placement for Unrestricted Navigation in Complex Environments

wildNeRF: Complete view synthesis of in-the-wild dynamic scenes captured using sparse monocular data

SN 2 eRF: A Framework for Neural Radiance Fields given Sparse and Noisy Poses

Multi-View Stereo and Depth Priors Guided NeRF for View Synthesis.

NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field.

Deblur e-NeRF: NeRF from Motion-Blurred Events under High-speed or Low-light Conditions

Collaborative neural radiance fields for novel view synthesis

EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields

StructNeRF: Neural Radiance Fields for Indoor Scenes With Structural Hints

NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review

Depth-supervised NeRF: Fewer Views and Faster Training for Free

NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

Dense Depth Priors for Neural Radiance Fields from Sparse Input Views

Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion

Enhancing View Synthesis with Depth-Guided Neural Radiance Fields and Improved Depth Completion