Abstract:Neural radiance fields (NeRF) synthesize realistic novel views by estimating point attributes (density and color), followed by the volume rendering method. However, accurately predicting the arbitrary point attributes poses a challenge for the single NeRF-based model. Such limitation directly impacts the quality of novel view synthesis. To address this problem, a collaborative strategy with multiple NeRF-based models is proposed. This strategy is the first to introduce a multi-model cascaded architecture into NeRF for achieving high-quality novel view synthesis. Its purpose is to utilize a cascading architecture in space for the progressive enhancement of point attribute accuracy. The cascading architecture includes point adjustment and snapshots fusion. Specifically, point adjustment leverages a pretrained NeRF-based model to predict the initial density and color of each point in space. This step affords an initial rendering of target scene. Then, these initial density and color of points are directly transferred to the subsequent NeRF-based model. This process guides the subsequent NeRF-based model to focus on the refinement of initial point attributes and synthesize more realistic novel views. Finally, snapshots fusion fuses outputs (referred as snapshots) from multiple parallel subsequent NeRF-based models to synthesize the ultimate high-quality novel views. The proposed strategy is tested with a range of established NeRF-based methods, such as NeRF, Instant-NGP, and TensoRF. Experimental data for this research are sourced from the realistic 360 synthetic dataset and the LLFF dataset. Results indicate that the proposed collaborative strategy with established NeRF-based methods can improve the quality of novel view synthesis, surpassing the corresponding single model. Our project page is available at https://github.com/ZhenyangLiu/Collaborative-Neural-Radiance-Fields-for-Novel-View-Synthesis.

Progressive Volume Distillation with Active Learning for Efficient NeRF Architecture Conversion

Enhance-NeRF: Multiple Performance Evaluation for Neural Radiance Fields

Divide and Conquer: Rethinking the Training Paradigm of Neural Radiance Fields

Adaptive Multi-NeRF: Exploit Efficient Parallelism in Adaptive Multiple Scale Neural Radiance Field Rendering

Volume Feature Rendering for Fast Neural Radiance Field Reconstruction

Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations

Omni-Recon: Harnessing Image-based Rendering for General-Purpose Neural Radiance Fields

Learning Neural Duplex Radiance Fields for Real-Time View Synthesis

Recursive-NeRF: An Efficient and Dynamically Growing NeRF

Where and How: Mitigating Confusion in Neural Radiance Fields from Sparse Inputs

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

Enhancing Neural Radiance Fields with Depth and Normal Completion Priors from Sparse Views

NeRF-SDP: Efficient Generalizable Neural Radiance Field with Scene Depth Perception.

Collaborative neural radiance fields for novel view synthesis

Taming Latent Diffusion Model for Neural Radiance Field Inpainting

Compressible-composable NeRF via Rank-residual Decomposition

FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models

NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning

Enhancing View Synthesis with Depth-Guided Neural Radiance Fields and Improved Depth Completion

Drantal-NeRF: Diffusion-Based Restoration for Anti-aliasing Neural Radiance Field

RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models