Abstract:Neural radiance fields (NeRF) synthesize realistic novel views by estimating point attributes (density and color), followed by the volume rendering method. However, accurately predicting the arbitrary point attributes poses a challenge for the single NeRF-based model. Such limitation directly impacts the quality of novel view synthesis. To address this problem, a collaborative strategy with multiple NeRF-based models is proposed. This strategy is the first to introduce a multi-model cascaded architecture into NeRF for achieving high-quality novel view synthesis. Its purpose is to utilize a cascading architecture in space for the progressive enhancement of point attribute accuracy. The cascading architecture includes point adjustment and snapshots fusion. Specifically, point adjustment leverages a pretrained NeRF-based model to predict the initial density and color of each point in space. This step affords an initial rendering of target scene. Then, these initial density and color of points are directly transferred to the subsequent NeRF-based model. This process guides the subsequent NeRF-based model to focus on the refinement of initial point attributes and synthesize more realistic novel views. Finally, snapshots fusion fuses outputs (referred as snapshots) from multiple parallel subsequent NeRF-based models to synthesize the ultimate high-quality novel views. The proposed strategy is tested with a range of established NeRF-based methods, such as NeRF, Instant-NGP, and TensoRF. Experimental data for this research are sourced from the realistic 360 synthetic dataset and the LLFF dataset. Results indicate that the proposed collaborative strategy with established NeRF-based methods can improve the quality of novel view synthesis, surpassing the corresponding single model. Our project page is available at https://github.com/ZhenyangLiu/Collaborative-Neural-Radiance-Fields-for-Novel-View-Synthesis.

STrCNeRF: an Implicit Neural Representation Method Combining Global and Local Features from an Image for Novel View Synthesis

SNeRF: Stylized Neural Implicit Representations for 3D Scenes

Collaborative neural radiance fields for novel view synthesis

A neural refinement network for single image view synthesis

Multi-View Stereo and Depth Priors Guided NeRF for View Synthesis.

CompNVS: Novel View Synthesis with Scene Completion

View Synthesis with Sculpted Neural Points

Remote Sensing Novel View Synthesis With Implicit Multiplane Representations

Multi-Scale Feature Fusion for Single Image Novel View Synthesis

ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning

Attention Beats Linear for Fast Implicit Neural Representation Generation

MMNeRF: Multi-Modal and Multi-View Optimized Cross-Scene Neural Radiance Fields

TransNeRF: Multi-View Optimization for General Neural Radiance Fields Across Scenes

CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistency

SG-NeRF: Sparse-Input Generalized Neural Radiance Fields for Novel View Synthesis

Novel View Synthesis from only a 6-DoF Camera Pose by Two-stage Networks

STSR-INR: Spatiotemporal super-resolution for multivariate time-varying volumetric data via implicit neural representation

Multi-view stereo-regulated NeRF for urban scene novel view synthesis

Depth assisted novel view synthesis using few images

Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis

Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals