Abstract:Neural radiance fields (NeRF) synthesize realistic novel views by estimating point attributes (density and color), followed by the volume rendering method. However, accurately predicting the arbitrary point attributes poses a challenge for the single NeRF-based model. Such limitation directly impacts the quality of novel view synthesis. To address this problem, a collaborative strategy with multiple NeRF-based models is proposed. This strategy is the first to introduce a multi-model cascaded architecture into NeRF for achieving high-quality novel view synthesis. Its purpose is to utilize a cascading architecture in space for the progressive enhancement of point attribute accuracy. The cascading architecture includes point adjustment and snapshots fusion. Specifically, point adjustment leverages a pretrained NeRF-based model to predict the initial density and color of each point in space. This step affords an initial rendering of target scene. Then, these initial density and color of points are directly transferred to the subsequent NeRF-based model. This process guides the subsequent NeRF-based model to focus on the refinement of initial point attributes and synthesize more realistic novel views. Finally, snapshots fusion fuses outputs (referred as snapshots) from multiple parallel subsequent NeRF-based models to synthesize the ultimate high-quality novel views. The proposed strategy is tested with a range of established NeRF-based methods, such as NeRF, Instant-NGP, and TensoRF. Experimental data for this research are sourced from the realistic 360 synthetic dataset and the LLFF dataset. Results indicate that the proposed collaborative strategy with established NeRF-based methods can improve the quality of novel view synthesis, surpassing the corresponding single model. Our project page is available at https://github.com/ZhenyangLiu/Collaborative-Neural-Radiance-Fields-for-Novel-View-Synthesis.

NOVA: NOvel View Augmentation for Neural Composition of Dynamic Objects

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images

Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects

Continuous Object Representation Networks: Novel View Synthesis without Target View Supervision

D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video

ViewFusion: Learning Composable Diffusion Models for Novel View Synthesis

NOVUM: Neural Object Volumes for Robust Object Classification

Re-Nerfing: Improving Novel View Synthesis through Novel View Synthesis

CompNVS: Novel View Synthesis with Scene Completion

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views

NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning

Free3D: Consistent Novel View Synthesis without 3D Representation

NVComposer: Boosting Generative Novel View Synthesis with Multiple Sparse and Unposed Images

FewShotNeRF: Meta-Learning-based Novel View Synthesis for Rapid Scene-Specific Adaptation

Collaborative neural radiance fields for novel view synthesis

Knowledge NeRF: Few-shot Novel View Synthesis for Dynamic Articulated Objects

NVS-Adapter: Plug-and-Play Novel View Synthesis from a Single Image

IOVS4NeRF:Incremental Optimal View Selection for Large-Scale NeRFs

Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis

GO-NeRF: Generating Objects in Neural Radiance Fields for Virtual Reality Content Creation

ProbNVS: Fast Novel View Synthesis with Learned Probability-Guided Sampling