Abstract:Neural radiance fields (NeRF) synthesize realistic novel views by estimating point attributes (density and color), followed by the volume rendering method. However, accurately predicting the arbitrary point attributes poses a challenge for the single NeRF-based model. Such limitation directly impacts the quality of novel view synthesis. To address this problem, a collaborative strategy with multiple NeRF-based models is proposed. This strategy is the first to introduce a multi-model cascaded architecture into NeRF for achieving high-quality novel view synthesis. Its purpose is to utilize a cascading architecture in space for the progressive enhancement of point attribute accuracy. The cascading architecture includes point adjustment and snapshots fusion. Specifically, point adjustment leverages a pretrained NeRF-based model to predict the initial density and color of each point in space. This step affords an initial rendering of target scene. Then, these initial density and color of points are directly transferred to the subsequent NeRF-based model. This process guides the subsequent NeRF-based model to focus on the refinement of initial point attributes and synthesize more realistic novel views. Finally, snapshots fusion fuses outputs (referred as snapshots) from multiple parallel subsequent NeRF-based models to synthesize the ultimate high-quality novel views. The proposed strategy is tested with a range of established NeRF-based methods, such as NeRF, Instant-NGP, and TensoRF. Experimental data for this research are sourced from the realistic 360 synthetic dataset and the LLFF dataset. Results indicate that the proposed collaborative strategy with established NeRF-based methods can improve the quality of novel view synthesis, surpassing the corresponding single model. Our project page is available at https://github.com/ZhenyangLiu/Collaborative-Neural-Radiance-Fields-for-Novel-View-Synthesis.

Deep Flow Rendering: View Synthesis via Layer-aware Reflection Flow

Forward Flow for Novel View Synthesis of Dynamic Scenes

Neural Radiance Flow for 4D View Synthesis and Video Processing

Neural Global Illumination Via Superposed Deformable Feature Fields

NeuLF: Efficient Novel View Synthesis with Neural 4D Light Field.

View Synthesis with Multi-scale Cost Aggregation and Confidence Prior

Learning Neural Duplex Radiance Fields for Real-Time View Synthesis

NeRFReN: Neural Radiance Fields with Reflections

ReShader: View-Dependent Highlights for Single Image View-Synthesis

Bidirectional Optical Flow NeRF: High Accuracy and High Quality under Fewer Views.

ProbIBR: Fast Image-Based Rendering with Learned Probability-Guided Sampling

Real‐time Neural Rendering of Dynamic Light Fields

IBRNet: Learning Multi-View Image-Based Rendering

Relit-NeuLF: Efficient Relighting and Novel View Synthesis Via Neural 4D Light Field

Depth assisted novel view synthesis using few images

CeRF: Convolutional Neural Radiance Fields for New View Synthesis with Derivatives of Ray Modeling

High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes.

Sampling for View Synthesis: From Local Light Field Fusion to Neural Radiance Fields and Beyond

Collaborative neural radiance fields for novel view synthesis

Self-Supervised Visibility Learning for Novel View Synthesis

Improved Neural Radiance Fields Using Pseudo-depth and Fusion