Abstract:Neural Radiance Fields (NeRF) have garnered remarkable success in novel view synthesis. Nonetheless, the task of generating high-quality images for novel views persists as a critical challenge. While the existing efforts have exhibited commendable progress, capturing intricate details, enhancing textures, and achieving superior Peak Signal-to-Noise Ratio (PSNR) metrics warrant further focused attention and advancement. In this work, we propose NeRF-VPT, an innovative method for novel view synthesis to address these challenges. Our proposed NeRF-VPT employs a cascading view prompt tuning paradigm, wherein RGB information gained from preceding rendering outcomes serves as instructive visual prompts for subsequent rendering stages, with the aspiration that the prior knowledge embedded in the prompts can facilitate the gradual enhancement of rendered image quality. NeRF-VPT only requires sampling RGB data from previous stage renderings as priors at each training stage, without relying on extra guidance or complex techniques. Thus, our NeRF-VPT is plug-and-play and can be readily integrated into existing methods. By conducting comparative analyses of our NeRF-VPT against several NeRF-based approaches on demanding real-scene benchmarks, such as Realistic Synthetic 360, Real Forward-Facing, Replica dataset, and a user-captured dataset, we substantiate that our NeRF-VPT significantly elevates baseline performance and proficiently generates more high-quality novel view images than all the compared state-of-the-art methods. Furthermore, the cascading learning of NeRF-VPT introduces adaptability to scenarios with sparse inputs, resulting in a significant enhancement of accuracy for sparse-view novel view synthesis. The source code and dataset are available at \url{

NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections

IE-NeRF: Inpainting Enhanced Neural Radiance Fields in the Wild

NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild

NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images

Neural Radiance Fields From Sparse RGB-D Images for High-Quality View Synthesis

Neural Radiance Fields (NeRFs): A Review and Some Recent Developments

NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review

DecentNeRFs: Decentralized Neural Radiance Fields from Crowdsourced Images

SN 2 eRF: A Framework for Neural Radiance Fields given Sparse and Noisy Poses

Pre-NeRF 360: Enriching Unbounded Appearances for Neural Radiance Fields

Bright-NeRF:Brightening Neural Radiance Field with Color Restoration from Low-light Raw Images

Lighting up NeRF via Unsupervised Decomposition and Enhancement

Hallucinated Neural Radiance Fields in the Wild

ViewNeRF: Unsupervised Viewpoint Estimation Using Category-Level Neural Radiance Fields

Dense Depth Priors for Neural Radiance Fields from Sparse Input Views

NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning

ActiveNeRF: Learning Where to See with Uncertainty Estimation

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields

NeRF synthesis with shading guidance