Improving NeRF Quality by Progressive Camera Placement for Unrestricted Navigation in Complex Environments

Georgios Kopanas,George Drettakis
2023-09-05
Abstract:Neural Radiance Fields, or NeRFs, have drastically improved novel view synthesis and 3D reconstruction for rendering. NeRFs achieve impressive results on object-centric reconstructions, but the quality of novel view synthesis with free-viewpoint navigation in complex environments (rooms, houses, etc) is often problematic. While algorithmic improvements play an important role in the resulting quality of novel view synthesis, in this work, we show that because optimizing a NeRF is inherently a data-driven process, good quality data play a fundamental role in the final quality of the reconstruction. As a consequence, it is critical to choose the data samples -- in this case the cameras -- in a way that will eventually allow the optimization to converge to a solution that allows free-viewpoint navigation with good quality. Our main contribution is an algorithm that efficiently proposes new camera placements that improve visual quality with minimal assumptions. Our solution can be used with any NeRF model and outperforms baselines and similar work.
Computer Vision and Pattern Recognition,Graphics,Image and Video Processing
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve The paper primarily addresses the issue of poor novel view synthesis quality of Neural Radiance Fields (NeRF) during free-viewpoint navigation in complex environments (such as rooms, houses, etc.). Specifically, although existing algorithmic improvements play a significant role in the final view synthesis quality, the paper emphasizes that optimizing NeRF is fundamentally a data-driven process, and thus high-quality data is crucial for the final reconstruction quality. The main contribution of the paper is the proposal of an algorithm that can efficiently suggest new camera positions to improve visual quality without requiring any modifications to the NeRF model itself. This method is applicable to any NeRF model and performs excellently in benchmark tests. ### Specific Goals - **Define an efficient metric**: To evaluate the observation frequency and angle uniformity, which can be calculated in real-time during the NeRF capture process without requiring additional images. - **Propose an algorithm**: To quickly estimate the reconstruction quality of a scene and suggest the next best camera position to maximize the capture quality improvement based on the aforementioned metric. Through these methods, the paper aims to automate and standardize the NeRF capture process, thereby reducing the user's trial-and-error burden. Additionally, this method can be used for automatic capture in practical applications, such as shooting with robots or drones.