Volume Feature Rendering for Fast Neural Radiance Field Reconstruction

Kang Han,Wei Xiang,Lu Yu

2023-05-31

Abstract:Neural radiance fields (NeRFs) are able to synthesize realistic novel views from multi-view images captured from distinct positions and perspectives. In NeRF's rendering pipeline, neural networks are used to represent a scene independently or transform queried learnable feature vector of a point to the expected color or density. With the aid of geometry guides either in occupancy grids or proposal networks, the number of neural network evaluations can be reduced from hundreds to dozens in the standard volume rendering framework. Instead of rendering yielded color after neural network evaluation, we propose to render the queried feature vectors of a ray first and then transform the rendered feature vector to the final pixel color by a neural network. This fundamental change to the standard volume rendering framework requires only one single neural network evaluation to render a pixel, which substantially lowers the high computational complexity of the rendering framework attributed to a large number of neural network evaluations. Consequently, we can use a comparably larger neural network to achieve a better rendering quality while maintaining the same training and rendering time costs. Our model achieves the state-of-the-art rendering quality on both synthetic and real-world datasets while requiring a training time of several minutes.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily aims to address the conflict between computational efficiency and rendering quality in neural radiance fields (NeRF) for view synthesis tasks. Specifically:

1. **Problem Background**: NeRF can synthesize realistic new views from multi-view images captured from different positions and perspectives. In the NeRF rendering process, a neural network is used to represent the scene or transform the queried learned feature vectors into the desired color or density values. Although techniques such as geometric guidance have been introduced to reduce the number of neural network evaluations, the standard volumetric rendering framework still requires multiple neural network computations for each pixel, leading to high computational complexity during training and rendering.

2. **Core Contribution**: To address the above problem, the paper proposes a new method called "Volume Feature Rendering" (VFR). This method first renders the feature vectors queried by the rays and then converts the fused feature vectors into the final pixel colors through a neural network. This approach requires only one neural network evaluation to render a pixel, significantly reducing the computational complexity of the rendering framework.

3. **Advantages**: This fundamental change allows researchers to use larger neural networks to improve rendering quality while maintaining the same training and rendering time costs. Experimental results show that VFR can complete training within a few minutes and achieve state-of-the-art rendering quality on both synthetic and real-world datasets.

In summary, the main goal of this paper is to propose a new volumetric feature rendering framework that improves rendering quality while reducing computational time consumption, thereby resolving the conflict between computational efficiency and rendering quality present in traditional NeRF methods.