Depth Guided Fast Rendering Neural Radiance Fields

Tao Zhang,Haoqian Wang
DOI: https://doi.org/10.1109/fasta61401.2024.10595382
2024-01-01
Abstract:Existing methods of neural radiance fields allow high-quality rendering of images from novel viewpoints in an implicit manner. However, their need for individual training on specific scenes and relatively slow rendering speeds significantly limit their application in real-time communication. To explore and expand their applicability in the realm of real-time holographic communication, we introduce a novel high-speed rendering framework. This framework requires only sparse multi-view image inputs (e.g., four viewpoints), leveraging depth information obtained from the front end as a prior, and integrating point cloud features.It effectively reduces the voxel rendering sampling space and improves the quality of sampling points. Moreover, it decouples the dependency between neural radiance fields and scenes, shifting this dependency onto the transformation matrix between external parameters. This shift endows our model with generalizability, eliminating the need for retraining on new scenes. Experimental evidence demonstrates that our model substantially accelerates the rendering speed of novel viewpoint images on the basis of its generalizability, without compromising the quality of the rendered results even with sparse viewpoint inputs.
What problem does this paper attempt to address?