Abstract:Thanks to the high-quality reconstruction and photorealistic rendering, the Neural Radiance Field (NeRF) has garnered extensive attention and has been continuously improved. Despite its high visual quality, the prohibitive training time limits its practical application. Although significant acceleration has been achieved, it is still far from real-time training, due to the need for tens of thousands of iterations. In this paper, a feasible solution is to reduce the number of required iterations by always training the rays with the highest loss values, instead of the traditional method of training each ray with a uniform probability. To this end, we propose an online active ray sampling strategy, ActRay. Specifically, to avoid the substantial overhead of calculating the actual loss values for all rays in each iteration, a rendering-gradient-based loss propagation algorithm is presented to efficiently estimate the loss values. To further narrow the gap between the estimated loss and the actual loss, an online learning algorithm based on the Upper Confidence Bound (UCB) is proposed to control the sampling probability of the rays, thereby compensating for the bias in loss estimation. We evaluate ActRay on both real-world and synthetic scenes, and the promising results show that it accelerates radiance field training by 6.5x. Besides, we test ActRay under all kinds of radiance field representations (implicit, explicit, and hybrid), proving that it is general and effective to different representations. We believe this work will contribute to the practical application of radiance fields, because it has taken a step closer to real-time radiance field training. ActRay is open-source at: https://pku-netvideo.github.io/actray/.

Accelerated Neural Radiation Field Based on Optimal Ray Sampling

Joint Optimization of Triangle Mesh, Material, and Light from Neural Fields with Neural Radiance Cache

Efficient Ray Sampling for Radiance Fields Reconstruction

AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields

Polynomial for real-time rendering of neural radiance fields

Depth Guided Fast Rendering Neural Radiance Fields

Volume Feature Rendering for Fast Neural Radiance Field Reconstruction

Adaptive Multi-NeRF: Exploit Efficient Parallelism in Adaptive Multiple Scale Neural Radiance Field Rendering

Efficient View Synthesis with Neural Radiance Distribution Field

Adaptive Shells for Efficient Neural Radiance Field Rendering

Learning-based Ray Sampling Strategy for Computation Efficient Neural Radiance Field Generation

Efficient Neural Radiance Fields with Learned Depth-Guided Sampling

Improving Neural Radiance Field using Near-Surface Sampling with Point Cloud Generation

Area-NeRF: Area-based Neural Radiance Fields

ProNeRF: Learning Efficient Projection-Aware Ray Sampling for Fine-Grained Implicit Neural Radiance Fields

SteerNeRF: Accelerating NeRF Rendering via Smooth Viewpoint Trajectory

Fast Dynamic Radiance Fields with Time-Aware Neural Voxels

A real-time rendering method for high albedo anisotropic materials with multiple scattering

Efficient NeRF Optimization -- Not All Samples Remain Equally Hard

ActRay: Online Active Ray Sampling for Radiance Fields.

Improved Neural Radiance Fields Using Pseudo-depth and Fusion