Abstract:Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the uncertainty problems that occur when reconstructing radiance fields from image observations. Specifically, the author proposes a novel and efficient method to quantify the uncertainty in the radiance field and reduce the ambiguity in the reconstruction process through a differential optimization strategy. The following are the specific problems the paper attempts to solve: 1. **Reconstruction uncertainty**: - Constructing a radiance field from image observations is an ill - posed inverse problem, that is, multiple different reconstruction results can explain the same data. Therefore, in addition to the original prediction, a complete and robust radiance field model must also provide a measure of its epistemic uncertainty. 2. **Efficient and fine - grained uncertainty estimation**: - Existing methods often require a large number of samples or complex approximation methods when calculating uncertainty, resulting in inefficiency. This paper proposes a method based on low - dimensional manifold sampling, which can efficiently estimate uncertainty during training and inference. 3. **Differentiated uncertainty**: - Uncertainty estimation should be differentiable so that the optimal viewing angle or lighting conditions captured later can be optimized through gradients, thereby systematically increasing the confidence of the model. This is especially important for resource - constrained applications. 4. **Capture planning in high - dimensional input domains**: - When performing capture planning in high - dimensional input domains (such as lighting conditions), simple enumeration and evaluation of candidate capture conditions are not feasible. The differential uncertainty estimation method proposed in this paper can effectively address this challenge. ### Method overview The author proposes a stochastic radiance field model based on low - dimensional manifold sampling, which has the following characteristics: - **Low - dimensional manifold sampling**: By modeling the uncertainty in the parameter space as a low - dimensional linear manifold, Monte Carlo integration can be carried out efficiently, thereby achieving efficient training and uncertainty estimation. - **Differentiated uncertainty estimation**: Since the uncertainty estimation is differentiable, the uncertainty can be minimized by optimizing camera parameters or lighting conditions, thereby guiding the best subsequent capture. - **Wide range of applications**: This method is not only applicable to next - best - view planning, but can also be applied to tasks such as lighting optimization, demonstrating its flexibility in dealing with high - dimensional domain problems. In conclusion, this paper proposes a new, general, and practical method for estimating the uncertainty of the radiance field. It can provide high - quality, differentiated uncertainty estimates while maintaining efficiency, thereby significantly outperforming existing methods.

Manifold Sampling for Differentiable Uncertainty in Radiance Fields

Estimating 3D Uncertainty Field: Quantifying Uncertainty for Neural Radiance Fields

Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations

Joint Optimization of Triangle Mesh, Material, and Light from Neural Fields with Neural Radiance Cache

Uncertainty modeling for fine-tuned implicit functions

FG-NeRF: Flow-GAN Based Probabilistic Neural Radiance Field for Independence-Assumption-Free Uncertainty Estimation

Adaptive Incident Radiance Field Sampling and Reconstruction Using Deep Reinforcement Learning.

Differentiable Rendering with Reparameterized Volume Sampling

Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields

GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation

Sources of Uncertainty in 3D Scene Reconstruction

Active Sampling and Gaussian Reconstruction for Radio Frequency Radiance Field

FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information

Neural Visibility Field for Uncertainty-Driven Active Mapping

Learning with Unreliability: Fast Few-shot Voxel Radiance Fields with Relative Geometric Consistency

Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields

Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering

Where and How: Mitigating Confusion in Neural Radiance Fields from Sparse Inputs

3D Gaussian Splatting for Real-Time Radiance Field Rendering

Differentiable Uncalibrated Imaging

Efficient Neural Radiance Fields with Learned Depth-Guided Sampling