Abstract:Although Neural Radiance Fields (NeRFs) have markedly improved novel view synthesis, accurate uncertainty quantification in their image predictions remains an open problem. The prevailing methods for estimating uncertainty, including the state-of-the-art Density-aware NeRF Ensembles (DANE) [29], quantify uncertainty without calibration. This frequently leads to over- or under-confidence in image predictions, which can undermine their real-world applications. In this paper, we propose a method which, for the first time, achieves calibrated uncertainties for NeRFs. To accomplish this, we overcome a significant challenge in adapting existing calibration techniques to NeRFs: a need to hold out ground truth images from the target scene, reducing the number of images left to train the NeRF. This issue is particularly problematic in sparse-view settings, where we can operate with as few as three images. To address this, we introduce the concept of a meta-calibrator that performs uncertainty calibration for NeRFs with a single forward pass without the need for holding out any images from the target scene. Our meta-calibrator is a neural network that takes as input the NeRF images and uncalibrated uncertainty maps and outputs a scene-specific calibration curve that corrects the NeRF's uncalibrated uncertainties. We show that the meta-calibrator can generalize on unseen scenes and achieves well-calibrated and state-of-the-art uncertainty for NeRFs, significantly beating DANE and other approaches. This opens opportunities to improve applications that rely on accurate NeRF uncertainty estimates such as next-best view planning and potentially more trustworthy image reconstruction for medical diagnosis. The code is available at <a class="link-external link-https" href="https://niki-amini-naieni.github.io/instantcalibration.github.io/" rel="external noopener nofollow">this https URL</a>.

Ambiguity-Free Radiometric Calibration for Internet Photo Collections.

Radiometric Calibration for Internet Photo Collections

Radiometric Calibration From Faces In Images

NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo

Radiometric calibration by transform invariant low-rank structure

Radiometric Calibration by Rank Minimization

A Linear Approach for Depth and Colour Camera Calibration Using Hybrid Parameters

A Soft Calibration Technique for Infrared Imaging Based on Physical Model of Thermal Radiation and Variational Bayesian Inference from Small Dataset

Instant Uncertainty Calibration of NeRFs Using a Meta-Calibrator

Relative Radiometric Calibration Method Based on Linear CCD Imaging the Same Region of Non-Uniform Scene

A Robust HDR Image Calibration Algorithm for Camera Response Function

Optimal Radiometric Calibration for Camera-Display Communication

An adaptive calibration algorithm for projected images in daily environment.

Self-Calibrating Polarising Radiometric Calibration

OSPC: Online Sequential Photometric Calibration

Automatic Intercalibration of Night-Time Light Imagery Using Robust Regression

Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras

A Method for Color Calibration Based on Simulated Annealing Optimization

A NeRF-Based Color Consistency Correction Method for Remote Sensing Images

Self-calibrating Photometric Stereo by Neural Inverse Rendering

Supplementary Material: Radiometric Calibration from Faces in Images