Abstract:The willingness to trust predictions formulated by automatic algorithms is key in a vast number of domains. However, a vast number of deep architectures are only able to formulate predictions without an associated uncertainty. In this paper, we propose a method to convert a standard neural network into a Bayesian neural network and estimate the variability of predictions by sampling different networks similar to the original one at each forward pass. We couple our methods with a tunable rejection-based approach that employs only the fraction of the dataset that the model is able to classify with an uncertainty below a user-set threshold. We test our model in a large cohort of brain images from Alzheimer's Disease patients, where we tackle discrimination of patients from healthy controls based on morphometric images only. We demonstrate how combining the estimated uncertainty with a rejection-based approach increases classification accuracy from 0.86 to 0.95 while retaining 75% of the test set. In addition, the model can select cases to be recommended for manual evaluation based on excessive uncertainty. We believe that being able to estimate the uncertainty of a prediction, along with tools that can modulate the behavior of the network to a degree of confidence that the user is informed about (and comfortable with) can represent a crucial step in the direction of user compliance and easier integration of deep learning tools into everyday tasks currently performed by human operators.

Inhibited Softmax for Uncertainty Estimation in Neural Networks

Understanding Softmax Confidence and Uncertainty

Revisiting Softmax for Uncertainty Approximation in Text Classification

Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shifts

Energy-Based Open-World Uncertainty Modeling for Confidence Calibration

Uncertainty Calibration with Energy Based Instance-wise Scaling in the Wild Dataset

Interval Neural Networks: Uncertainty Scores

What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?

Uncertainty sets for image classifiers using conformal prediction

Enabling Uncertainty Estimation in Iterative Neural Networks

BayesNetCNN: incorporating uncertainty in neural networks for image-based classification tasks

Probabilistic MIMO U-Net: Efficient and Accurate Uncertainty Estimation for Pixel-wise Regression

Uncertainty Quantification in Deep Neural Networks through Statistical Inference on Latent Space

Optimization under Uncertainty: A Perspective of Soft Computing

Rethinking Softmax with Cross-Entropy: Neural Network Classifier as Mutual Information Estimator

Boosting Out-of-Distribution Image Detection With Epistemic Uncertainty

Convex Bounds on the Softmax Function with Applications to Robustness Verification

Lightweight Model Uncertainty Estimation for Deep Neural Object Detection

Uncertainty quantification in neural network classifiers—A local linear approach

Softmax Cross Entropy Loss with Unbiased Decision Boundary for Image Classification

Mode-marginals: Expressing uncertainty via diverse M-best solutions