Abstract:Accurate uncertainty estimation is crucial for deploying neural networks in risk-sensitive applications such as medical diagnosis. Monte Carlo Dropout is a widely used technique for approximating predictive uncertainty by performing stochastic forward passes with dropout during inference. However, using static dropout rates across all layers and inputs can lead to suboptimal uncertainty estimates, as it fails to adapt to the varying characteristics of individual inputs and network layers. Existing approaches optimize dropout rates during training using labeled data, resulting in fixed inference-time parameters that cannot adjust to new data distributions, compromising uncertainty estimates in Monte Carlo simulations. In this paper, we propose Rate-In, an algorithm that dynamically adjusts dropout rates during inference by quantifying the information loss induced by dropout in each layer's feature maps. By treating dropout as controlled noise injection and leveraging information-theoretic principles, Rate-In adapts dropout rates per layer and per input instance without requiring ground truth labels. By quantifying the functional information loss in feature maps, we adaptively tune dropout rates to maintain perceptual quality across diverse medical imaging tasks and architectural configurations. Our extensive empirical study on synthetic data and real-world medical imaging tasks demonstrates that Rate-In improves calibration and sharpens uncertainty estimates compared to fixed or heuristic dropout rates without compromising predictive performance. Rate-In offers a practical, unsupervised, inference-time approach to optimizing dropout for more reliable predictive uncertainty estimation in critical applications.

Dropout Strikes Back: Improved Uncertainty Estimation via Diversity Sampling

Uncertainty Sampling Based Active Learning with Diversity Constraint by Sparse Selection.

Uncertainty-Based Active Learning Via Sparse Modeling for Image Classification

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

Uncertainty propagation for dropout-based Bayesian neural networks

Confidence Calibration for Convolutional Neural Networks Using Structured Dropout

Real-Time Uncertainty Estimation in Computer Vision via Uncertainty-Aware Distribution Distillation

Adaptive Sparse Dropout: Learning the Certainty and Uncertainty in Deep Neural Networks

ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference

Dropout injection at test time for post hoc uncertainty quantification in neural networks

RBUE: A ReLU-Based Uncertainty Estimation Method of Deep Neural Networks

Applying Monte Carlo Dropout to Quantify the Uncertainty of Skip Connection-Based Convolutional Neural Networks Optimized by Big Data

Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learning

On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation

The Peril of Popular Deep Learning Uncertainty Estimation Methods

Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation

Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization.

Uncertainty estimation via ensembles of deep learning models and dropout layers for seismic traces

Illustrative Discussion of MC-Dropout in General Dataset: Uncertainty Estimation in Bitcoin

Integrating Dropout and Kullback-Leibler Regularization in Bayesian Neural Networks for improved uncertainty estimation in Regression

DOS: Diverse Outlier Sampling for Out-of-Distribution Detection