Abstract:Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Although recent TTA has shown promising performance, we still face two key challenges: 1) prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications; 2) while existing TTA can significantly improve the test performance on out-of-distribution data, they often suffer from severe performance degradation on in-distribution data after TTA (known as forgetting). To this end, we have proposed an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples for test-time entropy minimization. To alleviate forgetting, EATA introduces a Fisher regularizer estimated from test samples to constrain important model parameters from drastic changes. However, in EATA, the adopted entropy loss consistently assigns higher confidence to predictions even for samples that are underlying uncertain, leading to overconfident predictions. To tackle this, we further propose EATA with Calibration (EATA-C) to separately exploit the reducible model uncertainty and the inherent data uncertainty for calibrated TTA. Specifically, we measure the model uncertainty by the divergence between predictions from the full network and its sub-networks, on which we propose a divergence loss to encourage consistent predictions instead of overconfident ones. To further recalibrate prediction confidence, we utilize the disagreement among predicted labels as an indicator of the data uncertainty, and then devise a min-max entropy regularizer to selectively increase and decrease prediction confidence for different samples. Experiments on image classification and semantic segmentation verify the effectiveness of our methods.

Test-Time Adaptation Induces Stronger Accuracy and Agreement-on-the-Line

Reliable Test-Time Adaptation via Agreement-on-the-Line

Distribution Alignment for Fully Test-Time Adaptation with Dynamic Online Data Streams

Efficient Test-Time Model Adaptation without Forgetting

ODS: Test-Time Adaptation in the Presence of Open-World Data Shift.

On Pitfalls of Test-Time Adaptation

Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correction

Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift

Generalized Robust Test-Time Adaptation in Continuous Dynamic Scenarios

Towards Test Time Adaptation Via Calibrated Entropy Minimization

ETAGE: Enhanced Test Time Adaptation with Integrated Entropy and Gradient Norms for Robust Model Performance

Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting

A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts

Test-Time Adaptation with Perturbation Consistency Learning

Confidence-based and sample-reweighted test-time adaptation

Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation

Learning to Adapt to Online Streams with Distribution Shifts

Beyond Invariance: Test-Time Label-Shift Adaptation for Distributions with "Spurious" Correlations

Robust gradient aware and reliable entropy minimization for stable test-time adaptation in dynamic scenarios

Towards Stable Test-time Adaptation in Dynamic Wild World

Test-Time Adaptation via Self-Training with Nearest Neighbor Information