Abstract:Deploying models on target domain data subject to distribution shift requires adaptation. Test-time training (TTT) emerges as a solution to this adaptation under a realistic scenario where access to full source domain data is not available, and instant inference on the target domain is required. Despite many efforts into TTT, there is a confusion over the experimental settings, thus leading to unfair comparisons. In this work, we first revisit TTT assumptions and categorize TTT protocols by two key factors. Among the multiple protocols, we adopt a realistic sequential test-time training (sTTT) protocol, under which we develop a test-time anchored clustering (TTAC) approach to enable stronger test-time feature learning. TTAC discovers clusters in both source and target domains and matches the target clusters to the source ones to improve adaptation. When source domain information is strictly absent (i.e. source-free) we further develop an efficient method to infer source domain distributions for anchored clustering. Finally, self-training~(ST) has demonstrated great success in learning from unlabeled data and we empirically figure out that applying ST alone to TTT is prone to confirmation bias. Therefore, a more effective TTT approach is introduced by regularizing self-training with anchored clustering, and the improved model is referred to as TTAC++. We demonstrate that, under all TTT protocols, TTAC++ consistently outperforms the state-of-the-art methods on five TTT datasets, including corrupted target domain, selected hard samples, synthetic-to-real adaptation and adversarially attacked target domain. We hope this work will provide a fair benchmarking of TTT methods, and future research should be compared within respective protocols.

CAST: Cluster-Aware Self-Training for Tabular Data via Reliable Confidence

Costra: Confidence-based self-training

FGCM: Noisy Label Learning via Fine-Grained Confidence Modeling

DCAST: Diverse Class-Aware Self-Training Mitigates Selection Bias for Fairer Learning

Improving self-training under distribution shifts via anchored confidence with theoretical guarantees

Knowledge Based Cluster Ensemble for Cancer Discovery from Biomolecular Data

RECAL: Sample-Relation Guided Confidence Calibration over Tabular Data

A Semi-Supervised Self-Training Ensemble Method Based on Clustering

Confident Learning: Estimating Uncertainty in Dataset Labels

Overcoming Overconfidence for Active Learning

Adaptive Teaching with Shared Classifier for Knowledge Distillation

Distributionally robust self-supervised learning for tabular data

Active Clustering Ensemble With Self-Paced Learning

A practical approach to novel class discovery in tabular data

Exploring the combination of self and mutual teaching for tabular-data-related semi-supervised regression

AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handler

Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering Regularized Self-Training

Leveraging Ensemble Diversity for Robust Self-Training in the Presence of Sample Selection Bias

Crowd-Certain: Label Aggregation in Crowdsourced and Ensemble Learning Classification