Abstract:Traditional machine learning paradigms are based on the assumption that both training and test data follow the same statistical pattern, which is mathematically referred to as Independent and Identically Distributed ($i.i.d.$). However, in real-world applications, this $i.i.d.$ assumption often fails to hold due to unforeseen distributional shifts, leading to considerable degradation in model performance upon deployment. This observed discrepancy indicates the significance of investigating the Out-of-Distribution (OOD) generalization problem. OOD generalization is an emerging topic of machine learning research that focuses on complex scenarios wherein the distributions of the test data differ from those of the training data. This paper represents the first comprehensive, systematic review of OOD generalization, encompassing a spectrum of aspects from problem definition, methodological development, and evaluation procedures, to the implications and future directions of the field. Our discussion begins with a precise, formal characterization of the OOD generalization problem. Following that, we categorize existing methodologies into three segments: unsupervised representation learning, supervised model learning, and optimization, according to their positions within the overarching learning process. We provide an in-depth discussion on representative methodologies for each category, further elucidating the theoretical links between them. Subsequently, we outline the prevailing benchmark datasets employed in OOD generalization studies. To conclude, we overview the existing body of work in this domain and suggest potential avenues for future research on OOD generalization. A summary of the OOD generalization methodologies surveyed in this paper can be accessed at <a class="link-external link-http" href="http://out-of-distribution-generalization.com" rel="external noopener nofollow">this http URL</a>.

A Survey on Out-of-Distribution Detection in NLP

Generalized Out-of-Distribution Detection: A Survey

Recent Advances in OOD Detection: Problems and Approaches

Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review

Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future

Out-of-Distribution Generalization in Text Classification: Past, Present, and Future

Out-of-distribution Detection in Medical Image Analysis: A survey

Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey

Towards Out-Of-Distribution Generalization: A Survey

Out-of-Distribution Detection with Deep Nearest Neighbors

Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey

Large Language Models for Anomaly and Out-of-Distribution Detection: A Survey

PnPOOD : Out-Of-Distribution Detection for Text Classification via Plug andPlay Data Augmentation

When and How Does In-Distribution Label Help Out-of-Distribution Detection?

Continual Unsupervised Out-of-Distribution Detection

Re-Benchmarking Out-of-Distribution Detection in Deep Neural Networks

ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms

Out-of-Distribution Detection Using Peer-Class Generated by Large Language Model

OpenOOD: Benchmarking Generalized Out-of-Distribution Detection

Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sources