Abstract:Out-of-distribution (OOD) generalisation is challenging because it involves not only learning from empirical data, but also deciding among various notions of generalisation, e.g., optimising the average-case risk, worst-case risk, or interpolations thereof. While this choice should in principle be made by the model operator like medical doctors, this information might not always be available at training time. The institutional separation between machine learners and model operators leads to arbitrary commitments to specific generalisation strategies by machine learners due to these deployment uncertainties. We introduce the Imprecise Domain Generalisation framework to mitigate this, featuring an imprecise risk optimisation that allows learners to stay imprecise by optimising against a continuous spectrum of generalisation strategies during training, and a model framework that allows operators to specify their generalisation preference at deployment. Supported by both theoretical and empirical evidence, our work showcases the benefits of integrating imprecision into domain generalisation.

What problem does this paper attempt to address?

The paper primarily focuses on addressing the problem of domain generalization in the field of machine learning, particularly in situations where the training data and deployment environment are inconsistent (i.e., non-independent and identically distributed, non-IID). Specifically, the paper attempts to solve the following core issues: 1. **How to effectively learn in the face of deployment environment uncertainty**: Due to the gap that often exists between training data and real-world application scenarios, such as differences in data between hospitals or changes in data distribution over time, the paper explores how to enable models to learn effectively despite this uncertainty. 2. **How to mitigate the impact of institutional separation between learners and operators**: The paper points out that during the development of machine learning models, there is usually a separation between learners (such as algorithm engineers) and operators (such as doctors), which may result in the model design not fully considering the specific needs of the operators. Therefore, the paper proposes a method to alleviate the negative impact of this separation. 3. **How to achieve domain generalization in fuzzy domains**: The paper proposes a new framework called "fuzzy domain generalization," which allows learners to maintain fuzziness during the training phase rather than committing to a specific generalization strategy in advance. This way, operators can choose the most appropriate generalization strategy based on the actual situation during deployment. By introducing concepts such as Imprecise Risk Optimization (IRO) and Augmented Hypothesis, the paper aims to build a model that can adapt to various possible generalization strategies and allows end-users (such as doctors) to specify their preferred generalization strategy during deployment based on specific circumstances. This approach helps improve the model's generalization ability in different deployment environments, especially when facing unknown or uncertain environments.

Domain Generalisation via Imprecise Learning

Knowledge Distillation-based Domain-invariant Representation Learning for Domain Generalization

Learning Domain-Invariant Relationship with Instrumental Variable for Domain Generalization

On the Limitations of General Purpose Domain Generalisation Methods

Domain Generalisation via Risk Distribution Matching

Non-stationary Domain Generalization: Theory and Algorithm

Idag: Invariant DAG Searching for Domain Generalization

Domain Generalization: A Survey

Model-Based Domain Generalization

Domain-Specific Risk Minimization for Out-of-Distribution Generalization

Probable Domain Generalization via Quantile Risk Minimization

Domain Generalization with Small Data

Domain Generalization for Domain-Linked Classes

Finding Competence Regions in Domain Generalization

Domain Generalization Via Conditional Invariant Representations

Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing

Fairness and Accuracy Under Domain Generalization

Domain Agnostic Conditional Invariant Predictions for Domain Generalization

Domain Generalization through Meta-Learning: A Survey

Improving Generalization with Domain Convex Game