Abstract:Domain generalization asks for models trained over a set of training\nenvironments to generalize well in unseen test environments. Recently, a series\nof algorithms such as Invariant Risk Minimization (IRM) have been proposed for\ndomain generalization. However, Rosenfeld et al. (2021) shows that in a simple\nlinear data model, even if non-convexity issues are ignored, IRM and its\nextensions cannot generalize to unseen environments with less than $d_s+1$\ntraining environments, where $d_s$ is the dimension of the spurious-feature\nsubspace. In this work, we propose Invariant-feature Subspace Recovery (ISR): a\nnew class of algorithms to achieve provable domain generalization across the\nsettings of classification and regression problems. First, in the binary\nclassification setup of Rosenfeld et al. (2021), we show that our first\nalgorithm, ISR-Mean, can identify the subspace spanned by invariant features\nfrom the first-order moments of the class-conditional distributions, and\nachieve provable domain generalization with $d_s+1$ training environments. Our\nsecond algorithm, ISR-Cov, further reduces the required number of training\nenvironments to $O(1)$ using the information of second-order moments. Notably,\nunlike IRM, our algorithms bypass non-convexity issues and enjoy global\nconvergence guarantees. Next, we extend ISR-Mean to the more general setting of\nmulti-class classification and propose ISR-Multiclass, which leverages class\ninformation and provably recovers the invariant-feature subspace with $\\lceil\nd_s/k\\rceil+1$ training environments for $k$-class classification. Finally, for\nregression problems, we propose ISR-Regression that can identify the\ninvariant-feature subspace with $d_s+1$ training environments. Empirically, we\ndemonstrate the superior performance of our ISRs on synthetic benchmarks.\nFurther, ISR can be used as post-processing methods for feature extractors such\nas neural nets.

Feature-Space Semantic Invariance: Enhanced OOD Detection for Open-Set Domain Generalization

IRLSG: Invariant Representation Learning for Single-Domain Generalization in Medical Image Segmentation

Towards Effective Semantic OOD Detection in Unseen Domains: A Domain Generalization Perspective

Adaptive Open Domain Recognition by Coarse-to-fine Prototype-Based Network

Open Set Domain Recognition Via Attention-Based GCN and Semantic Matching Optimization

Enlarging Feature Support Overlap for Domain Generalization

Invariant-Feature Subspace Recovery: A New Class of Provable Domain\n Generalization Algorithms

Semantic-Aware Domain Generalized Segmentation.

CSDG-FAS: Closed-Space Domain Generalization for Face Anti-spoofing

Rethinking Domain Generalization for Face Anti-spoofing: Separability and Alignment

Invariant-Feature Subspace Recovery: A New Class of Provable Domain Generalization Algorithms

Exploring Diverse Representations for Open Set Recognition

Provable Domain Generalization via Invariant-Feature Subspace Recovery

A Semantic Domain Adaption Framework for Cross-Domain Infrared Small Target Detection

Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts

MADOD: Generalizing OOD Detection to Unseen Domains via G-Invariance Meta-Learning

Selective-generative Feature Representations for Generalized Zero-Shot Open-Set Classification by Learning a Tightly Clustered Space

Two-Stage Domain Alignment Single-Source Domain Generalization Network for Cross-Scene Hyperspectral Images Classification

Inter-Class and Inter-Domain Semantic Augmentation for Domain Generalization

Open Set Face Anti-Spoofing in Unseen Attacks.

Likelihood-Aware Semantic Alignment for Full-Spectrum Out-of-Distribution Detection