Abstract: Recent domain generalization (DG) approaches typically use the hypothesis learned on source domains for inference on the unseen target domain. However, such a hypothesis can be arbitrarily far from the optimal one for the target domain, induced by a gap termed ``adaptivity gap''. Without exploiting the domain information from the unseen test samples, adaptivity gap estimation and minimization are intractable, which hinders us to robustify a model to any unknown distribution. In this paper, we first establish a generalization bound that explicitly considers the adaptivity gap. Our bound motivates two strategies to reduce the gap: the first one is ensembling multiple classifiers to enrich the hypothesis space, then we propose effective gap estimation methods for guiding the selection of a better hypothesis for the target. The other method is minimizing the gap directly by adapting model parameters using online target samples. We thus propose \textbf{Domain-specific Risk Minimization (DRM)}. During training, DRM models the distributions of different source domains separately; for inference, DRM performs online model steering using the source hypothesis for each arriving target sample. Extensive experiments demonstrate the effectiveness of the proposed DRM for domain generalization with the following advantages: 1) it significantly outperforms competitive baselines on different distributional shift settings; 2) it achieves either comparable or superior accuracies on all source domains compared to vanilla empirical risk minimization; 3) it remains simple and efficient during training, and 4) it is complementary to invariant learning approaches.

DRM Revisited: A Complete Error Analysis

Error Analysis of Deep Ritz Methods for Elliptic Equations.

Convergence Rate Analysis for Deep Ritz Method

Deep Ritz Methods for Laplace Equations with Dirichlet Boundary Condition

Convergence Analysis of a Quasi-Monte Carlo-based Deep Learning Algorithm for Solving Partial Differential Equations

Error Analysis of Three-Layer Neural Network Trained with PGD for Deep Ritz Method

A Priori Error Estimate of Deep Mixed Residual Method for Elliptic PDEs

Refined generalization analysis of the Deep Ritz Method and Physics-Informed Neural Networks

Programming Strategies Of Sequential Incremental-Scale Subproblems For Large Scale Data Reconciliation And Parameter Estimation With Multi-Operational Conditions

Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation

Defending Deep Regression Models against Backdoor Attacks

Sequential Sub-Problem Programming Strategies For Data Reconciliation And Parameter Estimation With Multiple Data Sets

Full error analysis for the training of deep neural networks

Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations

Error analysis for empirical risk minimization over clipped ReLU networks in solving linear Kolmogorov partial differential equations

Error Analysis of Option Pricing via Deep PDE Solvers: Empirical Study

A Deep Double Ritz Method (D$^2$RM) for solving Partial Differential Equations using Neural Networks

Domain-Specific Risk Minimization for Out-of-Distribution Generalization

Analyzing Robustness of the Deep Reinforcement Learning Algorithm in Ramp Metering Applications Considering False Data Injection Attack and Defense

A Dynamics Theory of Implicit Regularization in Deep Low-Rank Matrix Factorization

The Impact of Task Underspecification in Evaluating Deep Reinforcement Learning