Abstract:In this paper, we consider the contextual robust optimization problem under an out-of-distribution setting. The contextual robust optimization problem considers a risk-sensitive objective function for an optimization problem with the presence of a context vector (also known as covariates or side information) capturing related information. While the existing works mainly consider the in-distribution setting, and the resultant robustness achieved is in an out-of-sample sense, our paper studies an out-of-distribution setting where there can be a difference between the test environment and the training environment where the data are collected. We propose methods that handle this out-of-distribution setting, and the key relies on a density ratio estimation for the distribution shift. We show that additional structures such as covariate shift and label shift are not only helpful in defending distribution shift but also necessary in avoiding non-trivial solutions compared to other principled methods such as distributionally robust optimization. We also illustrate how the covariates can be useful in this procedure. Numerical experiments generate more intuitions and demonstrate that the proposed methods can help avoid over-conservative solutions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the context - robust optimization problem in the out - of - distribution (OOD) setting. Specifically, traditional context - optimization research mainly focuses on the same - distribution setting, that is, the training data and the test data come from the same distribution. However, in practical applications, the test environment may be different from the training environment, resulting in a change in the data distribution. This phenomenon is called out - of - distribution or distribution shift. The method proposed in the paper aims to deal with this out - of - distribution setting. By estimating the density ratio between the training distribution and the test distribution, the prediction of the model is adjusted, thereby improving the robustness and generalization ability of the model in an unknown test environment. ### Main contributions of the paper: 1. **Problem definition**: For the first time, the paper clearly proposes the out - of - distribution robust optimization problem and proposes a method using density ratio estimation to infer the situation in the test environment, even if these data are obtained from the training environment. 2. **Theoretical guarantee**: The author derives the theoretical guarantee of the proposed method and illustrates the importance of structured shift (such as covariate shift and label shift) in avoiding over - conservative solutions through an analytical example. 3. **Numerical experiment**: Generate more intuitive understanding through numerical experiments and show the effectiveness of the proposed method, especially being able to avoid overly conservative solutions. ### Key challenges in the out - of - distribution setting: - **Distribution shift**: The distribution of the test data is different from that of the training data, which may lead to poor performance of traditional methods in the test phase. - **Density ratio estimation**: In order to deal with distribution shift, it is necessary to accurately estimate the density ratio between the training distribution and the test distribution. The paper proposes several methods to estimate this density ratio, including Kernel Mean Matching (KMM) and Probabilistic Classification. ### Application scenarios: - **Financial risk assessment**: In the financial field, changes in market conditions may lead to differences in the data distribution when the model is trained and when it is actually applied. The method in the paper can help the model better adapt to these changes. - **Medical diagnosis**: In the medical field, the distribution of patient characteristics may change over time and location. The method in the paper can improve the robustness of the model in different patient groups. In conclusion, this paper deals with the context - robust optimization problem in the out - of - distribution setting by introducing density ratio estimation, providing an effective solution for dealing with data distribution changes in practical applications.

Out-of-distribution Robust Optimization

Out-of-distribution Robust Optimization

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.

A Robust Learning Algorithm for Regression Models Using Distributionally Robust Optimization under the Wasserstein Metric

Robustified Multivariate Regression and Classification Using Distributionally Robust Optimization under the Wasserstein Metric

Likelihood robust optimization for data-driven problems

O C ] 8 J an 2 01 9 Distributionally Robust Optimization with Confidence Bands for Probability Density Functions

Distributionally Robust Optimization: A Review

Robust Data-driven Prescriptiveness Optimization

Residuals-Based Contextual Distributionally Robust Optimization with Decision-Dependent Uncertainty

A Distributionally Robust Optimization Approach for Outlier Detection *

Contextual Optimization under Covariate Shift: A Robust Approach by Intersecting Wasserstein Balls

Outlier detection using distributionally robust optimization under the Wasserstein metric

Minimax Regret Optimization for Robust Machine Learning under Distribution Shift

Outlier Detection Using Robust Optimization with Uncertainty Sets Constructed from Risk Measures.

DORO: Distributional and Outlier Robust Optimization

Distributionally Robust Optimization with Confidence Bands for Probability Density Functions

Outlier-Robust Wasserstein DRO

Wasserstein Distributionally Robust Optimization with Heterogeneous Data Sources

Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization