Abstract:There has been a massive increase in research interest towards applying data driven methods to problems in mechanics. While traditional machine learning (ML) methods have enabled many breakthroughs, they rely on the assumption that the training (observed) data and testing (unseen) data are independent and identically distributed (i.i.d). Thus, traditional ML approaches often break down when applied to real world mechanics problems with unknown test environments and data distribution shifts. In contrast, out-of-distribution (OOD) generalization assumes that the test data may shift (i.e., violate the i.i.d. assumption). To date, multiple methods have been proposed to improve the OOD generalization of ML methods. However, because of the lack of benchmark datasets for OOD regression problems, the efficiency of these OOD methods on regression problems, which dominate the mechanics field, remains unknown. To address this, we investigate the performance of OOD generalization methods for regression problems in mechanics. Specifically, we identify three OOD problems: covariate shift, mechanism shift, and sampling bias. For each problem, we create two benchmark examples that extend the Mechanical MNIST dataset collection, and we investigate the performance of popular OOD generalization methods on these mechanics-specific regression problems. Our numerical experiments show that in most cases, while the OOD generalization algorithms perform better compared to traditional ML methods on these OOD problems, there is a compelling need to develop more robust OOD generalization methods that are effective across multiple OOD scenarios. Overall, we expect that this study, as well as the associated open access benchmark datasets, will enable further development of OOD generalization methods for mechanics specific regression problems.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the data distribution shift (i.e., non - independent and identically distributed, Out - of - Distribution, OOD) problem of machine learning (ML) models in the face of unknown test environments in the field of mechanics. Specifically, traditional machine learning methods assume that the training data and the test data are independent and identically distributed (i.i.d.), but in practical applications, especially in mechanics problems, this assumption is often not valid. When the distribution of the test data is different from that of the training data, the performance of traditional machine learning models may be very fragile, especially when dealing with regression problems, and these problems are more common in mechanics research than classification problems. To meet this challenge, the paper conducts research in the following aspects: 1. **Identifying three OOD generalization problems**: The paper identifies three main OOD generalization problems, namely Covariate Shift, Mechanism Shift and Sampling Bias. - **Covariate Shift**: Only the distribution of the input features changes, while the conditional distribution between the output and the input remains unchanged. - **Mechanism Shift**: The mapping relationship between the input and the output changes due to the change of the underlying mechanism. - **Sampling Bias**: The selection of the training data cannot represent the entire data pool, which may lead to a decline in the generalization performance of the model when it encounters unseen data after deployment. 2. **Creating benchmark datasets**: For each OOD generalization problem, the paper creates two benchmark examples based on the extended Mechanical MNIST dataset. These datasets are used to evaluate the performance of existing OOD generalization methods on mechanics - specific regression problems. 3. **Evaluating existing OOD methods**: Through numerical experiments, the paper evaluates the performance of several popular OOD generalization methods on these three mechanics - specific regression problems, and points out that although these OOD algorithms are superior to traditional machine learning methods in most cases, more robust OOD methods still need to be developed to improve the generalization ability for multiple OOD scenarios. Overall, this paper aims to promote the further development of this field by systematically studying and evaluating the application of OOD generalization methods in mechanics regression problems, especially providing an important research basis in the absence of relevant benchmark datasets.

Towards out of distribution generalization for problems in mechanics

Towards Out-Of-Distribution Generalization: A Survey

Towards a Theoretical Framework of Out-of-Distribution Generalization

Certifiable Out-of-Distribution Generalization.

OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalization

A Survey on Evaluation of Out-of-Distribution Generalization

Model-Agnostic Random Weighting for Out-of-Distribution Generalization

Mixture Data for Training Cannot Ensure Out-of-distribution Generalization

Out-of-Distribution Generalization in Text Classification: Past, Present, and Future

Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future

Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing

Improving Out-of-Distribution Generalization of Learned Dynamics by Learning Pseudometrics and Constraint Manifolds

Out-of-Distribution Generalization Analysis via Influence Function

Out-of-Distribution Learning with Human Feedback

Enhancing Distributional Stability among Sub-populations

Out-of-Domain Generalization in Dynamical Systems Reconstruction

On the Benefits of Over-parameterization for Out-of-Distribution Generalization

Probing out-of-distribution generalization in machine learning for materials

Generalized Out-of-Distribution Detection: A Survey

DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization