Abstract:The reliable prediction of the temporal behavior of complex systems is key in numerous scientific fields. This strong interest is however hindered by modeling issues: Often, the governing equations describing the physics of the system under consideration are not accessible or, when known, their solution might require a computational time incompatible with the prediction time constraints. Not surprisingly, approximating complex systems in a generic functional format and informing it ex–nihilo from available observations has become common practice in the age of machine learning, as illustrated by the numerous successful examples based on deep neural networks. However, generalizability of the models, margins of guarantee and the impact of data are often overlooked or examined mainly by relying on prior knowledge of the physics. We tackle these issues from a different viewpoint, by adopting a curriculum learning strategy. In curriculum learning, the dataset is structured such that the training process starts from simple samples toward more complex ones in order to favor convergence and generalization. The concept has been developed and successfully applied in robotics and control of systems. Here, we apply this concept for the learning of complex dynamical systems in a systematic way. First, leveraging insights from the ergodic theory, we assess the amount of data sufficient for a-priori guaranteeing a faithful model of the physical system and thoroughly investigate the impact of the training set and its structure on the quality of long-term predictions. Based on that, we consider entropy as a metric of complexity of the dataset; we show how an informed design of the training set based on the analysis of the entropy significantly improves the resulting models in terms of generalizability and provide insights on the amount and the choice of data required for an effective data-driven modeling.

Prediction and Inference: From Models and Data to Artificial Intelligence

Prediction and Inference: From Models and Data to Artificial Intelligence

An Intelligent Model with Chaos and Causality

The Role of Data in Model Building and Prediction: A Survey Through Examples

Data-driven prediction in dynamical systems: recent developments

Artificial Intelligence, Chaos, Prediction and Understanding in Science

Beyond Predictions in Neural ODEs: Identification and Interventions.

Prediction in the Hypothesis-Rich Regime

Inferring physical laws by artificial intelligence based causal models

Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

Machine Learning with Physics Knowledge for Prediction: A Survey

Predictive Dynamical Systems

Data-based prediction and causality inference of nonlinear dynamics

Physics-informed nonlinear vector autoregressive models for the prediction of dynamical systems

A Non-Probabilistic Model of Relativised Predictability in Physics

The Universal Arrow of Time V-VI: (Part V) Unpredictable dynamics (Part VI) Future of artificial intelligence - Art, not Science: Practical Application of Unpredictable Systems

Curriculum learning for data-driven modeling of dynamical systems

Physics-informed machine learning

Event prediction and causality inference despite incomplete information

Non-Intrusive Reduced Models based on Operator Inference for Chaotic Systems

Interpretable machine learning models: a physics-based view