Abstract:The development of data-informed predictive models for dynamical systems is of widespread interest in many disciplines. We present a unifying framework for blending mechanistic and machine-learning approaches to identify dynamical systems from noisily and partially observed data. We compare pure data-driven learning with hybrid models which incorporate imperfect domain knowledge, referring to the discrepancy between an assumed truth model and the imperfect mechanistic model as model error. Our formulation is agnostic to the chosen machine learning model, is presented in both continuous- and discrete-time settings, and is compatible both with model errors that exhibit substantial memory and errors that are memoryless. First, we study memoryless linear (w.r.t. parametric-dependence) model error from a learning theory perspective, defining excess risk and generalization error. For ergodic continuous-time systems, we prove that both excess risk and generalization error are bounded above by terms that diminish with the square-root of T T , the time-interval over which training data is specified. Secondly, we study scenarios that benefit from modeling with memory, proving universal approximation theorems for two classes of continuous-time recurrent neural networks (RNNs): both can learn memory-dependent model error, assuming that it is governed by a finite-dimensional hidden variable and that, together, the observed and hidden variables form a continuous-time Markovian system. In addition, we connect one class of RNNs to reservoir computing, thereby relating learning of memory-dependent error to recent work on supervised learning between Banach spaces using random features. Numerical results are presented (Lorenz ’63, Lorenz ’96 Multiscale systems) to compare purely data-driven and hybrid approaches, finding hybrid methods less datahungry and more parametrically efficient. We also find that, while a continuous-time framing allows for robustness to irregular sampling and desirable domain- interpretability, a discrete-time framing can provide similar or better predictive performance, especially when data are undersampled and the vector field defining the true dynamics cannot be identified. Finally, we demonstrate numerically how data assimilation can be leveraged to learn hidden dynamics from noisy, partially-observed data, and illustrate challenges in representing memory by this approach, and in the training of such models.

Combining data assimilation and machine learning to build data‐driven models for unknown long time dynamics—Applications in cardiovascular modeling

Combining data assimilation and machine learning to infer unresolved scale parametrisation

A machine learning method for real-time numerical simulations of cardiac electromechanics

Machine Learning for Cardiovascular Biomechanics Modeling: Challenges and Beyond

Machine learning for accelerating 2D flood models: Potential and challenges

Data-driven models of nonautonomous systems

Model selection focusing on longtime behavior of differential equations

Data-driven modeling of interrelated dynamical systems

Challenges and opportunities for machine learning in multiscale computational modeling

A framework for machine learning of model error in dynamical systems

Reduced order modeling of dynamical systems using artificial neural networks applied to water circulation

A domain decomposition-based autoregressive deep learning model for unsteady and nonlinear partial differential equations

Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks

Liquid Fourier Latent Dynamics Networks for fast GPU-based numerical simulations in computational cardiology

Linking Machine Learning with Multiscale Numerics: Data-Driven Discovery of Homogenized Equations

Reconstructing Nonlinear Dynamical Systems from Multi-Modal Time Series

Multiscale Modeling Meets Machine Learning: What Can We Learn?

Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review

Physics-constrained coupled neural differential equations for one dimensional blood flow modeling

A nonintrusive hybrid neural-physics modeling of incomplete dynamical systems: Lorenz equations

Data-driven cardiovascular flow modeling: examples and opportunities