Abstract:The development of data-informed predictive models for dynamical systems is of widespread interest in many disciplines. We present a unifying framework for blending mechanistic and machine-learning approaches to identify dynamical systems from noisily and partially observed data. We compare pure data-driven learning with hybrid models which incorporate imperfect domain knowledge, referring to the discrepancy between an assumed truth model and the imperfect mechanistic model as model error. Our formulation is agnostic to the chosen machine learning model, is presented in both continuous- and discrete-time settings, and is compatible both with model errors that exhibit substantial memory and errors that are memoryless. First, we study memoryless linear (w.r.t. parametric-dependence) model error from a learning theory perspective, defining excess risk and generalization error. For ergodic continuous-time systems, we prove that both excess risk and generalization error are bounded above by terms that diminish with the square-root of T T , the time-interval over which training data is specified. Secondly, we study scenarios that benefit from modeling with memory, proving universal approximation theorems for two classes of continuous-time recurrent neural networks (RNNs): both can learn memory-dependent model error, assuming that it is governed by a finite-dimensional hidden variable and that, together, the observed and hidden variables form a continuous-time Markovian system. In addition, we connect one class of RNNs to reservoir computing, thereby relating learning of memory-dependent error to recent work on supervised learning between Banach spaces using random features. Numerical results are presented (Lorenz ’63, Lorenz ’96 Multiscale systems) to compare purely data-driven and hybrid approaches, finding hybrid methods less datahungry and more parametrically efficient. We also find that, while a continuous-time framing allows for robustness to irregular sampling and desirable domain- interpretability, a discrete-time framing can provide similar or better predictive performance, especially when data are undersampled and the vector field defining the true dynamics cannot be identified. Finally, we demonstrate numerically how data assimilation can be leveraged to learn hidden dynamics from noisy, partially-observed data, and illustrate challenges in representing memory by this approach, and in the training of such models.

A Distributed Neural Hybrid System Learning Framework in Modeling Complex Dynamical Systems

Fast and Stable Learning of Dynamical Systems Based on Extreme Learning Machine.

Efficient Neural Hybrid System Learning and Transition System Abstraction for Dynamical Systems

A Data-Driven Hybrid Automaton Framework to Modeling Complex Dynamical Systems

Distributed Extreme Learning Machine for Nonlinear Learning over Network

Finite Element-Neural Network Hybrid Modeling of Complex Dynamic Systems

Learning Networked Dynamical System Models with Weak Form and Graph Neural Networks

Asynchronous Lebesgue Approximation Model for Distributed Continuous-Time Nonlinear Systems

Bridging the Gap: Machine Learning to Resolve Improperly Modeled Dynamics

Neural network modeling and dynamic behavior prediction of nonlinear dynamic systems

Neural Downscaling for Complex Systems: from Large-scale to Small-scale by Neural Operator

Hybrid modeling and prediction of dynamical systems

Meta-Dynamical State Space Models for Integrative Neural Data Analysis

Decomposed Linear Dynamical Systems (dLDS) for learning the latent components of neural dynamics

Data-driven modeling of interrelated dynamical systems

Modeling Nonlinear Manifolds with Mixtures of Localized Principal Subspaces under a Self-Organizing Framework

Distributed Neurodynamic Models for Solving a Class of System of Nonlinear Equations

Model-Based Control with Sparse Neural Dynamics

A framework for machine learning of model error in dynamical systems

A Framework for the General Design and Computation of Hybrid Neural Networks