Abstract:The quantitative formulation of evolution equations is the backbone for prediction, control, and understanding of dynamical systems across diverse scientific fields. Besides deriving differential equations for dynamical systems based on basic scientific reasoning or prior knowledge in recent times a growing interest emerged to infer these equations purely from data. In this article, we introduce a novel method for the sparse identification of nonlinear dynamical systems from observational data, based on the observation how the key challenges of the quality of time derivatives and sampling rates influence this problem. Our approach combines system identification based on thresholded least squares minimization with additional error measures that account for both the deviation between the model and the time derivative of the data, and the integrated performance of the model in forecasting dynamics. Specifically, we integrate a least squares error as well as the Wasserstein metric for estimated models and combine them within a Bayesian optimization framework to efficiently determine optimal hyperparameters for thresholding and weighting of the different error norms. Additionally, we employ distinct regularization parameters for each differential equation in the system, enhancing the method's precision and flexibility. We demonstrate the capabilities of our approach through applications to dynamical fMRI data and the prototypical example of a wake flow behind a cylinder. In the wake flow problem, our method identifies a sparse, accurate model that correctly captures transient dynamics, oscillation periods, and phase information, outperforming existing methods. In the fMRI example, we show how our approach extracts insights from a trained recurrent neural network, offering a novel avenue for explainable AI by inferring differential equations that capture potentially causal relationships.

What problem does this paper attempt to address?

The problems that this paper attempts to solve are: accurately identifying the evolution equations of nonlinear dynamical systems from observational data, especially how to overcome the challenges encountered by existing methods when dealing with real - data. Specifically, the author focuses on the following issues: 1. **The influence of the quality of time derivatives and the sampling rate**: When using a finite number of data points to estimate differential equations, the computational accuracy of numerical derivatives will be affected by the sampling rate, resulting in an increase in model error. 2. **Sparsity and interpretability**: How to ensure that the model can capture the true dynamic behavior of the system while ensuring the sparsity of the model, rather than just fitting noise or irrelevant terms. 3. **Limitations of existing methods**: Existing methods such as SINDy may not be able to find a suitable model in some cases (for example, the limit - cycle problem), or require very fine - grained time sampling. To solve these problems, the author proposes a new method based on Bayesian optimization, which combines different error metrics (such as least - squares error, Wasserstein distance, etc.) to more effectively determine the optimal hyper - parameters and improve the accuracy and interpretability of the model. This method is particularly suitable for extracting meaningful dynamic models from complex data, such as the vortex - street phenomenon in fluid mechanics and latent - space modeling in fMRI data. ### Specific problems and solutions - **The accuracy problem of time derivatives**: By introducing multiple error metrics (such as integral error and Wasserstein distance), the dependence on accurate time derivatives can be reduced, thereby improving the robustness of the model. - **Sparsity and interpretability**: By introducing L1 regularization and adaptive threshold selection, it is ensured that the model only contains necessary terms and over - fitting is avoided. - **Optimizing hyper - parameters**: Using the Bayesian optimization framework, hyper - parameters (such as the threshold λ and the weight α) are automatically adjusted to balance the sparsity and fitting accuracy of the model. ### Application examples The author demonstrates the effectiveness of this method through two specific applications: 1. **Cylinder wake problem**: Identify a low - dimensional model from fluid - mechanics data and successfully capture the correct transient behavior and limit - cycle phase. 2. **fMRI data analysis**: Extract a dynamic model of the latent space from a trained recurrent neural network, providing a new interpretable artificial - intelligence approach. In summary, this paper aims to overcome the limitations of existing techniques in dealing with real - data by improving system - identification methods, thereby more accurately identifying physically - meaningful dynamic - system models from data.

Sparse identification of evolution equations via Bayesian model selection

Automatically discovering ordinary differential equations from data with sparse regression

Bayesian Model Selection Methods and Their Application to Biological ODE Systems

Automatically identifying ordinary differential equations from data

A Bayesian Approach for Data-Driven Dynamic Equation Discovery

Rapid Bayesian identification of sparse nonlinear dynamics from scarce and noisy data

Model selection of chaotic systems from data with hidden variables using sparse data assimilation

Discovering governing equations from data: Sparse identification of nonlinear dynamical systems

Model selection for hybrid dynamical systems via sparse regression

Statistical Mechanics of Dynamical System Identification

Discovery of differential equations using sparse state and parameter regression

Sparse inference and active learning of stochastic differential equations from data

Discovering governing equations from data by sparse identification of nonlinear dynamical systems

Identifying Latent Dynamic Components in Biological Systems

Dynamical Modeling for Non-Gaussian Data with High-Dimensional Sparse Ordinary Differential Equations

Deep Generative Modeling for Identification of Noisy, Non-Stationary Dynamical Systems

Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertainty

Evolutionary sparse data-driven discovery of multibody system dynamics

Discovering Sparse Interpretable Dynamics from Partial Observations

Physics-informed AI and ML-based sparse system identification algorithm for discovery of PDE's representing nonlinear dynamic systems

An evolutionary approach for discovering non-Gaussian stochastic dynamical systems based on nonlocal Kramers-Moyal formulas