Abstract:Recently, Gaussian processes have been used to model the vector field of continuous dynamical systems, referred to as GPODEs, which are characterized by a probabilistic ODE equation. Bayesian inference for these models has been extensively studied and applied in tasks such as time series prediction. However, the use of standard GPs with basic kernels like squared exponential kernels has been common in GPODE research, limiting the model's ability to represent complex scenarios. To address this limitation, we introduce normalizing flows to reparameterize the ODE vector field, resulting in a data-driven prior distribution, thereby increasing flexibility and expressive power. We develop a data-driven variational learning algorithm that utilizes analytically tractable probability density functions of normalizing flows, enabling simultaneous learning and inference of unknown continuous dynamics. Additionally, we also apply normalizing flows to the posterior inference of GP ODEs to resolve the issue of strong mean-field assumptions in posterior inference. By applying normalizing flows in both these ways, our model improves accuracy and uncertainty estimates for Bayesian Gaussian Process ODEs. We validate the effectiveness of our approach on simulated dynamical systems and real-world human motion data, including time series prediction and missing data recovery tasks. Experimental results show that our proposed method effectively captures model uncertainty while improving accuracy.

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on two aspects: 1. **Improving the expressiveness and flexibility of the model**: When dealing with complex dynamic systems, the existing Gaussian Process Ordinary Differential Equation (GPODE) models are limited in their modeling capabilities because they use the standard Gaussian Process (GP) and basic kernel functions (such as the squared - exponential kernel). These basic kernel functions lack sufficient flexibility to capture complex dynamic behaviors, resulting in poor performance of the model when representing complex scenarios. 2. **Improving the accuracy of posterior inference**: In posterior inference, traditional GPODE models usually assume that the variational parameters have a mean - field Gaussian posterior distribution. However, according to Bayes' theorem, the posterior distribution is often non - Gaussian, and this strong mean - field assumption may lead to errors in uncertainty estimation. To solve these problems, the author introduced Normalizing Flows to re - parameterize the vector field of the Ordinary Differential Equation (ODE), thereby obtaining a data - driven prior distribution and increasing the flexibility and expressiveness of the model. At the same time, the author also applied Normalizing Flows to the posterior inference of GP ODE to generate a non - Gaussian posterior distribution, further improving the accuracy of the model and uncertainty estimation. Specifically, the main contributions of the paper include: - **Introducing double Normalizing Flows**: A new perspective is proposed, taking advantage of the application of Normalizing Flows in Bayesian dynamic system modeling, providing a flexible and data - driven framework for modeling complex systems. - **Developing a data - driven variational learning algorithm**: By using the analytical tractability of the probability density function of Normalizing Flows, an interpretable and tractable variational learning algorithm is constructed, achieving accurate and reliable learning and inference of unknown continuous dynamics. - **Verifying the effectiveness of the method**: Through experiments on simulated dynamic systems and real human motion data, the superiority of the proposed method in time - series prediction and missing - data recovery tasks is demonstrated. Through these improvements, the method proposed in the paper has achieved significant results in improving model accuracy and uncertainty estimation.

Data-driven Modeling and Inference for Bayesian Gaussian Process ODEs via Double Normalizing Flows

Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations

Gaussian Interpolation Flows

Towards Efficient Modeling and Inference in Multi-Dimensional Gaussian Process State-Space Models

PDE-constrained Gaussian process surrogate modeling with uncertain data locations

A dimension-reduced variational approach for solving physics-based inverse problems using generative adversarial network priors and normalizing flows

An adaptive Gaussian process method for multi-modal Bayesian inverse problems

Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides

Efficient, multimodal, and derivative-free Bayesian inference with Fisher-Rao gradient flows

Solving High Frequency and Multi-Scale PDEs with Gaussian Processes

Model-Embedded Gaussian Process Regression for Parameter Estimation in Dynamical System

Bayesian Learning of Orthogonal Embeddings for Multi-Fidelity Gaussian Processes

Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models

Bayesian Neural Ordinary Differential Equations

Bayesian learning with Gaussian processes for low-dimensional representations of time-dependent nonlinear systems

Fast multi-fidelity gaussian processes with derivatives for complex system modeling

Conditional Pseudo-Reversible Normalizing Flow for Surrogate Modeling in Quantifying Uncertainty Propagation

Time-changed normalizing flows for accurate SDE modeling

Variational Bayesian Optimal Experimental Design with Normalizing Flows