Abstract:Machine learning based solvers have garnered much attention in physical simulation and scientific computing, with a prominent example, physics-informed neural networks (PINNs). However, PINNs often struggle to solve high-frequency and multi-scale PDEs, which can be due to spectral bias during neural network training. To address this problem, we resort to the Gaussian process (GP) framework. To flexibly capture the dominant frequencies, we model the power spectrum of the PDE solution with a student $t$ mixture or Gaussian mixture. We apply the inverse Fourier transform to obtain the covariance function (by Wiener-Khinchin theorem). The covariance derived from the Gaussian mixture spectrum corresponds to the known spectral mixture kernel. Next, we estimate the mixture weights in the log domain, which we show is equivalent to placing a Jeffreys prior. It automatically induces sparsity, prunes excessive frequencies, and adjusts the remaining toward the ground truth. Third, to enable efficient and scalable computation on massive collocation points, which are critical to capture high frequencies, we place the collocation points on a grid, and multiply our covariance function at each input dimension. We use the GP conditional mean to predict the solution and its derivatives so as to fit the boundary condition and the equation itself. As a result, we can derive a Kronecker product structure in the covariance matrix. We use Kronecker product properties and multilinear algebra to promote computational efficiency and scalability, without low-rank approximations. We show the advantage of our method in systematic experiments. The code is released at \url{<a class="link-external link-https" href="https://github.com/xuangu-fang/Gaussian-Process-Slover-for-High-Freq-PDE" rel="external noopener nofollow">this https URL</a>}.

Separable Gaussian Neural Networks for High-Dimensional Nonlinear Stochastic Systems

Exact Stationary Solutions Independent of Energy for Strongly Nonlinear Stochastic Systems of Multiple Degrees of Freedom

Nonstationary Probability Densities Of Nonlinear Multi-Degree-Of-Freedom Systems Under Gaussian White Noise Excitations

Learning Traveling Solitary Waves Using Separable Gaussian Neural Networks

Conditional Gaussian Systems for Multiscale Nonlinear Stochastic Systems: Prediction, State Estimation and Uncertainty Quantification

A Novel Method for Response Probability Density of Nonlinear Stochastic Dynamic Systems

Solving High Frequency and Multi-Scale PDEs with Gaussian Processes

An accelerated novel meshless coupled algorithm for non-local nonlinear behavior in 2D/3D space-fractional GPEs

Exact Solutions for Nonlinear Systems under Parametric and External White-Noise Excitations

On the Optimal Design of Radial Basis Function Neural Networks for the Analysis of Nonlinear Stochastic Systems

Radial Basis Function Neural Networks Solution for Stationary Probability Density Function of Nonlinear Stochastic Systems

Non-Markovian dynamics: the governing equations of probability density functions for nonlinear dynamical systems with combined fractional Gaussian and Gaussian white noises

Extended Galerkin neural network approximation of singular variational problems with error control

A Mini-Batch Method for Solving Nonlinear PDEs with Gaussian Processes

Tensor neural networks for high-dimensional Fokker-Planck equations

Separable Physics-informed Neural Networks for Solving the BGK Model of the Boltzmann Equation

Hybrid Neural Networks for Solving Fully Coupled, High-Dimensional Forward–Backward Stochastic Differential Equations

Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations

A spectral method for a Fokker-Planck equation in neuroscience with applications in neural networks with learning rules

Artificial Neural Network Solver for Time-Dependent Fokker-Planck Equations