Abstract:In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its partial derivatives, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute ε \epsilon -approximate inverse Cholesky factors of the kernel matrices with complexity O ( N log d ⁡ ( N / ε ) ) O(N\log ^d(N/\epsilon )) in space and O ( N log 2 d ⁡ ( N / ε ) ) O(N\log ^{2d}(N/\epsilon )) in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm's near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Ampère equations. In summary, we provide a fast, scalable, and accurate method for solving general PDEs with GPs and kernel methods.

Fast One-Dimensional Convolution with General Kernels Using Sum-of-Exponential Approximation

Fast Kernel Summation in High Dimensions via Slicing and Fourier Transforms

A Dual-space Multilevel Kernel-splitting Framework for Discrete and Continuous Convolution

On the sharp estimates for convolution operators with oscillatory kernel

Uniform approximation of common Gaussian process kernels using equispaced Fourier grids

A Quasi-Monte Carlo Data Structure for Smooth Kernel Evaluations

A Fast Fourier-Galerkin Method Solving Boundary Integral Equations for the Helmholtz Equation with Exponential Convergence

Toward Efficient Kernel-Based Solvers for Nonlinear PDEs

Approximation of Integral Operators Using Product-Convolution Expansions

The Fast Kernel Transform

A Fast Convolution Method and Its Application in Photomask Synthesis for Wafer Contour Fidelity Using Linear Basis Expansions

A robust solver for H(curl) convection-diffusion and its local Fourier analysis

Fast approximation by periodic kernel-based lattice-point interpolation with application in uncertainty quantification

A hybrid stochastic interpolation and compression method for kernel matrices

Fast Computation of the Multidimensional Discrete Fourier Transform and Discrete Backward Fourier Transform on Sparse Grids

On the Convolution Quadrature Rule for Integral Transforms with Oscillatory Bessel Kernels

A Fast Algorithm for the Convolution of Functions with Compact Support Using Fourier Extensions.

A Fast Summation Method for translation invariant kernels

Sparse Cholesky factorization for solving nonlinear PDEs via Gaussian processes