Abstract:In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its \textit{partial derivatives}, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute $\epsilon$-approximate inverse Cholesky factors of the kernel matrices with complexity $O(N\log^d(N/\epsilon))$ in space and $O(N\log^{2d}(N/\epsilon))$ in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm's near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Ampère equations.

Kernel Multi-Grid on Manifolds

A High-Order Kernel Method for Diffusion and Reaction-Diffusion Equations on Surfaces

A GPU-accelerated Cartesian grid method is proposed for solving the heat, wave, and Schrodinger equations on irregular domains

A GPU-accelerated Cartesian grid method for PDEs on irregular domain

Toward Efficient Kernel-Based Solvers for Nonlinear PDEs

Multigrid Monte Carlo Revisited: Theory and Bayesian Inference

Product kernels are efficient and flexible tools for high-dimensional scattered data interpolation

Multigrid method for symmetric Toeplitz tridiagonal matrix: Convergence analysis and application to fractional Feynman-Kac equation

Greedy Kernel Methods for Center Manifold Approximation

An Adaptive Multi‐Grid Solver for Applications in Computer Graphics

Algebraic multigrid methods for metric-perturbed coupled problems

Efficient Multigrid Reduction-in-Time for Method-of-Lines Discretizations of Linear Advection

Extracting Manifold Information from Point Clouds

Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes

Fast Sparse Kernel Summation on Cartesian Grids

Structure-preserving Kernel-based methods for solving dissipative PDEs on surfaces

Sparse grid time-discontinuous Galerkin method with streamline diffusion for transport equations

A note on the convergence of multigrid methods for the Riesz-space equation and an application to image deblurring

Kernel-based active subspaces with application to computational fluid dynamics parametric problems using the discontinuous Galerkin method

Solving Partial Differential Equations on Point Clouds