Abstract:In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its \textit{partial derivatives}, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute $\epsilon$-approximate inverse Cholesky factors of the kernel matrices with complexity $O(N\log^d(N/\epsilon))$ in space and $O(N\log^{2d}(N/\epsilon))$ in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm's near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Ampère equations.

Toward Efficient Kernel-Based Solvers for Nonlinear PDEs

A Gaussian Process Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations

Kernel Neural Operator for Efficient Solving PDEs

Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes

Non-equispaced Fourier Neural Solvers for PDEs

A Hybrid Kernel-Free Boundary Integral Method with Operator Learning for Solving Parametric Partial Differential Equations In Complex Domains

A path-dependent PDE solver based on signature kernels

Preconditioned Krylov solvers for kernel regression

Koopman neural operator as a mesh-free solver of non-linear partial differential equations

Neural Operator: Graph Kernel Network for Partial Differential Equations

An $N \log N$ Parallel Fast Direct Solver for Kernel Matrices

A Reproducing Kernel Method for Solving Singularly Perturbed Delay Parabolic Partial Differential Equations

Approximation of Stochastic Partial Differential Equations by a Kernel-based Collocation Method

Kernel-based collocation methods for heat transport on evolving surfaces

Sparse Gaussian processes for solving nonlinear PDEs

Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEs

A Non-Gradient Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks.

Adaptive meshfree approximation for linear elliptic partial differential equations with PDE-greedy kernel methods