Abstract:The challenge of approximating functions in infinite-dimensional spaces from finite samples is widely regarded as formidable. We delve into the challenging problem of the numerical approximation of Sobolev-smooth functions defined on probability spaces. Our particular focus centers on the Wasserstein distance function, which serves as a relevant example. In contrast to the existing body of literature focused on approximating efficiently pointwise evaluations, we chart a new course to define functional approximants by adopting three machine learning-based approaches: 1. Solving a finite number of optimal transport problems and computing the corresponding Wasserstein potentials. 2. Employing empirical risk minimization with Tikhonov regularization in Wasserstein Sobolev spaces. 3. Addressing the problem through the saddle point formulation that characterizes the weak form of the Tikhonov functional's Euler-Lagrange equation. We furnish explicit and quantitative bounds on generalization errors for each of these solutions. We leverage the theory of metric Sobolev spaces and we combine it with techniques of optimal transport, variational calculus, and large deviation bounds. In our numerical implementation, we harness appropriately designed neural networks to serve as basis functions. These networks undergo training using diverse methodologies. This approach allows us to obtain approximating functions that can be rapidly evaluated after training. Our constructive solutions significantly enhance at equal accuracy the evaluation speed, surpassing that of state-of-the-art methods by several orders of magnitude. This allows evaluations over large datasets several times faster, including training, than traditional optimal transport algorithms. Our analytically designed deep learning architecture slightly outperforms the test error of state-of-the-art CNN architectures on datasets of images.

Numerical Analysis on Neural Network Projected Schemes for Approximating One Dimensional Wasserstein Gradient Flows

Energetic Variational Neural Network Discretizations of Gradient Flows

Neural Sinkhorn Gradient Flow

Large-Scale Wasserstein Gradient Flows

A new flow dynamic approach for Wasserstein gradient flows

Parameterized Wasserstein Gradient Flow

Discrete approximation of the minimizing movement scheme for evolution equations of Wasserstein gradient flow type with nonlinear mobility

Kernel Approximation of Fisher-Rao Gradient Flows

Approximation and Gradient Descent Training with Neural Networks

Hopfield Neural Network Flow: A Geometric Viewpoint

Approximation Theory, Computing, and Deep Learning on the Wasserstein Space

Deep JKO: time-implicit particle methods for general nonlinear gradient flows

Approximate inference with Wasserstein gradient flows

Error Analysis of Projection Schemes

Convergence Analysis of a Class of Nonsmooth Gradient Systems.

Parameterized Wasserstein Hamiltonian Flow

Gradient Flow Based Phase-Field Modeling Using Separable Neural Networks

High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems

Variational Extrapolation of Implicit Schemes for General Gradient Flows

Metric Flows with Neural Networks

Neural network approximation of coarse-scale surrogates in numerical homogenization