Abstract:The challenge of approximating functions in infinite-dimensional spaces from finite samples is widely regarded as formidable. We delve into the challenging problem of the numerical approximation of Sobolev-smooth functions defined on probability spaces. Our particular focus centers on the Wasserstein distance function, which serves as a relevant example. In contrast to the existing body of literature focused on approximating efficiently pointwise evaluations, we chart a new course to define functional approximants by adopting three machine learning-based approaches: 1. Solving a finite number of optimal transport problems and computing the corresponding Wasserstein potentials. 2. Employing empirical risk minimization with Tikhonov regularization in Wasserstein Sobolev spaces. 3. Addressing the problem through the saddle point formulation that characterizes the weak form of the Tikhonov functional's Euler-Lagrange equation. We furnish explicit and quantitative bounds on generalization errors for each of these solutions. We leverage the theory of metric Sobolev spaces and we combine it with techniques of optimal transport, variational calculus, and large deviation bounds. In our numerical implementation, we harness appropriately designed neural networks to serve as basis functions. These networks undergo training using diverse methodologies. This approach allows us to obtain approximating functions that can be rapidly evaluated after training. Our constructive solutions significantly enhance at equal accuracy the evaluation speed, surpassing that of state-of-the-art methods by several orders of magnitude. This allows evaluations over large datasets several times faster, including training, than traditional optimal transport algorithms. Our analytically designed deep learning architecture slightly outperforms the test error of state-of-the-art CNN architectures on datasets of images.

A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models

Sliced Score Matching: A Scalable Approach to Density and Score Estimation

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimiax Optimization

Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance

A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization

Generative Modeling by Minimizing the Wasserstein-2 Loss

Approximation Theory, Computing, and Deep Learning on the Wasserstein Space

Partial Gromov-Wasserstein Learning for Partial Graph Matching

Nonparametric Score Estimators

Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization

Minimizing $f$-Divergences by Interpolating Velocity Fields

Wasserstein gradient flow for optimal probability measure decomposition

Interpretation and Generalization of Score Matching

Learning High Dimensional Wasserstein Geodesics

Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models

Concrete Score Matching: Generalized Score Matching for Discrete Data

Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow

Inclusive KL Minimization: A Wasserstein-Fisher-Rao Gradient Flow Perspective

A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inference

Efficient Learning of Generative Models via Finite-Difference Score Matching

Mean-field Variational Inference via Wasserstein Gradient Flow