Abstract:Transport-based density estimation methods are receiving growing interest because of their ability to efficiently generate samples from the approximated density. We further invertigate the sequential transport maps framework proposed from <a class="link-https" data-arxiv-id="2106.04170" href="https://arxiv.org/abs/2106.04170">arXiv:2106.04170</a> <a class="link-https" data-arxiv-id="2303.02554" href="https://arxiv.org/abs/2303.02554">arXiv:2303.02554</a>, which builds on a sequence of composed Knothe-Rosenblatt (KR) maps. Each of those maps are built by first estimating an intermediate density of moderate complexity, and then by computing the exact KR map from a reference density to the precomputed approximate density. In our work, we explore the use of Sum-of-Squares (SoS) densities and $\alpha$-divergences for approximating the intermediate densities. Combining SoS densities with $\alpha$-divergence interestingly yields convex optimization problems which can be efficiently solved using semidefinite programming. The main advantage of $\alpha$-divergences is to enable working with unnormalized densities, which provides benefits both numerically and theoretically. In particular, we provide a new convergence analyses of the sequential transport maps based on information geometric properties of $\alpha$-divergences. The choice of intermediate densities is also crucial for the efficiency of the method. While tempered (or annealed) densities are the state-of-the-art, we introduce diffusion-based intermediate densities which permits to approximate densities known from samples only. Such intermediate densities are well-established in machine learning for generative modeling. Finally we propose low-dimensional maps (or lazy maps) for dealing with high-dimensional problems and numerically demonstrate our methods on Bayesian inference problems and unsupervised learning tasks.

Order-Preserving Optimal Transport for Distances Between Sequences

Order-Preserving Wasserstein Distance for Sequence Matching

Partial ordered Wasserstein distance for sequential data

Distances for Comparing Multisets and Sequences

Order-Preserving Metric Learning for Mining Multivariate Time Series

Autoregressive Optimal Transport Models

A new class of transport distances

Learning to Count via Unbalanced Optimal Transport

Semi-Discrete Optimal Transport: Nearly Minimax Estimation With Stochastic Gradient Descent and Adaptive Entropic Regularization

Fast and Accurate Approximations of the Optimal Transport in Semi-Discrete and Discrete Settings

A Consistent Extension of Discrete Optimal Transport Maps for Machine Learning Applications

Minibatch optimal transport distances; analysis and applications

Scaling algorithms for unbalanced optimal transport problems

Generating Temporally-ordered Event Sequences via Event Optimal Transport.

PT$\mathrm{L}^{p}$: Partial Transport $\mathrm{L}^{p}$ Distances

Solving Prior Distribution Mismatch in Diffusion Models via Optimal Transport

Sequential transport maps using SoS density estimation and $α$-divergences

Functional optimal transport: map estimation and domain adaptation for functional data

On Scalable and Efficient Computation of Large Scale Optimal Transport

Scaling Algorithms for Unbalanced Transport Problems

Fast Computation of Optimal Transport via Entropy-Regularized Extragradient Methods