Abstract:Transport-based density estimation methods are receiving growing interest because of their ability to efficiently generate samples from the approximated density. We further invertigate the sequential transport maps framework proposed from <a class="link-https" data-arxiv-id="2106.04170" href="https://arxiv.org/abs/2106.04170">arXiv:2106.04170</a> <a class="link-https" data-arxiv-id="2303.02554" href="https://arxiv.org/abs/2303.02554">arXiv:2303.02554</a>, which builds on a sequence of composed Knothe-Rosenblatt (KR) maps. Each of those maps are built by first estimating an intermediate density of moderate complexity, and then by computing the exact KR map from a reference density to the precomputed approximate density. In our work, we explore the use of Sum-of-Squares (SoS) densities and $\alpha$-divergences for approximating the intermediate densities. Combining SoS densities with $\alpha$-divergence interestingly yields convex optimization problems which can be efficiently solved using semidefinite programming. The main advantage of $\alpha$-divergences is to enable working with unnormalized densities, which provides benefits both numerically and theoretically. In particular, we provide a new convergence analyses of the sequential transport maps based on information geometric properties of $\alpha$-divergences. The choice of intermediate densities is also crucial for the efficiency of the method. While tempered (or annealed) densities are the state-of-the-art, we introduce diffusion-based intermediate densities which permits to approximate densities known from samples only. Such intermediate densities are well-established in machine learning for generative modeling. Finally we propose low-dimensional maps (or lazy maps) for dealing with high-dimensional problems and numerically demonstrate our methods on Bayesian inference problems and unsupervised learning tasks.

Transporting Densities Across Dimensions

Density Estimation via Measure Transport: Outlook for Applications in the Biological Sciences

Dimension of Spatially Embedded Networks

An introduction to sampling via measure transport

Space-Time Diffusion Bridge

From Small Scales to Large Scales: Distance-to-Measure Density based Geometric Analysis of Complex Data

Measure transport via polynomial density surrogates

A Distributed Framework for the Construction of Transport Maps

Reconstruction of sequential data with density models

Data-Induced Interactions of Sparse Sensors

Understanding High Dimensional Spaces through Visual Means Employing Multidimensional Projections

Building a Telescope to Look into High-Dimensional Image Spaces.

Joint Characterization of Multiscale Information in High Dimensional Data

High dimensionality: The latest challenge to data analysis

Onboard Payload-Data Dimensionality Reduction.

Sequential transport maps using SoS density estimation and $α$-divergences

Multi-dimensional Transfer Function Design Based on Flexible Dimension Projection Embedded in Parallel Coordinates.

Exploration of Data Space Through Trans-Dimensional Sampling: A Case Study of 4D Seismics

Multi-Sensor Fusion via Reduction of Dimensionality

Big Data Scaling through Metric Mapping: Exploiting the Remarkable Simplicity of Very High Dimensional Spaces using Correspondence Analysis

Generative modeling of time-dependent densities via optimal transport and projection pursuit