Abstract:The sliced Wasserstein (SW) distances between two probability measures are defined as the expectation of the Wasserstein distance between two one-dimensional projections of the two measures. The randomness comes from a projecting direction that is used to project the two input measures to one dimension. Due to the intractability of the expectation, Monte Carlo integration is performed to estimate the value of the SW distance. Despite having various variants, there has been no prior work that improves the Monte Carlo estimation scheme for the SW distance in terms of controlling its variance. To bridge the literature on variance reduction and the literature on the SW distance, we propose computationally efficient control variates to reduce the variance of the empirical estimation of the SW distance. The key idea is to first find Gaussian approximations of projected one-dimensional measures, then we utilize the closed-form of the Wasserstein-2 distance between two Gaussian distributions to design the control variates. In particular, we propose using a lower bound and an upper bound of the Wasserstein-2 distance between two fitted Gaussians as two computationally efficient control variates. We empirically show that the proposed control variate estimators can help to reduce the variance considerably when comparing measures over images and point-clouds. Finally, we demonstrate the favorable performance of the proposed control variate estimators in gradient flows to interpolate between two point-clouds and in deep generative modeling on standard image datasets, such as CIFAR10 and CelebA.

Control Variates as a Variance Reduction Technique for Random Projections

Neural Control Variates for Variance Reduction

Joint control variate for faster black-box variational inference

Using Large Ensembles of Control Variates for Variational Inference

Sliced Wasserstein Estimation with Control Variates

Variance Reduction via Simultaneous Importance Sampling and Control Variates Techniques Using Vegas

When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimality

Pathwise Gradient Variance Reduction with Control Variates in Variational Inference

Neural Control Variates

Variance Reduction for the Independent Metropolis Sampler

Approximation Based Variance Reduction for Reparameterization Gradients

Double Control Variates for Gradient Estimation in Discrete Latent Variable Models

Stabilizing Estimates of Shapley Values with Control Variates

A Projection Approach to Local Regression with Variable-Dimension Covariates

Variance Control for Black Box Variational Inference Using The James-Stein Estimator

Coordinate-wise Control Variates for Deep Policy Gradients

Tight Variational Bounds via Random Projections and I-Projections

Diffusion Approximations and Control Variates for MCMC

Variance reduction for dependent sequences with applications to Stochastic Gradient MCMC

Control Variates for MCMC

Variance reduction techniques for stochastic proximal point algorithms