Control Variates for MCMC

Leah South,Matthew Sutton

2024-02-12

Abstract:This chapter describes several control variate methods for improving estimates of expectations from MCMC.

Computation,Methodology

What problem does this paper attempt to address?

Diffusion Approximations and Control Variates for MCMC

Samsonov, S.

DOI: https://doi.org/10.1134/s0965542524700167

2024-06-08

Computational Mathematics and Mathematical Physics

Abstract:A new method is introduced for the construction of control variates to reduce the variance of additive functionals of Markov Chain Monte Carlo (MCMC) samplers. These control variates are obtained by minimizing the asymptotic variance associated with the Langevin diffusion over a family of functions. To motivate our approach, we then show that the asymptotic variance of some well-known MCMC algorithms, including the Random Walk Metropolis and the (Metropolis) Unadjusted/Adjusted Langevin Algorithm, are well approximated by that of the Langevin diffusion. We finally theoretically justify the use of a class of linear control variates we introduce. In particular, we show that the variance of the resulting estimators is smaller, for a given computational complexity, than the standard Monte Carlo estimator. Several examples of Bayesian inference problems support our findings showing, in some cases, very significant reduction of the variance.

mathematics, applied,physics, mathematical
Multivariable Control Performance Assessment Based on Generalized Minimum Variance Benchmark.

Zhao Yu,Su Hongye,Chu Jian,Gu Yong

DOI: https://doi.org/10.1109/cdc.2009.5400054

2010-01-01

Abstract:This paper is concerned with the control performance assessment based on the multivariable generalized minimum variance benchmark. Firstly an explicit expression for the feedback controller-invariant (here we call it generalized minimum variance) term of the multivariable control systems is obtained, and consequently the derived feedback controllerinvariant term is used as a standard benchmark for the assessment of the control performance of MIMO processes. The proposed approach is based on the multivariable minimum variance benchmark and univariate generalized minimum variance benchmark. It is the extension of the above two benchmarks. In comparison with the minimum variance benchmark, the new approach is more reasonable and practical for the control performance assessment of multivariable systems. However, it does need more information of the process. The utility of the developed approach is illustrated by simulation example.
Theoretical guarantees for neural control variates in MCMC

Denis Belomestny,Artur Goldman,Alexey Naumov,Sergey Samsonov

2024-10-28

Abstract:In this paper, we propose a variance reduction approach for Markov chains based on additive control variates and the minimization of an appropriate estimate for the asymptotic variance. We focus on the particular case when control variates are represented as deep neural networks. We derive the optimal convergence rate of the asymptotic variance under various ergodicity assumptions on the underlying Markov chain. The proposed approach relies upon recent results on the stochastic errors of variance reduction algorithms and function approximation theory.

Statistics Theory,Machine Learning,Probability,Methodology
Control Variates as a Variance Reduction Technique for Random Projections

Keegan Kang,Giles Hooker

DOI: https://doi.org/10.1007/978-3-319-93647-5_1

2018-01-01

Abstract:Control variates are used as a variance reduction technique in Monte Carlo integration, making use of positively correlated variables to bring about a reduction of variance for estimated data. By storing the marginal norms of our data, we can use control variates to reduce the variance of random projection estimates. We demonstrate the use of control variates in estimating the Euclidean distance and inner product between pairs of vectors, and give some insight on our control variate correction. Finally, we demonstrate our variance reduction through experiments on synthetic data and the arcene, colon, kos, nips datasets. We hope that our work provides a starting point for other control variate techniques in further random projection applications.
Using Large Ensembles of Control Variates for Variational Inference

Tomas Geffner,Justin Domke

DOI: https://doi.org/10.48550/arXiv.1810.12482

2020-10-22

Abstract:Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control variates. Despite the good results obtained, control variates developed for variational inference are typically looked at in isolation. In this paper we clarify the large number of control variates that are available by giving a systematic view of how they are derived. We also present a Bayesian risk minimization framework in which the quality of a procedure for combining control variates is quantified by its effect on optimization convergence rates, which leads to a very simple combination rule. Results show that combining a large number of control variates this way significantly improves the convergence of inference over using the typical gradient estimators or a reduced number of control variates.

Machine Learning
Post-Processing of MCMC

Leah F. South,Marina Riabiz,Onur Teymur,Chris. J. Oates

DOI: https://doi.org/10.48550/arXiv.2103.16048

2021-03-30

Methodology

Abstract:Markov chain Monte Carlo (MCMC) is the engine of modern Bayesian statistics, being used to approximate the posterior and derived quantities of interest. Despite this, the issue of how the output from a Markov chain is post-processed and reported is often overlooked. Convergence diagnostics can be used to control bias via burn-in removal, but these do not account for (common) situations where a limited computational budget engenders a bias-variance trade-off. The aim of this article is to review state-of-the-art techniques for post-processing Markov chain output. Our review covers methods based on discrepancy minimisation, which directly address the bias-variance trade-off, as well as general-purpose control variate methods for approximating expected quantities of interest.
When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimality

Jose Blanchet,Haoxuan Chen,Yiping Lu,Lexing Ying

DOI: https://doi.org/10.48550/arXiv.2305.16527

2023-05-26

Abstract:This paper studies the use of a machine learning-based estimator as a control variate for mitigating the variance of Monte Carlo sampling. Specifically, we seek to uncover the key factors that influence the efficiency of control variates in reducing variance. We examine a prototype estimation problem that involves simulating the moments of a Sobolev function based on observations obtained from (random) quadrature nodes. Firstly, we establish an information-theoretic lower bound for the problem. We then study a specific quadrature rule that employs a nonparametric regression-adjusted control variate to reduce the variance of the Monte Carlo simulation. We demonstrate that this kind of quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under a sufficient smoothness assumption. Due to the Sobolev Embedding Theorem, the sufficient smoothness assumption eliminates the existence of rare and extreme events. Finally, we show that, in the presence of rare and extreme events, a truncated version of the Monte Carlo algorithm can achieve the minimax optimal rate while the control variate cannot improve the convergence rate.

Statistics Theory,Information Theory,Numerical Analysis,Machine Learning
Multilevel Surrogate-based Control Variates

Mohamed Reda El Amri,Paul Mycek,Sophie Ricci,Matthias De Lozzo

2024-06-19

Abstract:Monte Carlo (MC) sampling is a popular method for estimating the statistics (e.g. expectation and variance) of a random variable. Its slow convergence has led to the emergence of advanced techniques to reduce the variance of the MC estimator for the outputs of computationally expensive solvers. The control variates (CV) method corrects the MC estimator with a term derived from auxiliary random variables that are highly correlated with the original random variable. These auxiliary variables may come from surrogate models. Such a surrogate-based CV strategy is extended here to the multilevel Monte Carlo (MLMC) framework, which relies on a sequence of levels corresponding to numerical simulators with increasing accuracy and computational cost. MLMC combines output samples obtained across levels, into a telescopic sum of differences between MC estimators for successive fidelities. In this paper, we introduce three multilevel variance reduction strategies that rely on surrogate-based CV and MLMC. MLCV is presented as an extension of CV where the correction terms devised from surrogate models for simulators of different levels add up. MLMC-CV improves the MLMC estimator by using a CV based on a surrogate of the correction term at each level. Further variance reduction is achieved by using the surrogate-based CVs of all the levels in the MLMC-MLCV strategy. Alternative solutions that reduce the subset of surrogates used for the multilevel estimation are also introduced. The proposed methods are tested on a test case from the literature consisting of a spectral discretization of an uncertain 1D heat equation, where the statistic of interest is the expected value of the integrated temperature along the domain at a given time. The results are assessed in terms of the accuracy and computational cost of the multilevel estimators, depending on whether the construction of the surrogates, and the associated computational cost, precede the evaluation of the estimator. It was shown that when the lower fidelity outputs are strongly correlated with the high-fidelity outputs, a significant variance reduction is obtained when using surrogate models for the coarser levels only. It was also shown that taking advantage of pre-existing surrogate models proves to be an even more efficient strategy.

Statistics Theory
Covariance Expressions for Multifidelity Sampling with Multioutput, Multistatistic Estimators: Application to Approximate Control Variates

Thomas O. Dixon,James E. Warner,Geoffrey F. Bomarito,Alex A. Gorodetsky

DOI: https://doi.org/10.1137/23m1607994

IF: 2.089

2024-09-10

SIAM/ASA Journal on Uncertainty Quantification

Abstract:SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 1005-1049, September 2024. We provide a collection of results on covariance expressions between Monte Carlo–based multioutput mean, variance, and Sobol main effect variance estimators from an ensemble of models. These covariances can be used within multifidelity uncertainty quantification strategies that seek to reduce the estimator variance of high-fidelity Monte Carlo estimators with an ensemble of low-fidelity models. Such covariance expressions are required within approaches such as the approximate control variate and multilevel best linear unbiased estimator. While the literature provides these expressions for some single-output cases such as mean and variance, our results are relevant to both multiple function outputs and multiple statistics across any sampling strategy. Following the description of these results, we use them within an approximate control variate scheme to show that leveraging multiple outputs can dramatically reduce estimator variance compared to single-output approaches. Synthetic examples are used to highlight the effects of optimal sample allocation and pilot sample estimation. A flight-trajectory simulation of entry, descent, and landing is used to demonstrate multioutput estimation in practical applications.

mathematics, interdisciplinary applications,physics, mathematical
Coupling Control Variates for Markov Chain Monte Carlo

Jonathan B. Goodman,Kevin K. Lin

DOI: https://doi.org/10.1016/j.jcp.2009.03.043

2008-09-13

Abstract:We show that Markov couplings can be used to improve the accuracy of Markov chain Monte Carlo calculations in some situations where the steady-state probability distribution is not explicitly known. The technique generalizes the notion of control variates from classical Monte Carlo integration. We illustrate it using two models of nonequilibrium transport.

Numerical Analysis,Computational Physics,Computation
Neural Control Variates for Variance Reduction

Ruosi Wan,Mingjun Zhong,Haoyi Xiong,Zhanxing Zhu

DOI: https://doi.org/10.48550/arXiv.1806.00159

2019-10-16

Abstract:In statistics and machine learning, approximation of an intractable integration is often achieved by using the unbiased Monte Carlo estimator, but the variances of the estimation are generally high in many applications. Control variates approaches are well-known to reduce the variance of the estimation. These control variates are typically constructed by employing predefined parametric functions or polynomials, determined by using those samples drawn from the relevant distributions. Instead, we propose to construct those control variates by learning neural networks to handle the cases when test functions are complex. In many applications, obtaining a large number of samples for Monte Carlo estimation is expensive, which may result in overfitting when training a neural network. We thus further propose to employ auxiliary random variables induced by the original ones to extend data samples for training the neural networks. We apply the proposed control variates with augmented variables to thermodynamic integration and reinforcement learning. Experimental results demonstrate that our method can achieve significant variance reduction compared with other alternatives.

Machine Learning
Control variates with splitting for aggregating results of Monte Carlo simulation and perturbation analysis

Cristóbal H. Acevedo,Marcos A. Valdebenito,Iván V. González,Héctor A. Jensen,Matthias G.R. Faes,Yong Liu

DOI: https://doi.org/10.1016/j.strusafe.2024.102445

IF: 5.712

2024-01-21

Structural Safety

Abstract:Estimation of second-order statistics allows characterizing the uncertainty associated with the response of stochastic finite element models. Two common approaches for estimating these statistics are Monte Carlo simulation and perturbation. The purpose of this paper is to present a framework to aggregate the results obtained by means of these two approaches under the umbrella of Control Variates with Splitting. This allows to produce estimates of the second-order statistics of the system's response with improved precision and accuracy. More specifically, Control Variates is implemented in such a way that the variance of the estimates of second-order statistics is minimized. In addition, the application of intervening variables for enhancing perturbation is considered as well, showing substantial advantages by increasing the accuracy of the estimates of second-order statistics. The application of the proposed framework is illustrated by means of an example involving the estimation of second-order statistics of a model involving confined seepage flow.

engineering, civil
Joint control variate for faster black-box variational inference

Xi Wang,Tomas Geffner,Justin Domke

2022-10-14

Abstract:Black-box variational inference performance is sometimes hindered by the use of gradient estimators with high variance. This variance comes from two sources of randomness: Data subsampling and Monte Carlo sampling. While existing control variates only address Monte Carlo noise, and incremental gradient methods typically only address data subsampling, we propose a new "joint" control variate that jointly reduces variance from both sources of noise. This significantly reduces gradient variance, leading to faster optimization in several applications.

Machine Learning
A splitting method to reduce MCMC variance

Robert J. Webber,David Aristoff,Gideon Simpson

DOI: https://doi.org/10.48550/arXiv.2011.13899

2020-12-17

Abstract:We explore whether splitting and killing methods can improve the accuracy of Markov chain Monte Carlo (MCMC) estimates of rare event probabilities, and we make three contributions. First, we prove that "weighted ensemble" is the only splitting and killing method that provides asymptotically consistent estimates when combined with MCMC. Second, we prove a lower bound on the asymptotic variance of weighted ensemble's estimates. Third, we give a constructive proof and numerical examples to show that weighted ensemble can approach this optimal variance bound, in many cases reducing the variance of MCMC estimates by multiple orders of magnitude.

Numerical Analysis
Double Control Variates for Gradient Estimation in Discrete Latent Variable Models

Michalis K. Titsias,Jiaxin Shi

DOI: https://doi.org/10.48550/arXiv.2111.05300

2022-06-05

Abstract:Stochastic gradient-based optimisation for discrete latent variable models is challenging due to the high variance of gradients. We introduce a variance reduction technique for score function estimators that makes use of double control variates. These control variates act on top of a main control variate, and try to further reduce the variance of the overall estimator. We develop a double control variate for the REINFORCE leave-one-out estimator using Taylor expansions. For training discrete latent variable models, such as variational autoencoders with binary latent variables, our approach adds no extra computational cost compared to standard training with the REINFORCE leave-one-out estimator. We apply our method to challenging high-dimensional toy examples and training variational autoencoders with binary latent variables. We show that our estimator can have lower variance compared to other state-of-the-art estimators.

Machine Learning
Stabilizing Estimates of Shapley Values with Control Variates

Jeremy Goldwasser,Giles Hooker

2024-04-10

Abstract:Shapley values are among the most popular tools for explaining predictions of blackbox machine learning models. However, their high computational cost motivates the use of sampling approximations, inducing a considerable degree of uncertainty. To stabilize these model explanations, we propose ControlSHAP, an approach based on the Monte Carlo technique of control variates. Our methodology is applicable to any machine learning model and requires virtually no extra computation or modeling effort. On several high-dimensional datasets, we find it can produce dramatic reductions in the Monte Carlo variability of Shapley estimates.

Machine Learning
Grouped approximate control variate estimators

Alex A. Gorodetsky,John D. Jakeman,Michael S. Eldred

2024-02-23

Abstract:This paper analyzes the approximate control variate (ACV) approach to multifidelity uncertainty quantification in the case where weighted estimators are combined to form the components of the ACV. The weighted estimators enable one to precisely group models that share input samples to achieve improved variance reduction. We demonstrate that this viewpoint yields a generalized linear estimator that can assign any weight to any sample. This generalization shows that other linear estimators in the literature, particularly the multilevel best linear unbiased estimator (ML-BLUE) of Schaden and Ullman in 2020, becomes a specific version of the ACV estimator of Gorodetsky, Geraci, Jakeman, and Eldred, 2020. Moreover, this connection enables numerous extensions and insights. For example, we empirically show that having non-independent groups can yield better variance reduction compared to the independent groups used by ML-BLUE. Furthermore, we show that such grouped estimators can use arbitrary weighted estimators, not just the simple Monte Carlo estimators used in ML-BLUE. Furthermore, the analysis enables the derivation of ML-BLUE directly from a variance reduction perspective, rather than a regression perspective.

Computation,Computational Engineering, Finance, and Science
Variational consensus Monte Carlo

Maxim Rabinovich,Elaine Angelino,Michael I. Jordan

DOI: https://doi.org/10.48550/arXiv.1506.03074

IF: 5.414

2015-06-09

Machine Learning

Abstract:Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Carlo algorithm removes this limitation by partitioning the data and drawing samples conditional on each partition in parallel (Scott et al, 2013). A fixed aggregation function then combines these samples, yielding approximate posterior samples. We introduce variational consensus Monte Carlo (VCMC), a variational Bayes algorithm that optimizes over aggregation functions to obtain samples from a distribution that better approximates the target. The resulting objective contains an intractable entropy term; we therefore derive a relaxation of the objective and show that the relaxed problem is blockwise concave under mild conditions. We illustrate the advantages of our algorithm on three inference tasks from the literature, demonstrating both the superior quality of the posterior approximation and the moderate overhead of the optimization step. Our algorithm achieves a relative error reduction (measured against serial MCMC) of up to 39% compared to consensus Monte Carlo on the task of estimating 300-dimensional probit regression parameter expectations; similarly, it achieves an error reduction of 92% on the task of estimating cluster comembership probabilities in a Gaussian mixture model with 8 components in 8 dimensions. Furthermore, these gains come at moderate cost compared to the runtime of serial MCMC, achieving near-ideal speedup in some instances.

Control Variates for MCMC

Diffusion Approximations and Control Variates for MCMC

Multivariable Control Performance Assessment Based on Generalized Minimum Variance Benchmark.

Theoretical guarantees for neural control variates in MCMC

Control Variates as a Variance Reduction Technique for Random Projections

Using Large Ensembles of Control Variates for Variational Inference

Post-Processing of MCMC

When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimality

Multilevel Surrogate-based Control Variates

Covariance Expressions for Multifidelity Sampling with Multioutput, Multistatistic Estimators: Application to Approximate Control Variates

Coupling Control Variates for Markov Chain Monte Carlo

Neural Control Variates for Variance Reduction

Control variates with splitting for aggregating results of Monte Carlo simulation and perturbation analysis

Joint control variate for faster black-box variational inference

A splitting method to reduce MCMC variance

Double Control Variates for Gradient Estimation in Discrete Latent Variable Models

Stabilizing Estimates of Shapley Values with Control Variates

Grouped approximate control variate estimators

Variational consensus Monte Carlo