Abstract:Ratios of normalizing constants for two distributions are needed in both Bayesian statistics, where they are used to compare models, and in statistical physics, where they correspond to differences in free energy. Two approaches have long been used to estimate ratios of normalizing constants. The `simple importance sampling' (SIS) or `free energy perturbation' method uses a sample drawn from just one of the two distributions. The `bridge sampling' or `acceptance ratio' estimate can be viewed as the ratio of two SIS estimates involving a bridge distribution. For both methods, difficult problems must be handled by introducing a sequence of intermediate distributions linking the two distributions of interest, with the final ratio of normalizing constants being estimated by the product of estimates of ratios for adjacent distributions in this sequence. Recently, work by Jarzynski, and independently by Neal, has shown how one can view such a product of estimates, each based on simple importance sampling using a single point, as an SIS estimate on an extended state space. This `Annealed Importance Sampling' (AIS) method produces an exactly unbiased estimate for the ratio of normalizing constants even when the Markov transitions used do not reach equilibrium. In this paper, I show how a corresponding `Linked Importance Sampling' (LIS) method can be constructed in which the estimates for individual ratios are similar to bridge sampling estimates. I show empirically that for some problems, LIS estimates are much more accurate than AIS estimates found using the same computation time, although for other problems the two methods have similar performance. Linked sampling methods similar to LIS are useful for other purposes as well.

On Standardizing the Signed Root Log Likelihood Ratio Statistic

Corrected Maximum Likelihood Estimations of the Lognormal Distribution Parameters

Marginal Likelihood Computation for Model Selection and Hypothesis Testing: An Extensive Review

Higher-order asymptotic normality of approximations to the modified signed likelihood ratio statistic for regular models

Estimating Ratios of Normalizing Constants Using Linked Importance Sampling

A Loss Function Approach to Model Specification Testing and Its Relative Efficiency

Saddlepoint approximations for likelihood ratio like statistics with applications to permutation tests

Level-Specific Correction for Nonparametric Likelihoods

Bi-Gaussianized calibration of likelihood ratios

Bartlett-corrected Two-Sample Adjusted Empirical Likelihood Via Resampling

Tail bounds for empirically standardized sums

Improved Asymptotic Formulae for Statistical Interpretation Based on Likelihood Ratio Tests

Moderate Deviation Principles for Classical Likelihood Ratio Tests of High-Dimensional Normal Distributions.

Likelihood-ratio test statistic for the finite-sample case in nonlinear ordinary differential equation models

Improved likelihood inference in generalized linear models

Asymptotic Expansions of the Distributions of Some Test Statistics in Generalized Linear Models

Enhancements of Non-parametric Generalized Likelihood Ratio Test: Bias Correction and Dimension Reduction

Inference for the normal coefficient of variation: an approximate marginal likelihood approach

Penalized asymptotic likelihood approach for linear transformation model selection

L 1 -Norm Estimation and Random Weighting Method in a Semiparametric Model

Performing Likelihood Ratio Tests with Multiply-Imputed Data Sets