Abstract:We consider Ising models on the hypercube with a general interaction matrix $J$, and give a polynomial time sampling algorithm when all but $O(1)$ eigenvalues of $J$ lie in an interval of length one, a situation which occurs in many models of interest. This was previously known for the Glauber dynamics when *all* eigenvalues fit in an interval of length one; however, a single outlier can force the Glauber dynamics to mix torpidly. Our general result implies the first polynomial time sampling algorithms for low-rank Ising models such as Hopfield networks with a fixed number of patterns and Bayesian clustering models with low-dimensional contexts, and greatly improves the polynomial time sampling regime for the antiferromagnetic/ferromagnetic Ising model with inconsistent field on expander graphs. It also improves on previous approximation algorithm results based on the naive mean-field approximation in variational methods and statistical physics. Our approach is based on a new fusion of ideas from the MCMC and variational inference worlds. As part of our algorithm, we define a new nonconvex variational problem which allows us to sample from an exponential reweighting of a distribution by a negative definite quadratic form, and show how to make this procedure provably efficient using stochastic gradient descent. On top of this, we construct a new simulated tempering chain (on an extended state space arising from the Hubbard-Stratonovich transform) which overcomes the obstacle posed by large positive eigenvalues, and combine it with the SGD-based sampler to solve the full problem.

Consistency of Importance Sampling estimates based on dependent sample sets and an application to models with factorizing likelihoods

Importance Sampling of Interval Markov Chains

Particle Efficient Importance Sampling

Using Autodiff to Estimate Posterior Moments, Marginals and Samples

Improving Importance Sampling Method in Structural Reliability

Efficient large deviation estimation based on importance sampling

Iterative Importance Sampling Algorithms for Parameter Estimation

Efficient High-Dimensional Importance Sampling

A General Framework for Importance Sampling with Latent Markov Processes

Improved marginal likelihood estimation via power posteriors and importance sampling

Efficient Importance Sampling for Rare Event Simulation with Applications

Efficient and Unbiased Sampling of Boltzmann Distributions via Consistency Models

An importance sampling method for Feldman-Cousins confidence intervals

Importance Sampling in Bayesian Networks: An Influence-Based Approximation Strategy for Importance Functions

Nonasymptotic Bounds for Suboptimal Importance Sampling

Sampling-Based Approaches to Calculating Marginal Densities

Estimation and confidence sets for sparse normal mixtures

Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods

Estimation in hidden Markov models via efficient importance sampling

Adaptive importance sampling via minimization of estimators of cross-entropy, mean square, and inefficiency constant

Curse-of-dimensionality revisited: Collapse of importance sampling in very large scale systems