Abstract:Abstract Markov chain Monte Carlo (MCMC) methods have been used in many fields (physics, chemistry, biology, and computer science) for simulation, inference, and optimization. The essence of these methods is to simulate aM arkov chain whose stateX follows a target probability X ∼ π(X). In many applications, π(X) is defined on a graph G whose vertices represent elements in the system and whose edges represent the connectivity of the elements. X is a vector of variables on the vertices which often take discrete values called labels or colors. Designing rapid mixing Markov chain is a challenging task when the variables in the graph are strongly coupled. Methods, like the single-site Gibbs sampler, often experience long waiting time. A well-celebrated algorithm for sampling on graphs is the Swendsen-Wang (1987) (SW) method. The SW method finds a cluster of vertices as a connected component after turning off some edges probabilistically, and flips the color of the cluster as a whole. It is shown to mix rapidly under certain conditions. Unfortunately, the SW method is only applicable to the Ising/Potts models and slow down critically in the presence of ”external fields” i.e. likelihood in Bayesian inference. In this paper, we present a general cluster sampling method which achieves the following objectives. Firstly, it extends the SW algorithm to general Bayesian inference on graphs. Especially we focus a number of image analysis problems where the graph sizes are in the order of O(10, )w ithO(1) connectivity. Secondly, the edge probability for clustering the vertices are discriminative probabilities computed from data. Empirically such data driven clustering leads to much improved efficiency. Thirdly, we present a generalized Gibbs sampler which samples the color of a cluster according to a conditional probability (like the Gibbs sampler) weighted by a product

Flat-histogram algorithms: optimal parameters and extended application

Flat histogram Monte Carlo method

Transition matrix Monte Carlo and flat-histogram algorithm

Dynamically Optimized Wang-Landau Sampling with Adaptive Trial Moves and Modification Factors

Flat Histogram Monte Carlo Simulations of Triangulated Fixed-Connectivity Surface Models

Sampling algorithms in statistical physics: a guide for statistics and machine learning

Histogram-Based Estimation of Distribution Algorithm: A Competent Method for Continuous Optimization.

Non-Log-Concave and Nonsmooth Sampling via Langevin Monte Carlo Algorithms

A Bayesian Approach To Histogram Comparison

Efficient sampling using Macrocanonical Monte Carlo and density of states mapping

Cluster Sampling and Its Applications in Image Analysis

Iterative Importance Sampling Algorithms for Parameter Estimation

Extension to the weighted histogram analysis method: combining umbrella sampling with free energy calculations

Superposition Enhanced Nested Sampling

Wide Sampling and Efficient Updating Monte Carlo Algorithms for Dimer Models

Wang-Landau Sampling with Logarithmic Windows for Continuous Models.

Efficient Irreversible Monte Carlo Samplers

Fast sampling from constrained spaces using the Metropolis-adjusted Mirror Langevin algorithm

Interpretations of Cluster Sampling by Swendsen-Wang

Computing Mixture Adsorption in Porous Materials through Flat Histogram Monte Carlo Methods

On importance sampling and independent Metropolis-Hastings with an unbounded weight function