Abstract:Efficiently sampling from un-normalized target distributions is a fundamental problem in scientific computing and machine learning. Traditional approaches such as Markov Chain Monte Carlo (MCMC) guarantee asymptotically unbiased samples from such distributions but suffer from computational inefficiency, particularly when dealing with high-dimensional targets, as they require numerous iterations to generate a batch of samples. In this paper, we introduce an efficient and scalable neural implicit sampler that overcomes these limitations. The implicit sampler can generate large batches of samples with low computational costs by leveraging a neural transformation that directly maps easily sampled latent vectors to target samples without the need for iterative procedures. To train the neural implicit samplers, we introduce two novel methods: the KL training method and the Fisher training method. The former method minimizes the Kullback-Leibler divergence, while the latter minimizes the Fisher divergence between the sampler and the target distributions. By employing the two training methods, we effectively optimize the neural implicit samplers to learn and generate from the desired target distribution. To demonstrate the effectiveness, efficiency, and scalability of our proposed samplers, we evaluate them on three sampling benchmarks with different scales. These benchmarks include sampling from 2D targets, Bayesian inference, and sampling from high-dimensional energy-based models (EBMs). Notably, in the experiment involving high-dimensional EBMs, our sampler produces samples that are comparable to those generated by MCMC-based methods while being more than 100 times more efficient, showcasing the efficiency of our neural sampler. Besides the theoretical contributions and strong empirical performances, the proposed neural samplers and corresponding training methods will shed light on further research on developing efficient samplers for various applications beyond the ones explored in this study.

BNEM: A Boltzmann Sampler Based on Bootstrapped Noised Energy Matching

Iterated Denoising Energy Matching for Sampling from Boltzmann Densities

Efficient and Unbiased Sampling of Boltzmann Distributions via Consistency Models

Entropy-based Training Methods for Scalable Neural Implicit Sampler

Entropy-based Training Methods for Scalable Neural Implicit Samplers

Energy based diffusion generator for efficient sampling of Boltzmann distributions

Nebula: Network Enhanced Boltzmann Machine With Universal Local Search Architecture

Moment Matching Denoising Gibbs Sampling

Iterated Energy-based Flow Matching for Sampling from Boltzmann Densities

Adaptive Equi-Energy Sampler : Convergence and Illustration

Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamics

Superposition Enhanced Nested Sampling

Particle Dynamics for Learning EBMs

Boltzmann Generators: Sampling Equilibrium States of Many-Body Systems with Deep Learning

Active learning of Boltzmann samplers and potential energies with quantum mechanical accuracy

NAUTILUS: boosting Bayesian importance nested sampling with deep learning

Balanced Training of Energy-Based Models with Adaptive Flow Sampling

A Neural Network MCMC Sampler That Maximizes Proposal Entropy

Boosting Monte Carlo simulations of spin glasses using autoregressive neural networks

The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

Emulators for scarce and noisy data: application to auxiliary field diffusion Monte Carlo for the deuteron