How to generate an object under an ordinary Boltzmann distribution via an exponential Boltzmann sampler

Olivier Bodini

DOI: https://doi.org/10.48550/arXiv.1006.2902

IF: 0.961

2010-06-15

Discrete Mathematics

Abstract:This short note presents an efficient way to derive from an exponential Boltzmann sampler a ordinary Boltzmann sampler

What problem does this paper attempt to address?

Coconuts and Islanders: A Statistics-First Guide to the Boltzmann Distribution

Brian Zhang

DOI: https://doi.org/10.48550/arXiv.1904.04669

2019-04-08

Abstract:The Boltzmann distribution is one of the key equations of thermal physics and is widely used in machine learning as well. Here I derive a Boltzmann distribution in a simple pedagogical example using only tools from a first-year probability course. The example is called "coconuts and islanders" and was taught to me by my father, Shoucheng Zhang (1963 - 2018), to whom these notes are dedicated. By focusing on this simple story, which can be easily simulated on a computer, I aim to provide a more accessible and intuitive presentation of the Boltzmann distribution. Yet I hope this exposition also inspires deep thinking about statistical physics. For instance, I show that the coconuts and islanders story illuminates a connection between the "fundamental assumption of statistical mechanics"---all microstates are equally probable---and the statistical property of detailed balance.

Statistical Mechanics,Physics Education
Boltzmann Generators and the New Frontier of Computational Sampling in Many-Body Systems

Alessandro Coretti,Sebastian Falkner,Jan Weinreich,Christoph Dellago,O. Anatole von Lilienfeld

DOI: https://doi.org/10.25950/bfa99422

2024-04-25

Abstract:The paper by Noé et al. [F. Noé, S. Olsson, J. Köhler and H. Wu, Science, 365:6457 (2019)] introduced the concept of Boltzmann Generators (BGs), a deep generative model that can produce unbiased independent samples of many-body systems. They can generate equilibrium configurations from different metastable states, compute relative stabilities between different structures of proteins or other organic molecules, and discover new states. In this commentary, we motivate the necessity for a new generation of sampling methods beyond molecular dynamics, explain the methodology, and give our perspective on the future role of BGs.

Computational Physics
Transferable Boltzmann Generators

Leon Klein,Frank Noé

2024-06-20

Abstract:The generation of equilibrium samples of molecular systems has been a long-standing problem in statistical physics. Boltzmann Generators are a generative machine learning method that addresses this issue by learning a transformation via a normalizing flow from a simple prior distribution to the target Boltzmann distribution of interest. Recently, flow matching has been employed to train Boltzmann Generators for small molecular systems in Cartesian coordinates. We extend this work and propose a first framework for Boltzmann Generators that are transferable across chemical space, such that they predict zero-shot Boltzmann distributions for test molecules without being retrained for these systems. These transferable Boltzmann Generators allow approximate sampling from the target distribution of unseen systems, as well as efficient reweighting to the target Boltzmann distribution. The transferability of the proposed framework is evaluated on dipeptides, where we show that it generalizes efficiently to unseen systems. Furthermore, we demonstrate that our proposed architecture enhances the efficiency of Boltzmann Generators trained on single molecular systems.

Machine Learning,Chemical Physics,Computational Physics
A Boltzmann generator for the isobaric-isothermal ensemble

Steyn van Leeuwen,Alberto Pérez de Alba Ortíz,Marjolein Dijkstra

2023-05-15

Abstract:Boltzmann generators (BGs) are now recognized as forefront generative models for sampling equilibrium states of many-body systems in the canonical ensemble, as well as for calculating the corresponding Helmholtz free energy. Furthermore, BGs can potentially provide a notable improvement in efficiency compared to conventional techniques such as molecular dynamics (MD) and Monte Carlo (MC) methods. By sampling from a clustered latent space, BGs can circumvent free-energy barriers and overcome the rare-event problem. However, one major limitation of BGs is their inability to sample across phase transitions between ordered phases. This is due to the fact that new phases may not be commensurate with the box dimensions, which remain fixed in the canonical ensemble. In this work, we present a novel BG model for the isothermal-isobaric (NPT) ensemble, which can successfully overcome this limitation. This unsupervised machine-learning model can sample equilibrium states at various pressures, as well as pressure-driven phase transitions. We demonstrate that the samples generated by this model are in good agreement with those obtained through MD simulations of two model systems. Additionally, we derive an estimate of the Gibbs free energy using samples generated by the NPT BG.

Statistical Mechanics
Efficient and Unbiased Sampling of Boltzmann Distributions via Consistency Models

Fengzhe Zhang,Jiajun He,Laurence I. Midgley,Javier Antorán,José Miguel Hernández-Lobato

2024-09-11

Abstract:Diffusion models have shown promising potential for advancing Boltzmann Generators. However, two critical challenges persist: (1) inherent errors in samples due to model imperfections, and (2) the requirement of hundreds of functional evaluations (NFEs) to achieve high-quality samples. While existing solutions like importance sampling and distillation address these issues separately, they are often incompatible, as most distillation models lack the necessary density information for importance sampling. This paper introduces a novel sampling method that effectively combines Consistency Models (CMs) with importance sampling. We evaluate our approach on both synthetic energy functions and equivariant n-body particle systems. Our method produces unbiased samples using only 6-25 NFEs while achieving a comparable Effective Sample Size (ESS) to Denoising Diffusion Probabilistic Models (DDPMs) that require approximately 100 NFEs.

Machine Learning
Accelerate Langevin Sampling with Birth-Death process and Exploration Component

Lezhi Tan,Jianfeng Lu

2023-05-07

Abstract:Sampling a probability distribution with known likelihood is a fundamental task in computational science and engineering. Aiming at multimodality, we propose a new sampling method that takes advantage of both birth-death process and exploration component. The main idea of this method is \textit{look before you leap}. We keep two sets of samplers, one at warmer temperature and one at original temperature. The former one serves as pioneer in exploring new modes and passing useful information to the other, while the latter one samples the target distribution after receiving the information. We derive a mean-field limit and show how the exploration process determines sampling efficiency. Moreover, we prove exponential asymptotic convergence under mild assumption. Finally, we test on experiments from previous literature and compared our methodology to previous ones.

Computation,Machine Learning,Probability,Statistics Theory
Sampling from Boltzmann densities with physics informed low-rank formats

Paul Hagemann,Janina Schütte,David Sommer,Martin Eigel,Gabriele Steidl

2024-12-11

Abstract:Our method proposes the efficient generation of samples from an unnormalized Boltzmann density by solving the underlying continuity equation in the low-rank tensor train (TT) format. It is based on the annealing path commonly used in MCMC literature, which is given by the linear interpolation in the space of energies. Inspired by Sequential Monte Carlo, we alternate between deterministic time steps from the TT representation of the flow field and stochastic steps, which include Langevin and resampling steps. These adjust the relative weights of the different modes of the target distribution and anneal to the correct path distribution. We showcase the efficiency of our method on multiple numerical examples.

Machine Learning,Optimization and Control
BNEM: A Boltzmann Sampler Based on Bootstrapped Noised Energy Matching

RuiKang OuYang,Bo Qiang,José Miguel Hernández-Lobato

2024-10-10

Abstract:Developing an efficient sampler capable of generating independent and identically distributed (IID) samples from a Boltzmann distribution is a crucial challenge in scientific research, e.g. molecular dynamics. In this work, we intend to learn neural samplers given energy functions instead of data sampled from the Boltzmann distribution. By learning the energies of the noised data, we propose a diffusion-based sampler, Noised Energy Matching, which theoretically has lower variance and more complexity compared to related works. Furthermore, a novel bootstrapping technique is applied to NEM to balance between bias and variance. We evaluate NEM and BNEM on a 2-dimensional 40 Gaussian Mixture Model (GMM) and a 4-particle double-well potential (DW-4). The experimental results demonstrate that BNEM can achieve state-of-the-art performance while being more robust.

Machine Learning,Artificial Intelligence,Computation
A short derivation of Boltzmann distribution and Gibbs entropy formula from the fundamental postulate

Didier Lairez

DOI: https://doi.org/10.48550/arXiv.2211.02455

2023-02-02

Abstract:Introducing the Boltzmann distribution very early in a statistical thermodynamics course (in the spirit of Feynmann) has many didactic advantages, in particular that of easily deriving the Gibbs entropy formula. In this note, a short derivation is proposed from the fundamental postulate of statistical mechanics and basics calculations accessible to undergraduate students.

Statistical Mechanics,Chemical Physics
Stein Boltzmann Sampling: A Variational Approach for Global Optimization

Gaëtan Serré,Argyris Kalogeratos,Nicolas Vayatis

2024-06-03

Abstract:In this paper, we present a flow-based method for global optimization of continuous Sobolev functions, called Stein Boltzmann Sampling (SBS). SBS initializes uniformly a number of particles representing candidate solutions, then uses the Stein Variational Gradient Descent (SVGD) algorithm to sequentially and deterministically move those particles in order to approximate a target distribution whose mass is concentrated around promising areas of the domain of the optimized function. The target is chosen to be a properly parametrized Boltzmann distribution. For the purpose of global optimization, we adapt the generic SVGD theoretical framework allowing to address more general target distributions over a compact subset of $\mathbb{R}^d$, and we prove SBS's asymptotic convergence. In addition to the main SBS algorithm, we present two variants: the SBS-PF that includes a particle filtering strategy, and the SBS-HYBRID one that uses SBS or SBS-PF as a continuation after other particle- or distribution-based optimization methods. A detailed comparison with state-of-the-art methods on benchmark functions demonstrates that SBS and its variants are highly competitive, while the combination of the two variants provides the best trade-off between accuracy and computational cost.

Optimization and Control,Machine Learning
On the origin of the Boltzmann distribution

Fedor Sandomirskiy,Omer Tamuz

2023-07-21

Abstract:The Boltzmann distribution is used in statistical mechanics to describe the distribution of states in systems with a given temperature. We give a novel characterization of this distribution as the unique one satisfying independence for uncoupled systems. The theorem boils down to a statement about symmetries of the convolution semigroup of finitely supported probability measures on the natural numbers, or, alternatively, about symmetries of the multiplicative semigroup of polynomials with non-negative coefficients.

Probability,Statistical Mechanics,Group Theory
Flow Perturbation to Accelerate Unbiased Sampling of Boltzmann distribution

Xin Peng,Ang Gao

2024-07-27

Abstract:Flow-based generative models have been employed for sampling the Boltzmann distribution, but their application to high-dimensional systems is hindered by the significant computational cost of obtaining the Jacobian of the flow. To overcome this challenge, we introduce the flow perturbation method, which incorporates optimized stochastic perturbations into the flow. By reweighting trajectories generated by the perturbed flow, our method achieves unbiased sampling of the Boltzmann distribution with orders of magnitude speedup compared to both brute force Jacobian calculations and the Hutchinson estimator. Notably, it accurately sampled the Chignolin protein with all atomic Cartesian coordinates explicitly represented, which, to our best knowledge, is the largest molecule ever Boltzmann sampled in such detail using generative models.

Machine Learning,Chemical Physics
A Practical Diffusion Path for Sampling

Omar Chehab,Anna Korba

2024-06-20

Abstract:Diffusion models are state-of-the-art methods in generative modeling when samples from a target probability distribution are available, and can be efficiently sampled, using score matching to estimate score vectors guiding a Langevin process. However, in the setting where samples from the target are not available, e.g. when this target's density is known up to a normalization constant, the score estimation task is challenging. Previous approaches rely on Monte Carlo estimators that are either computationally heavy to implement or sample-inefficient. In this work, we propose a computationally attractive alternative, relying on the so-called dilation path, that yields score vectors that are available in closed-form. This path interpolates between a Dirac and the target distribution using a convolution. We propose a simple implementation of Langevin dynamics guided by the dilation path, using adaptive step-sizes. We illustrate the results of our sampling method on a range of tasks, and shows it performs better than classical alternatives.

Machine Learning
Energy based diffusion generator for efficient sampling of Boltzmann distributions

Yan Wang,Ling Guo,Hao Wu,Tao Zhou

2024-09-14

Abstract:Sampling from Boltzmann distributions, particularly those tied to high-dimensional and complex energy functions, poses a significant challenge in many fields. In this work, we present the Energy-Based Diffusion Generator (EDG), a novel approach that integrates ideas from variational autoencoders and diffusion models. EDG leverages a decoder to transform latent variables from a simple distribution into samples approximating the target Boltzmann distribution, while the diffusion-based encoder provides an accurate estimate of the Kullback-Leibler divergence during training. Notably, EDG is simulation-free, eliminating the need to solve ordinary or stochastic differential equations during training. Furthermore, by removing constraints such as bijectivity in the decoder, EDG allows for flexible network design. Through empirical evaluation, we demonstrate the superior performance of EDG across a variety of complex distribution tasks, outperforming existing methods.

Machine Learning,Computation
Geometrical derivation of the Boltzmann factor

Ricardo Lopez-Ruiz,Jaime Sanudo,Xavier Calbet

DOI: https://doi.org/10.1119/1.2907776

2007-07-27

Abstract:We show that the Boltzmann factor has a geometrical origin. Its derivation follows from the microcanonical picture. The Maxwell-Boltzmann distribution or the wealth distribution in human society are some direct applications of this new interpretation.

Chaotic Dynamics,Statistical Mechanics,Multiagent Systems,Physics and Society
A Stochastic Calculus Approach to Boltzmann Transport

J. Darby Smith,Rich Lehoucq,Brian Franke

DOI: https://doi.org/10.1080/00295639.2024.2350086

IF: 1.2

2024-06-20

Nuclear Science and Engineering

Abstract:Traditional Monte Carlo methods for particle transport utilize source iteration to express the solution, the flux density, of the transport equation as a Neumann series. Our contribution is to show that the particle paths simulated within source iteration are associated with the adjoint flux density and the adjoint particle paths are associated with the flux density. We make our assertion rigorous through the use of stochastic calculus by representing the particle path used in source iteration as a solution to a stochastic differential equation (SDE). The solution to the adjoint Boltzmann equation is then expressed in terms of the same SDE, and the solution to the Boltzmann equation is expressed in terms of the SDE associated with the adjoint particle process. An important consequence is that the particle paths used within source iteration simultaneously provide Monte Carlo samples of the flux density and adjoint flux density in the detector and source regions, respectively. The significant practical implication is that particle trajectories can be reused to obtain both forward and adjoint quantities of interest. To the best our knowledge, the reuse of entire particles paths has not appeared in the literature. Monte Carlo simulations are presented to support the reuse of the particle paths.

nuclear science & technology
A macroscopic device described by a Boltzmann-like distribution

C. Stan,G. Whitesides,S. Tricard,E. Shakhnovich

DOI: https://doi.org/10.1039/C3SM27385G

2013-04-02

Abstract:Equilibrium thermodynamic phenomena such as the Maxwell–Boltzmann distribution of molecular velocities are rare in systems of macroscopic particles interacting by mechanical collisions. This paper reports a system composed of millimeter-sized polymer objects that under mechanical agitation exhibits a “discretization” of the configurations of the system, and has a distribution of the probabilities of these configurations that is analogous to a Boltzmann distribution. The system is composed of spheres and a three-link chain on a bounded horizontal surface, shaken with an aperiodic but not completely random horizontal motion. Experiments were performed at different strengths of agitation (quantified by the frequency of agitation, f, at constant amplitude) and densities of spheres (quantified by the filling ratio, FR). The chain was typically found in one of three conformations – extended, single folded, and double folded – because, under collisions with the spheres, adjacent links were stable mechanically only when fully extended or fully folded. The probabilities of the different conformations of the chain could be described by a Boltzmann distribution in which the “temperature” depended on f and the “energies” of conformations on FR. The predictions of the Boltzmann formula using empirically determined “temperatures” and “energies” agreed with measurements within two experimental standard deviations in 47 out of 48 experiments.

Physics,Materials Science
A gradient flow approach to the Boltzmann equation

Matthias Erbar

2023-06-13

Abstract:We show that the spatially homogeneous Boltzmann equation evolves as the gradient flow of the entropy with respect to a suitable geometry on the space of probability measures which takes the collision process into account. This gradient flow structure allows to give a new proof for the convergence of Kac's random walk to the homogeneous Boltzmann equation, exploiting the stability of gradient flows.

Analysis of PDEs
Chaotic Boltzmann machines with two elements

Hideyuki Suzuki

DOI: https://doi.org/10.48550/arXiv.1501.07039

2015-01-28

Chaotic Dynamics

Abstract:In this brief note, we show that chaotic Boltzmann machines truly yield samples from the probabilistic distribution of the corresponding Boltzmann machines if they are composed of only two elements. This note is an English translation (with slight modifications) of the article originally written in Japanese [H. Suzuki, Seisan Kenkyu 66 (2014), 315-316].
The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

Alexandre Bouchard-Côté,Sebastian J. Vollmer,Arnaud Doucet

DOI: https://doi.org/10.1080/01621459.2017.1294075

IF: 4.369

2018-04-03

Journal of the American Statistical Association

Abstract:Many Markov chain Monte Carlo techniques currently available rely on discrete-time reversible Markov processes whose transition kernels are variations of the Metropolis–Hastings algorithm. We explore and generalize an alternative scheme recently introduced in the physics literature (Peters and de With 2012) where the target distribution is explored using a continuous-time nonreversible piecewise-deterministic Markov process. In the Metropolis–Hastings algorithm, a trial move to a region of lower target density, equivalently of higher “energy,” than the current state can be rejected with positive probability. In this alternative approach, a particle moves along straight lines around the space and, when facing a high energy barrier, it is not rejected but its path is modified by bouncing against this barrier. By reformulating this algorithm using inhomogeneous Poisson processes, we exploit standard sampling techniques to simulate exactly this Markov process in a wide range of scenarios of interest. Additionally, when the target distribution is given by a product of factors dependent only on subsets of the state variables, such as the posterior distribution associated with a probabilistic graphical model, this method can be modified to take advantage of this structure by allowing computationally cheaper “local” bounces, which only involve the state variables associated with a factor, while the other state variables keep on evolving. In this context, by leveraging techniques from chemical kinetics, we propose several computationally efficient implementations. Experimentally, this new class of Markov chain Monte Carlo schemes compares favorably to state-of-the-art methods on various Bayesian inference tasks, including for high-dimensional models and large datasets. Supplementary materials for this article are available online.

statistics & probability

How to generate an object under an ordinary Boltzmann distribution via an exponential Boltzmann sampler

Coconuts and Islanders: A Statistics-First Guide to the Boltzmann Distribution

Boltzmann Generators and the New Frontier of Computational Sampling in Many-Body Systems

Transferable Boltzmann Generators

A Boltzmann generator for the isobaric-isothermal ensemble

Efficient and Unbiased Sampling of Boltzmann Distributions via Consistency Models

Accelerate Langevin Sampling with Birth-Death process and Exploration Component

Sampling from Boltzmann densities with physics informed low-rank formats

BNEM: A Boltzmann Sampler Based on Bootstrapped Noised Energy Matching

A short derivation of Boltzmann distribution and Gibbs entropy formula from the fundamental postulate

Stein Boltzmann Sampling: A Variational Approach for Global Optimization

On the origin of the Boltzmann distribution

Flow Perturbation to Accelerate Unbiased Sampling of Boltzmann distribution

A Practical Diffusion Path for Sampling

Energy based diffusion generator for efficient sampling of Boltzmann distributions

Geometrical derivation of the Boltzmann factor

A Stochastic Calculus Approach to Boltzmann Transport

A macroscopic device described by a Boltzmann-like distribution

A gradient flow approach to the Boltzmann equation

Chaotic Boltzmann machines with two elements

The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method