Abstract:We consider Ising mixed $p$-spin glasses at high-temperature and without external field, and study the problem of sampling from the Gibbs distribution $\mu$ in polynomial time. We develop a new sampling algorithm with complexity of the same order as evaluating the gradient of the Hamiltonian and, in particular, at most linear in the input size. We prove that, at sufficiently high-temperature, it produces samples from a distribution $\mu^{alg}$ which is close in normalized Wasserstein distance to $\mu$. Namely, there exists a coupling of $\mu$ and $\mu^{alg}$ such that if $({\boldsymbol x},{\boldsymbol x}^{alg})\in\{-1,+1\}^n\times \{-1,+1\}^n$ is a pair drawn from this coupling, then $n^{-1}{\mathbb E}\{\|{\boldsymbol x}-{\boldsymbol x}^{alg}\|_2^2\}=o_n(1)$. For the case of the Sherrington-Kirkpatrick model, our algorithm succeeds in the full replica-symmetric phase. We complement this result with a negative one for sampling algorithms satisfying a certain `stability' property, which is verified by many standard techniques. No stable algorithm can approximately sample at temperatures below the onset of shattering, even under the normalized Wasserstein metric. Further, no algorithm can sample at temperatures below the onset of replica symmetry breaking. Our sampling method implements a discretized version of a diffusion process that has become recently popular in machine learning under the name of `denoising diffusion.' We derive the same process from the general construction of stochastic localization. Implementing the diffusion process requires to efficiently approximate the mean of the tilted measure. To this end, we use an approximate message passing algorithm that, as we prove, achieves sufficiently accurate mean estimation.

Efficient sampling from the Bingham distribution

IID Sampling from Intractable Distributions

Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods

IID Sampling from Doubly Intractable Distributions

Efficient Sampling on Riemannian Manifolds via Langevin MCMC

Perfect Sampling for Hard Spheres from Strong Spatial Mixing

Sampling from the Sherrington-Kirkpatrick Gibbs measure via algorithmic stochastic localization

Sampling from Mean-Field Gibbs Measures via Diffusion Processes

Sequential sampling for a ranking and selection problem with exponential sampling distributions

Using a Non-Commutative Bernstein Bound to Approximate Some Matrix Algorithms in the Spectral Norm

Low-rank Bayesian matrix completion via geodesic Hamiltonian Monte Carlo on Stiefel manifolds

Geodesic slice sampling on the sphere

Fused $L_{1/2}$ prior for large scale linear inverse problem with Gibbs bouncy particle sampler

Sampling scattered data with Bernstein polynomials: stochastic and deterministic error estimates

Stochastic Computation of Barycentric Coordinates

Faster Sampling from Log-Concave Densities over Polytopes via Efficient Linear Solvers

Sampling from Spherical Spin Glasses in Total Variation via Algorithmic Stochastic Localization

Rejection and Importance Sampling based Perfect Simulation for Gibbs hard-sphere models

Stable and Accurate Least Squares Radial Basis Function Approximations on Bounded Domains

sampling with probability matching

Symmetrized Bingham Distribution for Representing Texture: Parameter estimation with respect to crystal and sample symmetries