Abstract:Improved mean-field technics are a central theme of statistical physics methods applied to inference and learning. We revisit here some of these methods using high-temperature expansions for disordered systems initiated by Plefka, Georges and Yedidia. We derive the Gibbs free entropy and the subsequent self-consistent equations for a generic class of statistical models with correlated matrices and show in particular that many classical approximation schemes, such as adaptive TAP, Expectation-Consistency, or the approximations behind the Vector Approximate Message Passing algorithm all rely on the same assumptions, that are also at the heart of high-temperature expansions. We focus on the case of rotationally invariant random coupling matrices in the `high-dimensional' limit in which the number of samples and the dimension are both large, but with a fixed ratio. This encapsulates many widely studied models, such as Restricted Boltzmann Machines or Generalized Linear Models with correlated data matrices. In this general setting, we show that all the approximation schemes described before are equivalent, and we conjecture that they are exact in the thermodynamic limit in the replica symmetric phases. We achieve this conclusion by resummation of the infinite perturbation series, which generalizes a seminal result of Parisi and Potters. A rigorous derivation of this conjecture is an interesting mathematical challenge. On the way to these conclusions, we uncover several diagrammatical results in connection with free probability and random matrix theory, that are interesting independently of the rest of our work.

High Temperature Expansions and Dynamical Systems

High-Temperature Cluster Expansion for Classical and Quantum Spin Lattice Systems With Multi-Body Interactions

Thermal form-factor expansion of the dynamical two-point functions of local operators in integrable quantum chains

High-temperature Expansions and Message Passing Algorithms

X-ray Luminous Radio Supernovae in the Center of M82?

Cumulant expansion for systems with large spins

Single Scale Cluster Expansions with Applications to Many Boson and Unbounded Spin Systems

Thermal form-factor approach to dynamical correlation functions of integrable lattice models

Critical dynamics of long range models on Dynamical Lévy Lattices

Discrete Laplacian Thermostat for Spin Systems with Conserved Dynamics

Effective Dynamics of Local Observables for Extended Fermi Gases in the High-Density Regime

Thermodynamic formalism for piecewise expanding maps in finite dimension

Adiabatic Evolution of Low-Temperature Many-Body Systems

Dynamics of the spin-half Heisenberg chain at intermediate temperatures

A study of the (m,d,N)=(1,3,2) Lifshitz point and of the three- dimensional XY universality class by high-temperature bivariate series for the XY models with anisotropic competing interactions

On the high-temperature expansion of the thermal energy on Einstein cylinders

High-fugacity expansion, Lee-Yang zeros and order-disorder transitions in hard-core lattice systems

Low-$T$ asymptotic expansion of the solution to the Yang-Yang equation

Recursive algorithm for generating high-temperature expansions for spin systems and the chiral non-linear susceptibility