Abstract:Improved mean-field technics are a central theme of statistical physics methods applied to inference and learning. We revisit here some of these methods using high-temperature expansions for disordered systems initiated by Plefka, Georges and Yedidia. We derive the Gibbs free entropy and the subsequent self-consistent equations for a generic class of statistical models with correlated matrices and show in particular that many classical approximation schemes, such as adaptive TAP, Expectation-Consistency, or the approximations behind the Vector Approximate Message Passing algorithm all rely on the same assumptions, that are also at the heart of high-temperature expansions. We focus on the case of rotationally invariant random coupling matrices in the `high-dimensional' limit in which the number of samples and the dimension are both large, but with a fixed ratio. This encapsulates many widely studied models, such as Restricted Boltzmann Machines or Generalized Linear Models with correlated data matrices. In this general setting, we show that all the approximation schemes described before are equivalent, and we conjecture that they are exact in the thermodynamic limit in the replica symmetric phases. We achieve this conclusion by resummation of the infinite perturbation series, which generalizes a seminal result of Parisi and Potters. A rigorous derivation of this conjecture is an interesting mathematical challenge. On the way to these conclusions, we uncover several diagrammatical results in connection with free probability and random matrix theory, that are interesting independently of the rest of our work.

Random matrix ensemble for the covariance matrix of Ornstein-Uhlenbeck processes with heterogeneous temperatures

Localization in covariance matrices of coupled heterogenous Ornstein-Uhlenbeck processes

The multivariate fractional Ornstein-Uhlenbeck process

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS

Critical Statistics in Quantum Chaos and Calogero-Sutherland Model at Finite Temperature

Eigenvalue spectra of finely structured random matrices

Properties of stable ensembles of Euclidean random matrices

The Spectral Boundary of Block Structured Random Matrices

Random matrix universality in dynamical correlation functions at late times

Random-matrix model for thermalization

Regularized fractional Ornstein-Uhlenbeck processes and their relevance to the modeling of fluid turbulence

An elementary mean-field approach to the spectral densities of random matrix ensembles

Random matrices theory elucidates the critical nonequilibrium phenomena

Tempered stable distributions and finite variation Ornstein-Uhlenbeck processes

High-temperature Expansions and Message Passing Algorithms

Strong convergence of the empirical spectral distribution of a unified matrix model

Levy scaling in random walks with fluctuating variance

Efficient Computational method using random matrices describing critical thermodynamics

Riemann-Liouville fractional Brownian motion with random Hurst exponent