On the GREM approximation of TAP free energies

Giulia Sebastiani,Marius Alexander Schmidt

2024-01-24

Abstract:We establish both a Boltzmann-Gibbs principle and a Parisi formula for the limiting free energy of an abstract GREM (Generalized Random Energy Model) which provides an approximation of the TAP (Thouless-Anderson-Palmer) free energies associated to the Sherrington-Kirkpatrick (SK) model.

Disordered Systems and Neural Networks,Probability

What problem does this paper attempt to address?

On the GREM approximation of TAP free energies

Giulia Sebastiani,Marius Alexander Schmidt

DOI: https://doi.org/10.1088/1751-8121/ad653b

2024-07-19

Journal of Physics A Mathematical and Theoretical

Abstract:We establish both a Boltzmann-Gibbs principle and a Parisi formula for the limiting free energy of an abstract GREM (Generalized Random Energy Model) which provides an approximation of the TAP (Thouless-Anderson-Palmer) free energies associated to the Sherrington-Kirkpatrick (SK) model.

physics, multidisciplinary, mathematical
On the TAP equations via the cavity approach in the generic mixed $p$-spin models

Wei-Kuo Chen,Si Tang

DOI: https://doi.org/10.1007/s00220-024-04971-2

2024-03-26

Abstract:In 1977, Thouless, Anderson, and Palmer (TAP) derived a system of consistent equations in terms of the effective magnetization in order to study the free energy in the Sherrington-Kirkpatrick (SK) spin glass model. The solutions to their equations were predicted to contain vital information about the landscapes in the SK Hamiltonian and the TAP free energy and moreover have direct connections to Parisi's replica ansatz. In this work, we aim to investigate the validity of the TAP equations in the generic mixed $p$-spin model. By utilizing the ultrametricity of the overlaps, we show that the TAP equations are asymptotically satisfied by the conditional local magnetizations on the asymptotic pure states.

Probability,Mathematical Physics
On the TAP Equations via the Cavity Approach in the Generic Mixed p-Spin Models

Wei-Kuo Chen,Si Tang

DOI: https://doi.org/10.1007/s00220-024-04971-2

IF: 2.361

2024-03-19

Communications in Mathematical Physics

Abstract:In 1977, Thouless, Anderson, and Palmer (TAP) derived a system of consistent equations in terms of the effective magnetization in order to study the free energy in the Sherrington–Kirkpatrick (SK) spin glass model. The solutions to their equations were predicted to contain vital information about the landscapes in the SK Hamiltonian and the TAP free energy and moreover have direct connections to Parisi's replica ansatz. In this work, we aim to investigate the validity of the TAP equations in the generic mixed p -spin model. By utilizing the ultrametricity of the overlaps, we show that the TAP equations are asymptotically satisfied by the conditional local magnetizations on the asymptotic pure states.

physics, mathematical
Contributions to Random Energy Models

Nabin Kumar Jana

DOI: https://doi.org/10.48550/arXiv.0711.1249

2007-11-08

Abstract:In this thesis, we consider several Random Energy Models. This includes Derrida's Random Energy Model (REM) and Generalized Random Energy Model (GREM) and a nonhierarchical version (BK-GREM) by Bolthausen and Kistler. The limiting free energy in all these models along with Word GREM, a model proposed by us, turn out to be a cute consequence of large deviation principle (LDP). This LDP argument allows us to consider non-Gaussian driving distributions as well as external field. We could also consider random trees as the underlying tree structure in GREM. In all these models, as expected, limiting free energy is not 'universal' unlike the SK model. However it is 'rate specific'. Consideration of non-Gaussian driving distribution as well as different driving distributions for the different levels of the underlying trees in GREM leads to interesting phenomena. For example in REM, if the Hamiltonian is Binomial with parameter $N$ and $p$ then the existence of phase transition depends on the parameter $p$. More precisely, phase transition takes place only when $p>{1/2}$. For another example, consider a 2 level GREM with exponential driving distribution at the first level and Gaussian in the second with equal weights at both the levels. Then even if the limiting ratio for the second level particles, $p_2$ is 0.00001 (very small), the system reduces to a Gaussian REM. On the other hand, if we consider a 2 level GREM with Gaussian driving distribution at the first level and exponential in the second, the system will never reduce to a Gaussian REM. In either case, the system will never reduce to that of an exponential REM. etc.

Probability,Dynamical Systems
Emergence of near-TAP free energy functional in the SK model at high temperature

Véronique Gayrard

2023-06-05

Abstract:We study the SK model at inverse temperature $\beta>0$ and strictly positive field $h>0$ in the region of $(\beta,h)$ where the replica-symmetric formula is valid. An integral representation of the partition function derived from the Hubbard-Stratonovitch transformation combined with a duality formula is used to prove that the infinite volume free energy of the SK model can be expressed as a variational formula on the space of magnetisations, $m$. The resulting free energy functional differs from that of Thouless, Anderson and Palmer (TAP) by the term $-\frac{\beta^2}{4}\left(q-q_{\text{EA}}(m)\right)^2$ where $q_{\text{EA}}(m)$ is the Edwards-Anderson parameter and $q$ is the minimiser of the replica-symmetric formula. Thus, both functionals have the same critical points and take the same value on the subspace of magnetisations satisfying $q_{\text{EA}}(m)=q$. This result is based on an in-depth study of the global maximum of this near-TAP free energy functional using Bolthausen's solutions of the TAP equations, Bandeira & van Handel's bounds on the spectral norm of non-homogeneous Wigner-type random matrices, and Gaussian comparison techniques. It holds for $(\beta,h)$ in a large subregion of the de Almeida and Thouless high-temperature stability region.

Probability
Perturbative Interpretation of Adaptive Thouless-Anderson-Palmer Free Energy

Muneki Yasuda,Chako Takahashi,Kazuyuki Tanaka

DOI: https://doi.org/10.7566/JPSJ.85.075001

2018-05-11

Abstract:In conventional well-known derivation methods for the adaptive Thouless-Anderson-Palmer (TAP) free energy, special assumptions that are difficult to mathematically justify except in some mean-field models, must be made. Here, we present a new adaptive TAP free energy derivation method. Using this derivation technique, without any special assumptions, the adaptive TAP free energy can be simply obtained as a high-temperature expansion of the Gibbs free energy.

Disordered Systems and Neural Networks,Statistical Mechanics
On concavity of TAP free energy in the SK model

Stephan Gufler,Adrien Schertzer,Marius A. Schmidt

2023-07-07

Abstract:We analyse the Hessian of the Thouless-Anderson-Palmer (TAP) free energy for the Sherrington-Kirkpatrick model, below the de Almeida-Thouless line, evaluated in Bolthausen's approximate solutions of the TAP equations. We show that the empirical spectral distribution weakly converges to a measure with negative support below the AT line, an that the support includes zero on the AT line. In this ``macroscopic'' sense, TAP free energy is concave in the order parameter of the theory, i.e. the random spin-magnetisations. This proves a spectral interpretation of the AT line. However, for specific magnetizations, the Hessian of the TAP free energy can have positive outlier eigenvalues. The question whether such outliers may also occur close to the TAP solutions is left open. In a simplified setting where the magnetizations are independent of the disorder, we prove that Plefka's second condition is equivalent to all eigenvalues being negative.

Probability
Dynamical Approach to the TAP Equations for the Sherrington-Kirkpatrick Model

Arka Adhikari,Christian Brennecke,Per von Soosten,Horng-Tzer Yau

DOI: https://doi.org/10.1007/s10955-021-02773-7

2021-02-20

Abstract:We present a new dynamical proof of the Thouless-Anderson-Palmer (TAP) equations for the classical Sherrington-Kirkpatrick spin glass at sufficiently high temperature. In our derivation, the TAP equations are a simple consequence of the decay of the two point correlation functions. The methods can also be used to establish the decay of higher order correlation functions. We illustrate this by proving a suitable decay bound on the three point functions from which we derive an analogue of the TAP equations for the two point functions.

Mathematical Physics,Probability
TAP free energy, spin glasses, and variational inference

Zhou Fan,Song Mei,Andrea Montanari

DOI: https://doi.org/10.48550/arXiv.1808.07890

2020-03-26

Abstract:We consider the Sherrington-Kirkpatrick model of spin glasses with ferromagnetically biased couplings. For a specific choice of the couplings mean, the resulting Gibbs measure is equivalent to the Bayesian posterior for a high-dimensional estimation problem known as `$Z_2$ synchronization'. Statistical physics suggests to compute the expectation with respect to this Gibbs measure (the posterior mean in the synchronization problem), by minimizing the so-called Thouless-Anderson-Palmer (TAP) free energy, instead of the mean field (MF) free energy. We prove that this identification is correct, provided the ferromagnetic bias is larger than a constant (i.e. the noise level is small enough in synchronization). Namely, we prove that the scaled $\ell_2$ distance between any low energy local minimizers of the TAP free energy and the mean of the Gibbs measure vanishes in the large size limit. Our proof technique is based on upper bounding the expected number of critical points of the TAP free energy using the Kac-Rice formula.

Probability,Mathematical Physics,Statistics Theory
TAP variational principle for the constrained overlap multiple spherical Sherrington-Kirkpatrick model

David Belius,Leon Fröber,Justin Ko

2023-04-08

Abstract:Spin glass models involving multiple replicas with constrained overlaps have been studied in [FPV92; PT07; Pan18a]. For the spherical versions of these models [Ko19; Ko20] showed that the limiting free energy is given by a Parisi type minimization. In this work we show that for Sherrington-Kirkpatrick (i.e. 2-spin) interactions, it can also be expressed in terms of a Thouless-Andersson-Palmer (TAP) variational principle. This is only the second spin glass model where a mathematically rigorous TAP computation of the free energy at all temperatures and external fields has been achieved. The variational formula we derive here also confirms that the model is replica symmetric, a fact which is natural but not obviously deducible from its Parisi formula.

Probability,Mathematical Physics
A Constructive Proof of the Spherical Parisi Formula

Brice Huang,Mark Sellke

2024-05-02

Abstract:The Parisi formula for the free energy is among the crown jewels in the theory of spin glasses. We present a simpler proof of the lower bound in the case of the spherical mean-field model. Our method follows the TAP approach developed recently in e.g. (Subag, 2018): we obtain an ultrametric tree of pure states, each with approximately the same free energy as the entire model, which are hierarchically arranged in accordance with the Parisi ansatz. We construct this tree ``layer by layer'' given the minimizer to Parisi's variational problem. On overlap intervals with full RSB, the tree is built by an optimization algorithm due to Subag. On overlap intervals with finite RSB, the tree is constructed by a new truncated second moment argument; a similar argument also characterizes the free energy of the resulting pure states. Notably we do not use the Aizenman--Sims--Starr scheme, and require interpolation bounds only up to the 1RSB level. Our methods also yield results for large deviations of the ground state, including the entire upper tail rate function for all 1RSB models without external field.

Probability,Disordered Systems and Neural Networks,Mathematical Physics
Multireference Generalization of the Weighted Thermodynamic Perturbation Method.

Timothy J Giese,Jinzhe Zeng,Darrin M York

DOI: https://doi.org/10.1021/acs.jpca.2c06201

2022-01-01

Abstract:We describe the generalized weighted thermodynamic perturbation (gwTP) method for estimating the free energy surface of an expensive "high-level" potential energy function from the umbrella sampling performed with multiple inexpensive "low-level" reference potentials. The gwTP method is a generalization of the weighted thermodynamic perturbation (wTP) method developed by Li and co-workers [J. Chem. Theory Comput. 2018, 14, 5583-5596] that uses a single "low-level" reference potential. The gwTP method offers new possibilities in model design whereby the sampling generated from several low-level potentials may be combined (e.g., specific reaction parameter models that might have variable accuracy at different stages of a multistep reaction). The gwTP method is especially well suited for use with machine learning potentials (MLPs) that are trained against computationally expensive ab initio quantum mechanical/molecular mechanical (QM/MM) energies and forces using active learning procedures that naturally produce multiple distinct neural network potentials. Simulations can be performed with greater sampling using the fast MLPs and then corrected to the ab initio level using gwTP. The capabilities of the gwTP method are demonstrated by creating reference potentials based on the MNDO/d and DFTB2/MIO semiempirical models supplemented with the "range-corrected deep potential" (DPRc). The DPRc parameters are trained to ab initio QM/MM data, and the potentials are used to calculate the free energy surface of stepwise mechanisms for nonenzymatic RNA 2'--transesterification model reactions. The extended sampling made possible by the reference potentials allows one to identify unequilibrated portions of the simulations that are not always evident from the short time scale commonly used with ab initio QM/MM potentials. We show that the reference potential approach can yield more accurate ab initio free energy predictions than the wTP method or what can be reasonably afforded from explicit ab initio QM/MM sampling.
Efficient simulation of Grassmann Tensor Product States

Zheng-Cheng Gu

DOI: https://doi.org/10.1103/PhysRevB.88.115139

2013-09-24

Abstract:Recently, the Grassmann-tensor-entanglement renormalization group(GTERG) approach was proposed as a generic variational approach to study strongly correlated boson/fermion systems. However, the weakness of such a simple variational approach is that generic Grassmann tensor product states(GTPS) with large inner dimension $D$ will contain a large number of variational parameters and be hard to be determined through usual minimization procedures. In this paper, we first introduce a standard form of GTPS which significantly simplifies the representations. Then we describe a simple imaginary-time-evolution algorithm to efficiently update the GTPS based on the fermion coherent state representation and show all the algorithm developed for usual tensor product states(TPS) can be implemented for GTPS in a similar way. Finally, we study the environment effect for the GTERG approach and propose a simple method to further improve its accuracy. We demonstrate our algorithms by studying a simple 2D free fermion system on honeycomb lattice, including both off-critical and critical cases.

Strongly Correlated Electrons,Quantum Physics
A simplified Parisi Ansatz II: REM universality

Simone Franchini

2024-08-12

Abstract:In a previous work [A simplified Parisi Ansatz, Franchini, S., Commun. Theor. Phys., 73, 055601 (2021)] we introduced a simple method to compute the Random Overlap Structure of Aizenmann, Simm and Stars and the full-RSB Parisi formula for the Sherrington-Kirckpatrick Model without using replica theory. The method consists in partitioning the system into smaller sub-systems that we call layers, and iterate the Bayes rule. A central ansatz in our derivation was that these layers could be approximated by Random Energy Models of the Derrida type. In this paper we analyze the properties of the interface in detail, and show the equivalence with the Random Energy Model at any temperature.

Statistical Mechanics
Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit

Elena Agliari,Adriano Barra,Brunello Tirozzi

DOI: https://doi.org/10.1088/1742-5468/ab02ef

2019-03-09

Abstract:Restricted Boltzmann machines (RBMs) constitute one of the main models for machine statistical inference and they are widely employed in Artificial Intelligence as powerful tools for (deep) learning. However, in contrast with countless remarkable practical successes, their mathematical formalization has been largely elusive: from a statistical-mechanics perspective these systems display the same (random) Gibbs measure of bi-partite spin-glasses, whose rigorous treatment is notoriously difficult. In this work, beyond providing a brief review on RBMs from both the learning and the retrieval perspectives, we aim to contribute to their analytical investigation, by considering two distinct realizations of their weights (i.e., Boolean and Gaussian) and studying the properties of their related free energies. More precisely, focusing on a RBM characterized by digital couplings, we first extend the Pastur-Shcherbina-Tirozzi method (originally developed for the Hopfield model) to prove the self-averaging property for the free energy, over its quenched expectation, in the infinite volume limit, then we explicitly calculate its simplest approximation, namely its annealed bound. Next, focusing on a RBM characterized by analogical weights, we extend Guerra's interpolating scheme to obtain a control of the quenched free-energy under the assumption of replica symmetry: we get self-consistencies for the order parameters (in full agreement with the existing Literature) as well as the critical line for ergodicity breaking that turns out to be the same obtained in AGS theory. As we discuss, this analogy stems from the slow-noise universality. Finally, glancing beyond replica symmetry, we analyze the fluctuations of the overlaps for an estimate of the (slow) noise affecting the retrieval of the signal, and by a stability analysis we recover the Aizenman-Contucci identities typical of glassy systems.

Physics
Total Energy beyond GW: Exact Results and Guidelines for Approximations

Abdallah El-Sahili,Francesco Sottile,Lucia Reining

DOI: https://doi.org/10.1021/acs.jctc.3c01200

2024-02-09

Journal of Chemical Theory and Computation

Abstract:The total energy and electron addition and removal spectra can, in principle, be obtained exactly from the one-body Green's function (GF). In practice, the GF is obtained from an approximate self-energy. In the framework of many-body perturbation theory, we derive different expressions that are based on an approximate self-energy, but that yield nevertheless, in principle, the exact exchange-correlation contribution to the total energy for any interaction strength. Response functions play a...

chemistry, physical,physics, atomic, molecular & chemical
On the Convexity and Reliability of the Bethe Free Energy Approximation

Harald Leisenberger,Christian Knoll,Franz Pernkopf

2024-05-24

Abstract:The Bethe free energy approximation provides an effective way for relaxing NP-hard problems of probabilistic inference. However, its accuracy depends on the model parameters and particularly degrades if a phase transition in the model occurs. In this work, we analyze when the Bethe approximation is reliable and how this can be verified. We argue and show by experiment that it is mostly accurate if it is convex on a submanifold of its domain, the 'Bethe box'. For verifying its convexity, we derive two sufficient conditions that are based on the definiteness properties of the Bethe Hessian matrix: the first uses the concept of diagonal dominance, and the second decomposes the Bethe Hessian matrix into a sum of sparse matrices and characterizes the definiteness properties of the individual matrices in that sum. These theoretical results provide a simple way to estimate the critical phase transition temperature of a model. As a practical contribution we propose $\texttt{BETHE-MIN}$, a projected quasi-Newton method to efficiently find a minimum of the Bethe free energy.

Machine Learning,Artificial Intelligence
Targeted free energy perturbation revisited: Accurate free energies from mapped reference potentials

Andrea Rizzi,Paolo Carloni,Michele Parrinello

DOI: https://doi.org/10.1021/acs.jpclett.1c02135

2021-08-09

Abstract:We present an approach that extends the theory of targeted free energy perturbation (TFEP) to calculate free energy differences and free energy surfaces at an accurate quantum mechanical level of theory from a cheaper reference potential. The convergence is accelerated by a mapping function that increases the overlap between the target and the reference distributions. Building on recent work, we show that this map can be learned with a normalizing flow neural network, without requiring simulations with the expensive target potential but only a small number of single-point calculations, and, crucially, avoiding the systematic error that was found previously. We validate the method by numerically evaluating the free energy difference in a system with a double-well potential and by describing the free energy landscape of a simple chemical reaction in the gas phase.

Computational Physics,Chemical Physics
Assessing the accuracy and efficiency of free energy differences obtained from reweighted flow-based probabilistic generative models

Matteo Salvalaglio,Michael Shirts,Ahmad Y Sheikh,Yifei Michelle Liu,Nada Mehio,Edgar Olehnovics

DOI: https://doi.org/10.26434/chemrxiv-2024-z9g39

2024-04-22

Abstract:Computing free energy differences between metastable states characterized by non-overlapping Boltzmann distributions is often a computationally intensive endeavour, usually requiring chains of intermediate states to connect these metastable states. Targeted free energy perturbation (TFEP) can significantly lower the computational cost of FEP calculations by choosing a set of invertible maps used to directly transform the distributions of interest, achieving the necessary statistically significant overlaps without sampling any intermediate states. Probabilistic generative models (PGMs) based on normalising-flow architectures can make it much easier via machine learning to train invertible maps needed for TFEP. However, the accuracy and applicability of approaches based on empirically learned maps depend crucially on the choice of reweighting method adopted to estimate the free energy differences. In this work, we assess the accuracy, rate of convergence, and data efficiency of different free energy estimators, including exponential averaging, BAR, and MBAR, in reweighting PGMs trained by maximum likelihood on limited amounts of molecular dynamics data sampled only from end-states of interest. We carry out the comparisons on a set of simple but representative case studies, including conformational ensembles of alanine dipeptide and ibuprofen. Our results indicate that BAR and MBAR are both data efficient and robust, even in the presence of significant model overfitting in the generation of invertible maps. This analysis can serve as a stepping stone for the deployment of efficient and quantitatively accurate ML-based FE calculation methods in complex systems.

Chemistry
Airy-averaged gradient corrections for two-dimensional fermion gases

Martin-Isbjörn Trappe,Yink Loong Len,Hui Khoon Ng,Berthold-Georg Englert

DOI: https://doi.org/10.1016/j.aop.2017.07.020

2017-04-17

Abstract:Building on the discussion in PRA 93, 042510 (2016), we present a systematic derivation of gradient corrections to the kinetic-energy functional and the one-particle density, in particular for two-dimensional systems. We derive the leading gradient corrections from a semiclassical expansion based on Wigner's phase space formalism and demonstrate that the semiclassical kinetic-energy density functional at zero temperature cannot be evaluated unambiguously. In contrast, a density-potential functional description that effectively incorporates interactions provides unambiguous gradient corrections. Employing an averaging procedure that involves Airy functions, thereby partially resumming higher-order gradient corrections, we facilitate a smooth transition of the particle density into the classically forbidden region of arbitrary smooth potentials. We find excellent agreement of the semiclassical Airy-averaged particle densities with the exact densities for very low but finite temperatures, illustrated for a Fermi gas with harmonic potential energy. We furthermore provide criteria for the applicability of the semiclassical expansions at low temperatures. Finally, we derive a well-behaved ground-state kinetic-energy functional, which improves on the Thomas-Fermi approximation.

Quantum Gases,Atomic Physics,Quantum Physics

On the GREM approximation of TAP free energies

On the GREM approximation of TAP free energies

On the TAP equations via the cavity approach in the generic mixed $p$-spin models

On the TAP Equations via the Cavity Approach in the Generic Mixed p-Spin Models

Contributions to Random Energy Models

Emergence of near-TAP free energy functional in the SK model at high temperature

Perturbative Interpretation of Adaptive Thouless-Anderson-Palmer Free Energy

On concavity of TAP free energy in the SK model

Dynamical Approach to the TAP Equations for the Sherrington-Kirkpatrick Model

TAP free energy, spin glasses, and variational inference

TAP variational principle for the constrained overlap multiple spherical Sherrington-Kirkpatrick model

A Constructive Proof of the Spherical Parisi Formula

Multireference Generalization of the Weighted Thermodynamic Perturbation Method.

Efficient simulation of Grassmann Tensor Product States

A simplified Parisi Ansatz II: REM universality

Free energies of Boltzmann Machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit

Total Energy beyond GW: Exact Results and Guidelines for Approximations

On the Convexity and Reliability of the Bethe Free Energy Approximation

Targeted free energy perturbation revisited: Accurate free energies from mapped reference potentials

Assessing the accuracy and efficiency of free energy differences obtained from reweighted flow-based probabilistic generative models

Airy-averaged gradient corrections for two-dimensional fermion gases