Abstract:Recently, Hopfield and Krotov introduced the concept of {\em dense associative memories} [DAM] (close to spin-glasses with $P$-wise interactions in a disordered statistical mechanical jargon): they proved a number of remarkable features these networks share and suggested their use to (partially) explain the success of the new generation of Artificial Intelligence. Thanks to a remarkable ante-litteram analysis by Baldi \& Venkatesh, among these properties, it is known these networks can handle a maximal amount of stored patterns $K$ scaling as $K \sim N^{P-1}$.\\ In this paper, once introduced a {\em minimal dense associative network} as one of the most elementary cost-functions falling in this class of DAM, we sacrifice this high-load regime -namely we force the storage of {\em solely} a linear amount of patterns, i.e. $K = \alpha N$ (with $\alpha>0$)- to prove that, in this regime, these networks can correctly perform pattern recognition even if pattern signal is $O(1)$ and is embedded in a sea of noise $O(\sqrt{N})$, also in the large $N$ limit. To prove this statement, by extremizing the quenched free-energy of the model over its natural order-parameters (the various magnetizations and overlaps), we derived its phase diagram, at the replica symmetric level of description and in the thermodynamic limit: as a sideline, we stress that, to achieve this task, aiming at cross-fertilization among disciplines, we pave two hegemon routes in the statistical mechanics of spin glasses, namely the replica trick and the interpolation technique.\\ Both the approaches reach the same conclusion: there is a not-empty region, in the noise-$T$ vs load-$\alpha$ phase diagram plane, where these networks can actually work in this challenging regime; in particular we obtained a quite high critical (linear) load in the (fast) noiseless case resulting in $\lim_{\beta \to \infty}\alpha_c(\beta)=0.65$.

About the ergodic regime in the analogical Hopfield neural networks. Moments of the partition function

The thermodynamic limit in mean field neural networks

Storage and Learning phase transitions in the Random-Features Hopfield Model

A generalized Hopfield model to store and retrieve mismatched memory patterns

Effect of spatial correlations on Hopfield Neural Network and Dense Associative Memories

A new mechanical approach to handle generalized Hopfield neural networks

About the de Almeida–Thouless line in neural networks

Hebbian Learning from First Principles

Out-of-equilibrium dynamics of the Hopfield model in its spin-glass phase

Generalized Ising Model on a Scale-Free Network: An Interplay of Power Laws

About the de Almeida-Thouless line in neural networks

Nonergodic delocalized paramagnetic states in quantum neural networks

Anomalous thermalization in ergodic systems

Replica symmetry breaking in neural networks: a few steps toward rigorous results

Retrieval dynamics of associative memory of the Hopfield type

Replica symmetry breaking in supervised and unsupervised Hebbian networks

Interpolating between boolean and extremely high noisy patterns through Minimal Dense Associative Memories

On the equivalence of Hopfield networks and Boltzmann Machines

Monotonicity of the logarithmic energy for random matrices

Universality of the mean-field equations of networks of Hopfield-like neurons

Dreaming neural networks: Forgetting spurious memories and reinforcing pure ones