Abstract:Dense Associative Memories or modern Hopfield networks permit storage and reliable retrieval of an exponentially large (in the dimension of feature space) number of memories. At the same time, their naive implementation is non-biological, since it seemingly requires the existence of many-body synaptic junctions between the neurons. We show that these models are effective descriptions of a more microscopic (written in terms of biological degrees of freedom) theory that has additional (hidden) neurons and only requires two-body interactions between them. For this reason our proposed microscopic theory is a valid model of large associative memory with a degree of biological plausibility. The dynamics of our network and its reduced dimensional equivalent both minimize energy (Lyapunov) functions. When certain dynamical variables (hidden neurons) are integrated out from our microscopic theory, one can recover many of the models that were previously discussed in the literature, e.g. the model presented in "Hopfield Networks is All You Need" paper. We also provide an alternative derivation of the energy function and the update rule proposed in the aforementioned paper and clarify the relationships between various models of this class.

A New Approach to the Design of Hopfield Associative Memory

Design of Complex-valued Hopfield Associative Memory Based on Prespecified Attractive Domain

Designation of complex-valued hopfield associative memory with prespecified attractive domain

A neuron-weighted learning algorithm and its hardware implementation in associative memories

Designing Bidirectional Associative Memories with Optimal Stability

Pattern Storage and Hopfield Neural Associative Memory with Hidden Structure.

Weighted Learning of Bidirectional Associative Memories by Global Minimization

Learning associative memories by error backpropagation.

Algorithm for Optimizing Bipolar Interconnection Weights with Applications in Associative Memories and Multitarget Classification.

Genetic Approach to the Design of Bidirectional Associative Memory.

A Synthesis Method for the Complex-Valued Associative Memory Constrained by the Attractive Domain

A Memristive Hopfield Network for Associative Memory

Improved Robustness and Hyperparameter Selection in the Dense Associative Memory

A Novel Association Model Optimized By Neural-Net

Storage Capacity of the Hopfield Network Associative Memory

The Hopfield Neural Networks Are Applied In The Conceptual Design For Example

Dense Associative Memory Through the Lens of Random Features

Analysis and Synthesis for A Class of Complex-Valued Associative Memories

The capacity of the Hopfield associative memory

Large Associative Memory Problem in Neurobiology and Machine Learning

Two-dimensional inner-product model for auto-hetero-associative memory and its new optoelectronic implementation