Abstract:Associative memories in the brain receive and store patterns of activity registered by the sensory neurons, and are able to retrieve them when necessary. Due to their importance in human intelligence, computational models of associative memories have been developed for several decades now. They include autoassociative memories, which allow for storing data points and retrieving a stored data point $s$ when provided with a noisy or partial variant of $s$, and heteroassociative memories, able to store and recall multi-modal data. In this paper, we present a novel neural model for realizing associative memories, based on a hierarchical generative network that receives external stimuli via sensory neurons. This model is trained using predictive coding, an error-based learning algorithm inspired by information processing in the cortex. To test the capabilities of this model, we perform multiple retrieval experiments from both corrupted and incomplete data points. In an extensive comparison, we show that this new model outperforms in retrieval accuracy and robustness popular associative memory models, such as autoencoders trained via backpropagation, and modern Hopfield networks. In particular, in completing partial data points, our model achieves remarkable results on natural image datasets, such as ImageNet, with a surprisingly high accuracy, even when only a tiny fraction of pixels of the original images is presented. Furthermore, we show that this method is able to handle multi-modal data, retrieving images from descriptions, and vice versa. We conclude by discussing the possible impact of this work in the neuroscience community, by showing that our model provides a plausible framework to study learning and retrieval of memories in the brain, as it closely mimics the behavior of the hippocampus as a memory index and generative model.

Dense Associative Memory for Pattern Recognition

Dense Associative Memory Through the Lens of Random Features

Large Associative Memory Problem in Neurobiology and Machine Learning

Dense Associative Memory is Robust to Adversarial Inputs

A New Pattern Associative Memory Model for Image Recognition Based on Hebb Rules and Dot Product

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

Coexistence of Cyclic Sequential Pattern Recognition and Associative Memory in Neural Networks by Attractor Mechanisms

An effective associative memory for pattern recognition

Pattern Storage and Hopfield Neural Associative Memory with Hidden Structure.

The Exponential Capacity of Dense Associative Memories

Associative Memories via Predictive Coding

Exponential Capacity of Dense Associative Memories

Semantically-correlated memories in a dense associative model

A Novel Continuous-Time Neural Network for Realizing Associative Memory

Improved Robustness and Hyperparameter Selection in the Dense Associative Memory

Competitive learning to generate sparse representations for associative memory

Classification and Generation of real-world data with an Associative Memory Model

A Hopfield-like model with complementary encodings of memories

Memory Dynamics in Attractor Networks with Saliency Weights.

Sequential Learning in the Dense Associative Memory

Associative Memory Using Dictionary Learning and Expander Decoding