Abstract:Associative memories in the brain receive and store patterns of activity registered by the sensory neurons, and are able to retrieve them when necessary. Due to their importance in human intelligence, computational models of associative memories have been developed for several decades now. They include autoassociative memories, which allow for storing data points and retrieving a stored data point $s$ when provided with a noisy or partial variant of $s$, and heteroassociative memories, able to store and recall multi-modal data. In this paper, we present a novel neural model for realizing associative memories, based on a hierarchical generative network that receives external stimuli via sensory neurons. This model is trained using predictive coding, an error-based learning algorithm inspired by information processing in the cortex. To test the capabilities of this model, we perform multiple retrieval experiments from both corrupted and incomplete data points. In an extensive comparison, we show that this new model outperforms in retrieval accuracy and robustness popular associative memory models, such as autoencoders trained via backpropagation, and modern Hopfield networks. In particular, in completing partial data points, our model achieves remarkable results on natural image datasets, such as ImageNet, with a surprisingly high accuracy, even when only a tiny fraction of pixels of the original images is presented. Furthermore, we show that this method is able to handle multi-modal data, retrieving images from descriptions, and vice versa. We conclude by discussing the possible impact of this work in the neuroscience community, by showing that our model provides a plausible framework to study learning and retrieval of memories in the brain, as it closely mimics the behavior of the hippocampus as a memory index and generative model.

A Hopfield-like model with complementary encodings of memories

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models

Distinguishing examples while building concepts in hippocampal and artificial networks

Associative Memories via Predictive Coding

A Hierarchically Organized Memory Model With Temporal Population Coding

Spiking Neuron Based Cognitive Memory Model

Simplicial Hopfield networks

Large Associative Memory Problem in Neurobiology and Machine Learning

A Hierarchical Organized Memory Model Using Spiking Neurons

A Computationally Efficient Associative Memory Model of Hippocampus CA3 by Spiking Neurons

Storage Capacity of the Hopfield Network Associative Memory

Dense Associative Memory for Pattern Recognition

Modern Hopfield Networks meet Encoded Neural Representations -- Addressing Practical Considerations

Pattern Storage and Hopfield Neural Associative Memory with Hidden Structure.

Long Sequence Hopfield Memory

Memory Dynamics in Attractor Networks with Saliency Weights.

Semantically-correlated memories in a dense associative model

Long sequence Hopfield memory*

Synaptic Conditions for Auto-Associative Memory Storage and Pattern Completion in Jensen Et Al.’s Model of Hippocampal Area CA3