Abstract:Associative memory or content addressable memory is an important component function in computer science and information processing and is a key concept in cognitive and computational brain science. Many different neural network architectures and learning rules have been proposed to model associative memory of the brain while investigating key functions like pattern completion and rivalry, noise reduction, and storage capacity. A less investigated but important function is prototype extraction where the training set comprises pattern instances generated by distorting prototype patterns and the task of the trained network is to recall the correct prototype pattern given a new instance. In this paper we characterize these different aspects of associative memory performance and benchmark six different learning rules on storage capacity and prototype extraction. We consider only models with Hebbian plasticity that operate on sparse distributed representations with unit activities in the interval [0,1]. We evaluate both non-modular and modular network architectures and compare performance when trained and tested on different kinds of sparse random binary pattern sets, including correlated ones. We show that covariance learning has a robust but low storage capacity under these conditions and that the Bayesian Confidence Propagation learning rule (BCPNN) is superior with a good margin in all cases except one, reaching a three times higher composite score than the second best learning rule tested.

Learning algorithm of associative memory with optimal stability of pattern components

Designing Bidirectional Associative Memories with Optimal Stability

Optimal learning algorithm in bidirectional associative memories

Efficient learning algorithm for associative memory neural network

Weighted Learning of Bidirectional Associative Memories by Global Minimization

A neuron-weighted learning algorithm and its hardware implementation in associative memories

Analysis and Optimal Design of Continuous Neural Networks with Applications to Associative Memory

Analysis and Design of Associative Memories Based on Stability of Cellular Neural Networks

Analysis and optimal design of nonlinear continuous associative memory neural networks

The optimal encodings for biased association in linear associative memories

Unified approach for applications of oscillatory associative-memory networks with error-free retrieval

Pattern Memory And Acquisition Based On Stability Of Cellular Neural Networks

Optimum associative neural network utilizing maximum likelihood

Benchmarking Hebbian learning rules for associative memory

Effect of Similarity Between Patterns in Associative Memory

Optimal Design for Adaptive Associative Memory Cellular Neural Networks

Learning algorithms of bidirectional associative memory based on max-min criterion

A New Pattern Associative Memory Model for Image Recognition Based on Hebb Rules and Dot Product

An effective associative memory for pattern recognition

Pattern Storage and Hopfield Neural Associative Memory with Hidden Structure.

Pattern Recognition Based on Stability of Discrete Time Cellular Neural Networks