Abstract:Neural associative memories are single layer perceptrons with fast synaptic learning typically storing discrete associations between pairs of neural activity patterns. Previous works have analyzed the optimal networks under naive Bayes assumptions of independent pattern components and heteroassociation, where the task is to learn associations from input to output patterns. Here I study the optimal Bayesian associative network for auto-association where input and output layers are identical. In particular, I compare performance to different variants of approximate Bayesian learning rules, like the BCPNN (Bayesian Confidence Propagation Neural Network), and try to explain why sometimes the suboptimal learning rules achieve higher storage capacity than the (theoretically) optimal model. It turns out that performance can depend on subtle dependencies of input components violating the ``naive Bayes'' assumptions. This includes patterns with constant number of active units, iterative retrieval where patterns are repeatedly propagated through recurrent networks, and winners-take-all activation of the most probable units. Performance of all learning rules can improve significantly if they include a novel adaptive mechanism to estimate noise in iterative retrieval steps (ANE). The overall maximum storage capacity is achieved again by the Bayesian learning rule with ANE.

Optimum associative neural network utilizing maximum likelihood

Bidirectional Associative Memory Utilizing Maximum Likelihood

Optimal learning algorithm in bidirectional associative memories

Designing Bidirectional Associative Memories with Optimal Stability

A neuron-weighted learning algorithm and its hardware implementation in associative memories

Object Recognition and Tracking with Maximum Likelihood Bidirectional Associative Memory Networks

Weighted Learning of Bidirectional Associative Memories by Global Minimization

A Bidirectional Associative Memory Based on Optimal Linear Associative Memory

Learning algorithms of bidirectional associative memory based on max-min criterion

Neural auto-association with optimal Bayesian learning

Efficient learning algorithm for associative memory neural network

An Efficient Learning Algorithm for Fuzzy Bi-directional Associative Memory

Analysis and Optimal Design of Continuous Neural Networks with Applications to Associative Memory

Biological lateral inhibition and digital cellular neural network applied on associations memory

Dynamics Analysis and Analog Associative Memory of Networks with LT Neurons.

A robust learning algorithm for the extended AM neural network

A Novel Continuous-Time Neural Network for Realizing Associative Memory

A learning algorithm of multilaver dynamics associative neural network based on generalized hebb rule

Loop Neural-Network Model for Associative Memory

A Neural Network with a Single Recurrent Unit for Associative Memories Based on Linear Optimization

Learning associative memories by error backpropagation.