Abstract:Two major “categories” of unsupervised learning rules are used in artificial neural networks: (i) competitive learning, which is used in the adaptive resonance theory (ART), the self-organizing map, and the neocognitron; and (ii) Hebbian learning without lateral inhibition, which is used in the Hopfield network. Since the competitive learning is essentially Hebbian learning in the presence of lateral inhibition, the author attempts here to discuss general properties of these unsupervised learning rules in a unified paradigm. As a first effort, this paper presents analytical studies of a performance comparison between a competitive learning neural network (CLNN) and the Hopfield neural network (HNN). Specifically, it discusses their abilities as classifiers after they are trained with noisy patterns. First, the HNN is generalized to perform pattern classification in addition to its well-known capability for pattern completion. The Hopfield formulation of the Hebbian learning rule is generalized to allow the existence of noise in training patterns. It is shown that the performance of the generalized HNN as a classifier decreases as noise in training patterns increases. A parallel study is then carried out for a CLNN. First, a simple CLNN is developed with the same components used by the generalized HNN and features used in existing CLNNs. In contrast, this simple CLNN is shown to be robust with respect to noise in training patterns. These discussions suggest that the reason for this difference in performance between the two types of networks is that in the CLNN each synapse is devoted to only one memory, whereas in the HNN each synapse is responsible for many memories. It is concluded that competitive learning, which leads to localized memory, is superior to Hebbian learning without lateral inhibition, which leads to distributed memory, at tolerating noise in training patterns.

Robustness in Hopfield neural networks with biased memory patterns

Dreaming neural networks: Forgetting spurious memories and reinforcing pure ones

Prototype Analysis in Hopfield Networks with Hebbian Learning

Memory Dynamics in Attractor Networks with Saliency Weights.

Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivity

Capacity of the Hebbian-Hopfield network associative memory

Input-Driven Dynamics for Robust Memory Retrieval in Hopfield Networks

Distributed Memory and Localized Memory in Neural Networks

A generalized Hopfield model to store and retrieve mismatched memory patterns

Heteroclinic cycles in Hopfield networks

Dynamic Capacity Estimation in Hopfield Networks

Learning associative memories by error backpropagation.

Effect of Similarity Between Patterns in Associative Memory

Brain memory working. Optimal control behavior for improved Hopfield-like models

Large Associative Memory Problem in Neurobiology and Machine Learning

Noise Facilitation in Associative Memories of Exponential Capacity

Exploring mechanisms of Neural Robustness: probing the bridge between geometry and spectrum

Effect of spatial correlations on Hopfield Neural Network and Dense Associative Memories

A spectral approach to Hebbian-like neural networks

Optimizing working memory with heterogeneity of recurrent cortical excitation

Optimal modularity and memory capacity of neural reservoirs