Abstract:The standard Hopfield model for associative neural networks accounts for biological Hebbian learning and acts as the harmonic oscillator for pattern recognition, however its maximal storage capacity is α∼0.14, far from the theoretical bound for symmetric networks, i.e. α=1. Inspired by sleeping and dreaming mechanisms in mammal brains, we propose an extension of this model displaying the standard on-line (awake) learning mechanism (that allows the storage of external information in terms of patterns) and an off-line (sleep) unlearning&consolidating mechanism (that allows spurious-pattern removal and pure-pattern reinforcement): this obtained daily prescription is able to saturate the theoretical bound α=1, remaining also extremely robust against thermal noise. The emergent neural and synaptic features are analyzed both analytically and numerically. In particular, beyond obtaining a phase diagram for neural dynamics, we focus on synaptic plasticity and we give explicit prescriptions on the temporal evolution of the synaptic matrix. We analytically prove that our algorithm makes the Hebbian kernel converge with high probability to the projection matrix built over the pure stored patterns. Furthermore, we obtain a sharp and explicit estimate for the "sleep rate" in order to ensure such a convergence. Finally, we run extensive numerical simulations (mainly Monte Carlo sampling) to check the approximations underlying the analytical investigations (e.g., we developed the whole theory at the so called replica-symmetric level, as standard in the Amit-Gutfreund-Sompolinsky reference framework) and possible finite-size effects, finding overall full agreement with the theory.

Pattern Storage and Hopfield Neural Associative Memory with Hidden Structure.

Storage Capacity of the Hopfield Network Associative Memory

A Hierarchically Organized Memory Model With Temporal Population Coding

A New Approach to the Design of Hopfield Associative Memory

A Hierarchical Organized Memory Model Using Spiking Neurons

Learning associative memories by error backpropagation.

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier.

Dense Associative Memory for Pattern Recognition

Long Sequence Hopfield Memory

Dense Associative Memory Through the Lens of Random Features

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

Long sequence Hopfield memory*

A Hopfield-like model with complementary encodings of memories

Large Associative Memory Problem in Neurobiology and Machine Learning

A hippocampus CA3 spiking neural network model for storage and retrieval of sequential memory

A Memristive Hopfield Network for Associative Memory

Associative memory realized by a reconfigurable memristive Hopfield neural network

Analysis of Hopfield Model as Associative Memory

Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models

Dreaming neural networks: Forgetting spurious memories and reinforcing pure ones

Simplicial Hopfield networks