Abstract:It is widely believed that memory storage depends on activity-dependent synaptic modifications. Classical studies of learning and memory in neural networks describe synaptic efficacy either as continuous or discrete. However, recent results suggest an intermediate scenario in which synaptic efficacy can be described by a continuous variable, but whose distribution is peaked around a small set of discrete values. Motivated by these results, we explored a model in which each synapse is described by a continuous variable that evolves in a potential with multiple minima. External inputs to the network can switch synapses from one potential well to another. Our analytical and numerical results show that this model can interpolate between models with discrete synapses which correspond to the deep potential limit, and models in which synapses evolve in a single quadratic potential. We find that the storage capacity of the network with double well synapses exhibits a power law dependence on the network size, rather than the logarithmic dependence observed in models with single well synapses. In addition, synapses with deeper potential wells lead to more robust information storage in the presence of noise. When memories are sparsely encoded, the scaling of the capacity with network size is similar to previously studied network models in the sparse coding limit. A long-lasting question in neuroscience is whether synaptic efficacies should be described as continuous variable or discrete variables. Recent experiments indicate that it is a combination of both: synaptic efficacy changes continuously, but its distribution peaks at several discrete values. In this study, we introduce a synapse model described by a double well potential, and investigate the memory properties of networks of neurons connected with such synapses. Our results show in networks with a bimodal weight distribution, the storage capacity depends on network size as a power law. In addition, we demonstrate that networks with such synapses store information more robustly in the presence of noise, compared to networks with synapses with a single well potential.

Estimation of Attraction Domain and Exponential Convergence Rate of Continuous Feedback Associative Memory and Its Applications

Estimation of domain of attraction and exponential convergence rate of Hopfield's continuous-time associative memory and its applications

Estimation of the domain of attraction and the convergence rate of a Hopfield associative memory and an application

Estimation on Domain of Attraction and Convergence Rate of Hopfield Continuous Feedback Neural Networks

Exponential Attraction Domain and Exponential Convergent Rate of Associative Memory Neural Networks with Delays

Memory Dynamics in Attractor Networks with Saliency Weights.

Analysis and Optimal Design of Continuous Neural Networks with Applications to Associative Memory

Exponential Stability of Continuous-Time and Discrete-Time Bidirectional Associative Memory Networks with Delays

Global Exponential Stability in Hybrid Bi-Directional Associative Memory Neural Networks with Time Delays

Convergence Analysis Of Hybrid Bi-Directional Associative Memory Neural Networks With Discrete Delays

Attractor neural networks with double well synapses

Dynamics Analysis and Analog Associative Memory of Networks with LT Neurons.

Efficient learning algorithm for associative memory neural network

Associative Memory Synthesis Based on Region Attractive Recurrent Neural Networks

A METHOD FOR CALCULATING THE RADIUS OF CONVERGENCE OF AN EQUILIBRIUM POINT OF A CONTINUOUS FEEDBACK NEURAL NETWORK

Non-equilibrium statistical mechanics of continuous attractors

A Novel Continuous-Time Neural Network for Realizing Associative Memory

A Synthesis Method for the Complex-Valued Associative Memory Constrained by the Attractive Domain

Two-Layer Feedback Neural Networks with Associative Memories

Design of Complex-valued Hopfield Associative Memory Based on Prespecified Attractive Domain

Coexistence of Cyclic Sequential Pattern Recognition and Associative Memory in Neural Networks by Attractor Mechanisms