Abstract:Numerous electrophysiological experiments have reported that the prefrontal cortex (PFC) is involved in the process of working memory. PFC neurons continue firing to maintain stimulus information in the delay period without external stimuli in working memory tasks. Further findings indicate that while the activity of single neurons exhibits strong temporal and spatial dynamics (heterogeneity), the activity of population neurons can encode spatiotemporal information of stimuli stably and reliably. From the perspective of neural networks, the computational mechanism underlying this phenomenon is not well demonstrated. The main purpose of this paper is to adopt a new strategy to explore the neural computation mechanism of working memory. We used reinforcement learning to train a recurrent neural network model to learn a spatial working memory task. The model is composed of a decision network and a baseline network. The decision network is responsible for updating strategies to make action choices, while the baseline network evaluates action choices to predict rewards. Simulated results demonstrate that the model can perform the spatial working memory task. The activity of the recurrent units has characteristics such as temporal dynamics and preferred direction selectivity, but their population activity encodes the stimulus information stably during the delay period in a low-dimensional subspace. These activity characteristics displayed by the model units are similar to those of PFC neurons observed in the same experiments. Meanwhile, as the network model continued learning the task, the temporal stability and spatial separability of the stimulus information encoded by the activity of model units in the low-dimensional subspace gradually strengthened, and the accuracy of the network's action choices also increased. In summary, this network model provides a new simulation method for spatial working memory tasks and a new perspective for understanding the characteristics of neuron activity in the PFC.

Recurrent Network Models Of Sequence Generation And Memory

Coherent noise enables probabilistic sequence replay in spiking neuronal networks

A hippocampus CA3 spiking neural network model for storage and retrieval of sequential memory

A Hierarchical Organized Memory Model Using Spiking Neurons

Sequential Memory: a Putative Neural and Synaptic Dynamical Mechanism.

Connected Cortical Recurrent Networks

A model for cortical activity sequences

A theory of sequence indexing and working memory in recurrent neural networks

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier.

A working memory model based on recurrent neural networks using reinforcement learning

A dynamic attractor network model of memory formation, reinforcement and forgetting

Neuronal Sequence Models for Bayesian Online Inference

Sequence learning, prediction, and replay in networks of spiking neurons

Sequence memory for prediction, inference and behaviour

Predictive Attractor Models

Computation with Sequences of Assemblies in a Model of the Brain

Episodic Memory Theory for the Mechanistic Interpretation of Recurrent Neural Networks

Coexistence of Cyclic Sequential Pattern Recognition and Associative Memory in Neural Networks by Attractor Mechanisms

Short-Term Memory Through Persistent Activity: Evolution of Self-Stopping and Self-Sustaining Activity in Spiking Neural Networks

Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes