Abstract:Numerous electrophysiological experiments have reported that the prefrontal cortex (PFC) is involved in the process of working memory. PFC neurons continue firing to maintain stimulus information in the delay period without external stimuli in working memory tasks. Further findings indicate that while the activity of single neurons exhibits strong temporal and spatial dynamics (heterogeneity), the activity of population neurons can encode spatiotemporal information of stimuli stably and reliably. From the perspective of neural networks, the computational mechanism underlying this phenomenon is not well demonstrated. The main purpose of this paper is to adopt a new strategy to explore the neural computation mechanism of working memory. We used reinforcement learning to train a recurrent neural network model to learn a spatial working memory task. The model is composed of a decision network and a baseline network. The decision network is responsible for updating strategies to make action choices, while the baseline network evaluates action choices to predict rewards. Simulated results demonstrate that the model can perform the spatial working memory task. The activity of the recurrent units has characteristics such as temporal dynamics and preferred direction selectivity, but their population activity encodes the stimulus information stably during the delay period in a low-dimensional subspace. These activity characteristics displayed by the model units are similar to those of PFC neurons observed in the same experiments. Meanwhile, as the network model continued learning the task, the temporal stability and spatial separability of the stimulus information encoded by the activity of model units in the low-dimensional subspace gradually strengthened, and the accuracy of the network's action choices also increased. In summary, this network model provides a new simulation method for spatial working memory tasks and a new perspective for understanding the characteristics of neuron activity in the PFC.

Reservoir Computing with a Small-World Network for Discriminating Two Sequential Stimuli.

Efficient Reinforcement Learning of a Reservoir Network Model of Parametric Working Memory Achieved with a Cluster Population Winner-Take-all Readout Mechanism.

Identification of Stimulus-Response Relations for Cultured Neuronal Networks Using Features of Multiple Temporal Resolution Levels

Versatile networks of simulated spiking neurons displaying winner-take-all behavior

Spatiotemporal Information Processing with a Reservoir Decision-making Network

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Stochastic Resonance Enhancing Detectability Of Weak Signal By Neuronal Networks Model For Receiver

Fast and Robust Image Segmentation by Small-World Neural Oscillator Networks

Learning Real-World Stimuli By Single-Spike Coding And Tempotron Rule

Memorizing morph patterns in small-world neuronal network

Temporal sequence learning in winner-take-all networks of spiking neurons demonstrated in a brain-based device

Rapid Feedforward Computation by Temporal Encoding and Learning with Spiking Neurons

State dependent computation using coupled recurrent networks

Convolutional Reservoir Computing for World Models

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

Memristive Reservoirs Learn to Learn

Efficient Online Learning for Networks of Two-Compartment Spiking Neurons

A Gaussian attractor network for memory and recognition with experience-dependent learning.

Sequential Memory with Temporal Predictive Coding

A working memory model based on recurrent neural networks using reinforcement learning