Abstract:It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.

A recurrent neural network framework for flexible and adaptive decision making based on sequence learning

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Predicting human decision making in psychological tasks with recurrent neural networks.

Predicting human decision making in psychological tasks with recurrent neural networks

Coherent noise enables probabilistic sequence replay in spiking neuronal networks

Discovering Cognitive Strategies with Tiny Recurrent Neural Networks

Recurrent networks endowed with structural priors explain suboptimal animal behavior

A Spiking Neural Network System for Robust Sequence Recognition

Recurrent Network Models Of Sequence Generation And Memory

Flexible gating between subspaces in a neural network model of internally guided task switching

Exploration-exploitation mechanisms in recurrent neural networks and human learners in restless bandit problems

Spiking Recurrent Neural Networks Represent Task-Relevant Neural Sequences in Rule-Dependent Computation

Neuron-level prediction and noise can implement flexible reward-seeking behavior

Improving Learning Efficiency of Recurrent Neural Network Through Adjusting Weights of All Layers in a Biologically-Inspired Framework.

Incorporating structural plasticity into self-organization recurrent networks for sequence learning

Neural networks with motivation

Predictive learning by a burst-dependent learning rule

Episodic memory governs choices: An RNN-based reinforcement learning model for decision-making task

GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL

Category Learning in a Recurrent Neural Network with Reinforcement Learning.

ARLIF: A Flexible and Efficient Recurrent Neuronal Model for Sequential Tasks.