Abstract:It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.

Learning and Unlearning: Bridging classification, memory and generative modeling in Recurrent Neural Networks

Current State and Future Directions for Learning in Biological Recurrent Neural Networks: A Perspective Piece

Demolition and Reinforcement of Memories in Spin-Glass-like Neural Networks

Supervised Generative Reconstruction: An Efficient Way To Flexibly Store and Recognize Patterns

Lifelong Learning Process: Self-Memory Supervising and Dynamically Growing Networks

Recurrent Network Models Of Sequence Generation And Memory

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

Continual learning with invertible generative models

Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks

Controlling Recurrent Neural Networks by Conceptors

Neocortical plasticity: an unsupervised cake but no free lunch

Continual Learning with Deep Artificial Neurons

Memory Networks: Towards Fully Biologically Plausible Learning

Generating new concepts with hybrid neuro-symbolic models

Recurrent Joint Embedding Predictive Architecture with Recurrent Forward Propagation Learning

Category Learning in a Recurrent Neural Network with Reinforcement Learning.

Continual Lifelong Learning with Neural Networks: A Review

Towards New Generation, Biologically Plausible Deep Neural Network Learning

Generative NeuroEvolution for Deep Learning

Triple-Memory Networks: A Brain-Inspired Method for Continual Learning

Deep Generative Dual Memory Network for Continual Learning