Abstract:It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.

Hierarchical categorization learning is associated with representational changes in the dorsal striatum and posterior frontal and parietal cortex

Category learning increases discriminability of relevant object dimensions in visual cortex

Learning Shapes Spatiotemporal Brain Patterns for Flexible Categorical Decisions

Learning Shapes the Representation of Visual Categories in the Aging Human Brain

Dynamic categorization rules alter representations in human visual cortex

Category Learning Selectively Enhances Representations of Boundary-Adjacent Exemplars in Early Visual Cortex

Dissociable changes in functional network topology underlie early category learning and development of automaticity

Category learning can alter perception and its neural correlates

Category Representation in Primary Visual Cortex after Visual Perceptual Learning

Categorical learning revealed in activity pattern of left fusiform cortex.

Learning Shapes the Representation of Behavioral Choice in the Human Brain

Neural Hierarchy of Color Categorization: from Prototype Encoding to Boundary Encoding.

How Deep is the Feature Analysis underlying Rapid Visual Categorization?

Neural mechanisms for learning hierarchical structures of information

Computational evidence for hierarchically structured reinforcement learning in humans

On the interaction of prior knowledge and stimulus structure in category learning

Flexible Coding for Categorical Decisions in the Human Brain.

Category Learning in a Recurrent Neural Network with Reinforcement Learning.

Learning Hierarchical Visual Representations in Deep Neural Networks Using Hierarchical Linguistic Labels

Object Categorization Based on Hierarchical Learning

Dissociable Roles of the Dorsolateral and Ventromedial Prefrontal Cortex in Human Categorization