Abstract:The basal ganglia (BG) is a collection of nuclei located deep beneath the cerebral cortex that is involved in learning and selection of rewarded actions. Here, we analyzed BG mechanisms that enable these functions. We implemented a rate model of a BG-thalamo-cortical loop and simulated its performance in a standard action selection task. We have shown that potentiation of corticostriatal synapses enables learning of a rewarded option. However, these synapses became redundant later as direct connections between prefrontal and premotor cortices (PFC-PMC) were potentiated by Hebbian learning. After we switched the reward to the previously unrewarded option (reversal), the BG was again responsible for switching to the new option. Due to the potentiated direct cortical connections, the system was biased to the previously rewarded choice, and establishing the new choice required a greater number of trials. Guided by physiological research, we then modified our model to reproduce pathological states of mild Parkinson's and Huntington's diseases. We found that in the Parkinsonian state PMC activity levels become extremely variable, which is caused by oscillations arising in the BG-thalamo-cortical loop. The model reproduced severe impairment of learning and predicted that this is caused by these oscillations as well as a reduced reward prediction signal. In the Huntington state, the potentiation of the PFC-PMC connections produced better learning, but altered BG output disrupted expression of the rewarded choices. This resulted in random switching between rewarded and unrewarded choices resembling an exploratory phase that never ended. Along with other computational studies, our results further reconcile the apparent contradiction between the critical involvement of the BG in execution of previously learned actions and yet no impairment of these actions after BG output is ablated by lesions or deep brain stimulation. We predict that the cortico-BG-thalamo-cortical loop conforms to previously learned choice in healthy conditions, but impedes those choices in disease states.

Implementation of a Model of the Cortex Basal Ganglia Loop

A robot model of the basal ganglia: Behavior and intrinsic processing

Cortex Versus Basal Ganglia Design for Selection

Reward-driven adaptation of movements requires strong recurrent basal ganglia-cortical loops

Working memory and response selection: a computational account of interactions among cortico-basalganglio-thalamic loops

The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia

Dynamical model of salience gated working memory, action selection and reinforcement based on basal ganglia and dopamine feedback

[Cortico-basal ganglia circuits--parallel closed loops and convergent/divergent connections]

Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states

Integrated model of cerebellal supervised learning and basal ganglia's reinforcement learning for mobile robot behavioral decision-making

A computational model of inhibitory control in frontal cortex and basal ganglia

Modeling Basal Ganglia for Understanding Parkinsonian Reaching Movements

A model for integrating elementary neural functions into delayed-response behavior

Biomimetic race model of the loop between the superior colliculus and the basal ganglia: Subcortical selection of saccade targets

Action-modulated midbrain dopamine activity arises from distributed control policies

Basal ganglia network dynamics and function: Role of direct, indirect and hyper-direct pathways in action selection

From Cognitive to Habit Behavior During Navigation, Through Cortical-Basal Ganglia Loops

Multiple dynamic interactions from basal ganglia direct and indirect pathways mediate action selection

A Simplified Cerebellum-Based Model for Motor Control in Brain Based Devices.

A biologically constrained model of the whole basal ganglia addressing the paradoxes of connections and selection

Hierarchical control of goal-directed action in the cortical–basal ganglia network