Abstract:The basal ganglia (BG) is a collection of nuclei located deep beneath the cerebral cortex that is involved in learning and selection of rewarded actions. Here, we analyzed BG mechanisms that enable these functions. We implemented a rate model of a BG-thalamo-cortical loop and simulated its performance in a standard action selection task. We have shown that potentiation of corticostriatal synapses enables learning of a rewarded option. However, these synapses became redundant later as direct connections between prefrontal and premotor cortices (PFC-PMC) were potentiated by Hebbian learning. After we switched the reward to the previously unrewarded option (reversal), the BG was again responsible for switching to the new option. Due to the potentiated direct cortical connections, the system was biased to the previously rewarded choice, and establishing the new choice required a greater number of trials. Guided by physiological research, we then modified our model to reproduce pathological states of mild Parkinson's and Huntington's diseases. We found that in the Parkinsonian state PMC activity levels become extremely variable, which is caused by oscillations arising in the BG-thalamo-cortical loop. The model reproduced severe impairment of learning and predicted that this is caused by these oscillations as well as a reduced reward prediction signal. In the Huntington state, the potentiation of the PFC-PMC connections produced better learning, but altered BG output disrupted expression of the rewarded choices. This resulted in random switching between rewarded and unrewarded choices resembling an exploratory phase that never ended. Along with other computational studies, our results further reconcile the apparent contradiction between the critical involvement of the BG in execution of previously learned actions and yet no impairment of these actions after BG output is ablated by lesions or deep brain stimulation. We predict that the cortico-BG-thalamo-cortical loop conforms to previously learned choice in healthy conditions, but impedes those choices in disease states.

Dissociable roles of ventral pallidum neurons in the basal ganglia reinforcement learning network

Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations

Ventral pallidum encodes relative reward value earlier and more robustly than nucleus accumbens

Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states

The ventral pallidum: Subregion-specific functional anatomy and roles in motivated behaviors

The ventral striato-pallidal pathway mediates the effect of predictive learning on choice between goal-directed actions

Distinct dynamics and intrinsic properties in ventral tegmental area populations mediate reward association and motivation

Functional Dissection of Basal Ganglia Inhibitory Inputs onto Substantia Nigra Dopaminergic Neurons

Differential roles of ventral pallidum subregions during cocaine self-administration behaviors

Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types

Dissociable Functions of Reward Inference in the Lateral Prefrontal Cortex and the Striatum

A novel critic signal in identified midbrain dopaminergic neurons of mice training in operant tasks

Properties of Neurons in External Globus Pallidus Can Support Optimal Action Selection

A cerebro-cerebellar network for learning visuomotor associations

Mixed representations of choice and outcome by GABA/glutamate cotransmitting neurons in the entopeduncular nucleus

Mixed representations of choice direction and outcome by GABA/glutamate cotransmitting neurons in the entopeduncular nucleus

Single-cell sequencing of rodent ventral pallidum reveals diverse neuronal subtypes with non-canonical interregional continuity

A computational model explains and predicts substantia nigra pars reticulata responses to pallidal and striatal inputs

Activity of neurons in the ventral tegmental region of the behaving monkey.

Parvalbumin-expressing basal forebrain neurons mediate learning from negative experience

Dissociable roles of the ventral, medial and lateral striatum on the acquisition and performance of a complex visual stimulus-response habit