Basal ganglia: Appreciating the 'value' of the GPe

Daniel L Albaugh,Aryn H Gittis

DOI: https://doi.org/10.1016/j.cub.2023.09.034

2023-10-23

Abstract:Reward predictions and prediction errors are encoded in the GPe in a cell type-specific manner. A newly discovered cell type, the Slow Pacemaker, robustly encodes reward value and generates prediction errors in a manner remarkably similar to midbrain dopamine neurons.

What problem does this paper attempt to address?

Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations

Michael A Farries,Thomas W Faust,Ali Mohebi,Joshua D Berke

DOI: https://doi.org/10.1016/j.cub.2023.08.042

2023-10-09

Abstract:Basal ganglia (BG) circuits help guide and invigorate actions using predictions of future rewards (values). Within the BG, the globus pallidus pars externa (GPe) may play an essential role in aggregating and distributing value information. We recorded from the GPe in unrestrained rats performing both Pavlovian and instrumental tasks to obtain rewards and distinguished neuronal subtypes by their firing properties across the wake/sleep cycle and optogenetic tagging. In both tasks, the parvalbumin-positive (PV+), faster-firing "prototypical" neurons showed strong, sustained modulation by value, unlike other subtypes, including the "arkypallidal" cells that project back to striatum. Furthermore, we discovered that a distinct minority (7%) of GP cells display slower, pacemaker-like firing and encode reward prediction errors (RPEs) almost identically to midbrain dopamine neurons. These cell-specific forms of GPe value representation help define the circuit mechanisms by which the BG contribute to motivation and reinforcement learning.
Properties of Neurons in External Globus Pallidus Can Support Optimal Action Selection

Rafal Bogacz,Eduardo Martin Moraud,Azzedine Abdi,Peter J. Magill,Jérôme Baufreton

DOI: https://doi.org/10.1371/journal.pcbi.1005004

2016-07-07

PLoS Computational Biology

Abstract:The external globus pallidus (GPe) is a key nucleus within basal ganglia circuits that are thought to be involved in action selection. A class of computational models assumes that, during action selection, the basal ganglia compute for all actions available in a given context the probabilities that they should be selected. These models suggest that a network of GPe and subthalamic nucleus (STN) neurons computes the normalization term in Bayes' equation. In order to perform such computation, the GPe needs to send feedback to the STN equal to a particular function of the activity of STN neurons. However, the complex form of this function makes it unlikely that individual GPe neurons, or even a single GPe cell type, could compute it. Here, we demonstrate how this function could be computed within a network containing two types of GABAergic GPe projection neuron, so-called 'prototypic' and 'arkypallidal' neurons, that have different response properties in vivo and distinct connections. We compare our model predictions with the experimentally-reported connectivity and input-output functions (f-I curves) of the two populations of GPe neurons. We show that, together, these dichotomous cell types fulfil the requirements necessary to compute the function needed for optimal action selection. We conclude that, by virtue of their distinct response properties and connectivities, a network of arkypallidal and prototypic GPe neurons comprises a neural substrate capable of supporting the computation of the posterior probabilities of actions.

biochemical research methods,mathematical & computational biology
Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states

Garrett Mulcahy,Brady Atwood,Alexey Kuznetsov

DOI: https://doi.org/10.1371/journal.pone.0228081

IF: 3.7

2020-02-10

PLoS ONE

Abstract:The basal ganglia (BG) is a collection of nuclei located deep beneath the cerebral cortex that is involved in learning and selection of rewarded actions. Here, we analyzed BG mechanisms that enable these functions. We implemented a rate model of a BG-thalamo-cortical loop and simulated its performance in a standard action selection task. We have shown that potentiation of corticostriatal synapses enables learning of a rewarded option. However, these synapses became redundant later as direct connections between prefrontal and premotor cortices (PFC-PMC) were potentiated by Hebbian learning. After we switched the reward to the previously unrewarded option (reversal), the BG was again responsible for switching to the new option. Due to the potentiated direct cortical connections, the system was biased to the previously rewarded choice, and establishing the new choice required a greater number of trials. Guided by physiological research, we then modified our model to reproduce pathological states of mild Parkinson's and Huntington's diseases. We found that in the Parkinsonian state PMC activity levels become extremely variable, which is caused by oscillations arising in the BG-thalamo-cortical loop. The model reproduced severe impairment of learning and predicted that this is caused by these oscillations as well as a reduced reward prediction signal. In the Huntington state, the potentiation of the PFC-PMC connections produced better learning, but altered BG output disrupted expression of the rewarded choices. This resulted in random switching between rewarded and unrewarded choices resembling an exploratory phase that never ended. Along with other computational studies, our results further reconcile the apparent contradiction between the critical involvement of the BG in execution of previously learned actions and yet no impairment of these actions after BG output is ablated by lesions or deep brain stimulation. We predict that the cortico-BG-thalamo-cortical loop conforms to previously learned choice in healthy conditions, but impedes those choices in disease states.
Organization of reward and movement signals in the basal ganglia and cerebellum

Noga Larry,Gil Zur,Mati Joshua

DOI: https://doi.org/10.1038/s41467-024-45921-9

IF: 16.6

2024-03-08

Nature Communications

Abstract:Abstract The basal ganglia and the cerebellum are major subcortical structures in the motor system. The basal ganglia have been cast as the reward center of the motor system, whereas the cerebellum is thought to be involved in adjusting sensorimotor parameters. Recent findings of reward signals in the cerebellum have challenged this dichotomous view. To compare the basal ganglia and the cerebellum directly, we recorded from oculomotor regions in both structures from the same monkeys. We partitioned the trial-by-trial variability of the neurons into reward and eye-movement signals to compare the coding across structures. Reward expectation and movement signals were the most pronounced in the output structure of the basal ganglia, intermediate in the cerebellum, and the smallest in the input structure of the basal ganglia. These findings suggest that reward and movement information is sharpened through the basal ganglia, resulting in a higher signal-to-noise ratio than in the cerebellum.

multidisciplinary sciences
Reward Bases: A simple mechanism for adaptive acquisition of multiple reward types

Beren Millidge,Yuhang Song,Armin Lak,Mark E. Walton,Rafal Bogacz

DOI: https://doi.org/10.1371/journal.pcbi.1012580

2024-11-24

PLoS Computational Biology

Abstract:Animals can adapt their preferences for different types of reward according to physiological state, such as hunger or thirst. To explain this ability, we employ a simple multi-objective reinforcement learning model that learns multiple values according to different reward dimensions such as food or water. We show that by weighting these learned values according to the current needs, behaviour may be flexibly adapted to present preferences. This model predicts that individual dopamine neurons should encode the errors associated with some reward dimensions more than with others. To provide a preliminary test of this prediction, we reanalysed a small dataset obtained from a single primate in an experiment which to our knowledge is the only published study where the responses of dopamine neurons to stimuli predicting distinct types of rewards were recorded. We observed that in addition to subjective economic value, dopamine neurons encode a gradient of reward dimensions; some neurons respond most to stimuli predicting food rewards while the others respond more to stimuli predicting fluids. We also proposed a possible implementation of the model in the basal ganglia network, and demonstrated how the striatal system can learn values in multiple dimensions, even when dopamine neurons encode mixtures of prediction error from different dimensions. Additionally, the model reproduces the instant generalisation to new physiological states seen in dopamine responses and in behaviour. Our results demonstrate how a simple neural circuit can flexibly guide behaviour according to animals' needs. Animals and humans can search for different resources depending on their needs. For example, when you are thirsty at work, you may go to a common room where hopefully coffee or water is available, while if you are hungry, you would rather go to a canteen. Such ability to seek different resources based on a physiological state is so fundamental to survival, that is present also in simple animals. This paper proposes how this ability could arise from a simple neural circuit that can be mapped on evolutionary older parts of the vertebrate brain, called the basal ganglia. The model suggests that this circuit learns the availability of different reward types, and then combines them according to the physiological state to control behaviour.

biochemical research methods,mathematical & computational biology
High-dimensional encoding of movement by single neurons in basal ganglia output

Gil Zur,Noga Larry,Matan Cain,Adi Lixenberg,Merav Yarkoni,Stuart Behling,Mati Joshua

DOI: https://doi.org/10.1016/j.isci.2024.110667

IF: 5.8

2024-08-03

iScience

Abstract:The substantia nigra pars reticulata (SNpr), an output structure of the basal ganglia, is hypothesized to gate movement execution. Previous studies in the eye movement system focusing mostly on saccades have reported that SNpr neurons are tonically active and either pause or increase their firing during movements, consistent with the gating role. We recorded activity in the SNpr of two monkeys during smooth pursuit and saccadic eye movements. SNpr neurons exhibited highly diverse reaction patterns during pursuit, including frequent increases and decreases in firing rate, uncorrelated responses in different movement directions and in reward conditions that resulted in the high dimensional activity of single neurons. These diverse temporal patterns surpassed those in other oculomotor areas in the medial-temporal cortex, frontal cortex, basal ganglia, and cerebellum. These findings suggest that temporal properties of the responses enrich the coding capacity of the basal ganglia output beyond gating or permitting movement.
Dopamine Prediction Errors and the Relativity of Value

Masamichi Sakagami,Shingo Tanaka

DOI: https://doi.org/10.1007/978-981-10-0207-6_9

2016-01-01

Abstract:Value, or reward prediction, is thought to be generated by neural circuits among the basal ganglia and midbrain dopamine area. Midbrain dopamine neurons calculate the difference between estimated and actual rewards and send the reward prediction error to the basal ganglia, particularly striatum where the value is coded. The value reflects the quality and quantity of reward, the timing of reward delivery, etc. In many cases, however, actually experienced values depend on the context in which rewards are given and cannot be explained merely by physicochemical components of reward.
The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia

Brian Lau,Tiago Monteiro,Joseph J Paton

DOI: https://doi.org/10.1016/j.conb.2017.08.015

IF: 7.07

2017-10-01

Current Opinion in Neurobiology

Abstract:Computational models of reinforcement learning (RL) strive to produce behavior that maximises reward, and thus allow software or robots to behave adaptively [1]. At the core of RL models is a learned mapping between 'states'-situations or contexts that an agent might encounter in the world-and actions. A wealth of physiological and anatomical data suggests that the basal ganglia (BG) is important for learning these mappings [2,3]. However, the computations performed by specific circuits are unclear. In this brief review, we highlight recent work concerning the anatomy and physiology of BG circuits that suggest refinements in our understanding of computations performed by the basal ganglia. We focus on one important component of basal ganglia circuitry, midbrain dopamine neurons, drawing attention to data that has been cast as supporting or departing from the RL framework that has inspired experiments in basal ganglia research over the past two decades. We suggest that the parallel circuit architecture of the BG might be expected to produce variability in the response properties of different dopamine neurons, and that variability in response profile may not reflect variable functions, but rather different arguments that serve as inputs to a common function: the computation of prediction error.

neurosciences
Basal ganglia output - entopeduncular nucleus - coding of contextual kinematics and reward in the freely moving mouse

Anil K Verma-Rodríguez,Josué O Ramírez-Jarquín,Román Rossi-Pool,Fatuel Tecuapetla

DOI: https://doi.org/10.1101/2024.03.22.586140

2024-10-19

Abstract:The entopeduncular nucleus (EPN) is often termed as one of the output nuclei of the basal ganglia owing to their highly convergent anatomy. The rodent EPN has been implicated in reward and value coding whereas the primate analogue internal Globus Pallidus has been found to be modulated by some movements and in some circumstances. In this study we sought to understand how the rodent EPN might be coding kinematic, reward, and difficulty parameters, particularly during locomotion. Furthermore, we aimed to understand the level of movement representation: whole-body or specific body parts. To this end, mice were trained in a freely moving two-alternative forced choice task with two periods of displacement (return and go trajectories) and performed electrophysiological recordings together with video-based tracking. We found 1) robust reward coding but not difficulty. 2) Spatio-temporal variables better explain EPN activity during movement compared to kinematic variables, while both types of variables were more robustly represented in reward-related movement. 3) Reward sensitive units encode kinematics similarly to reward insensitive ones. 4) Population dynamics that best account for differences between these two periods of movement can be explained by allocentric references like distance to reward port. 5) The representation of paw and licks is not mutually exclusive, discarding a somatotopic muscle-level representation of movement in the EPN. Our data suggest that EPN activity represents movements and reward in a complex way: highly multiplexed, influenced by the objective of the displacement, where trajectories that lead to reward better represent spatial and kinematic variables. Interestingly, there are intertwining representations of whole-body movement kinematics with single paw and licking variables. Further, reward sensitive units encode kinematics similarly to reward insensitive ones, challenging the notion of distinct pathways for reward and movement processing.

Neuroscience
Parallel Excitatory and Inhibitory Neural Circuit Pathways Underlie Reward-Based Phasic Neural Responses

Huanyuan Zhou,KongFatt Wong-Lin,Da-Hui Wang

DOI: https://doi.org/10.1155/2018/4356767

IF: 2.3

2018-01-01

Complexity

Abstract:Phasic activity of dopaminergic (DA) neurons in the ventral tegmental area or substantia nigra compacta (VTA/SNc) has been suggested to encode reward-prediction error signal for reinforcement learning. Recent studies have shown that the lateral habenula (LHb) neurons exhibit a similar response, but for nonrewarding or punishment signals. Hence, the transient signaling role of LHb neurons is opposite that of DA neurons and also that of several other brain nuclei such as the border region of the globus pallidus internal segment (GPb) and the rostral medial tegmentum (RMTg). Previous theoretical models have investigated the neural circuit mechanism underlying reward-based phasic activity of DA neurons, but the feasibility of a larger neural circuit model to account for the observed reward-based phasic activity in other brain nuclei such as the LHb has yet to be shown. Here, we propose a large-scale neural circuit model and show that parallel excitatory and inhibitory pathways underlie the learned neural responses across multiple brain regions. Specifically, the model can account for the phasic neural activity observed in the GPb, LHb, RMTg, and VTA/SNc. Based on sensitivity analysis, the model is found to be robust against changes in the overall neural connectivity strength. The model also predicts that striosomes play a key role in the phasic activity of VTA/SNc and LHb neurons by encoding previous and expected rewards. Taken together, our model identifies the important role of parallel neural circuit pathways in accounting for phasic activity across multiple brain areas during reward and punishment processing.

mathematics, interdisciplinary applications,multidisciplinary sciences
Rethinking dopamine as generalized prediction error

Matthew P. H. Gardner,Geoffrey Schoenbaum,Samuel J. Gershman

DOI: https://doi.org/10.1098/rspb.2018.1645

2018-11-21

Proceedings of the Royal Society B: Biological Sciences

Abstract:Midbrain dopamine neurons are commonly thought to report a reward prediction error (RPE), as hypothesized by reinforcement learning (RL) theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here, we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signalling errors in both sensory and reward predictions, dopamine supports a form of RL that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and RPEs, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.
Distinct dynamics and intrinsic properties in ventral tegmental area populations mediate reward association and motivation

Jordan E Elum,Eric R Szelenyi,Barbara Juarez,Alexandria D Murry,Grigory Loginov,Catalina A Zamorano,Pan Gao,Ginny Wu,Scott Ng-Evans,Xiangmin Xu,Sam A Golden,Larry S Zweifel

DOI: https://doi.org/10.1101/2024.02.05.578997

2024-02-07

Abstract:Ventral tegmental area (VTA) dopamine neurons regulate reward-related associative learning and reward-driven motivated behaviors, but how these processes are coordinated by distinct VTA neuronal subpopulations remains unresolved. Here we examine the neural correlates of reward-related prediction-error, action, cue, and outcome encoding as well as effort exertion and reward anticipation during reward-seeking behaviors. We compare the contribution of two primarily dopaminergic and largely non-overlapping VTA subpopulations, all VTA dopamine neurons, and VTA GABAergic neurons of the mouse midbrain to these processes. The dopamine subpopulation that projects to the nucleus accumbens (NAc) core preferentially encodes prediction-error and reward-predictive cues. In contrast, the dopamine subpopulation that projects to the NAc shell preferentially encodes goal-directed actions and reflects relative reward anticipation. VTA GABA neuron activity strongly contrasts VTA dopamine population activity and preferentially encodes reward outcome and retrieval. Electrophysiology, targeted optogenetics, and whole-brain input mapping reveal heterogeneity among VTA dopamine subpopulations. Our results demonstrate that VTA subpopulations carry distinct reward-related learning and motivation signals and reveal a striking pattern of functional heterogeneity among projection-defined VTA dopamine neuron populations.

Neuroscience
Believing in dopamine

Samuel J. Gershman,Naoshige Uchida

DOI: https://doi.org/10.1038/s41583-019-0220-7

IF: 38.755

2019-09-30

Nature Reviews Neuroscience

Abstract:Midbrain dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in various probabilistic computations, such as encoding uncertainty and controlling exploration. Here, we show how these different facets of dopamine signalling can be brought together under a common reinforcement learning framework. The key idea is that multiple sources of uncertainty impinge on reinforcement learning computations: uncertainty about the state of the environment, the parameters of the value function and the optimal action policy. Each of these sources plays a distinct role in the prefrontal cortex–basal ganglia circuit for reinforcement learning and is ultimately reflected in dopamine activity. The view that dopamine plays a central role in the encoding and updating of beliefs brings the classical prediction error theory into alignment with more recent theories of Bayesian reinforcement learning.

neurosciences
How cortico-basal ganglia-thalamic subnetworks can shift decision policies to maximize reward rate

Jyotika Bahuguna,Timothy V Verstynen,Jonathan Rubin

DOI: https://doi.org/10.1101/2024.05.21.595174

2024-05-22

Abstract:All mammals exhibit flexible decision policies that depend, at least in part, on the cortico-basal ganglia-thalamic (CBGT) pathways. Yet understanding how the complex connectivity, dynamics, and plasticity of CBGT circuits translates into experience-dependent shifts of decision policies represents a longstanding challenge in neuroscience. Here we used a computational approach to address this problem. Specifically, we simulated decisions driven by CBGT circuits under baseline, unrewarded conditions using a spiking neural network, and fit the resulting behavior to an evidence accumulation model. Using canonical correlation analysis, we then replicated the existence of three recently identified control ensembles (responsiveness, pliancy and choice) within CBGT circuits, with each ensemble mapping to a specific configuration of the evidence accumulation process. We subsequently simulated learning in a simple two-choice task with one optimal (i.e., rewarded) target. We find that value-based learning, via dopaminergic signals acting on cortico-striatal synapses, effectively manages the speed-accuracy tradeoff so as to increase reward rate over time. Within this process, learning-related changes in decision policy can be decomposed in terms of the contributions of each control ensemble, and these changes are driven by sequential reward prediction errors on individual trials. Our results provide a clear and simple mechanism for how dopaminergic plasticity shifts specific subnetworks within CBGT circuits so as to strategically modulate decision policies in order to maximize effective reward rate.

Neuroscience
Dopamine, Inference, and Uncertainty

Samuel J. Gershman

DOI: https://doi.org/10.1162/neco_a_01023

IF: 3.278

2017-12-01

Neural Computation

Abstract:The hypothesis that the phasic dopamine response reports a reward prediction error has become deeply entrenched. However, dopamine neurons exhibit several notable deviations from this hypothesis. A coherent explanation for these deviations can be obtained by analyzing the dopamine response in terms of Bayesian reinforcement learning. The key idea is that prediction errors are modulated by probabilistic beliefs about the relationship between cues and outcomes, updated through Bayesian inference. This account can explain dopamine responses to inferred value in sensory preconditioning, the effects of cue preexposure (latent inhibition), and adaptive coding of prediction errors when rewards vary across orders of magnitude. We further postulate that orbitofrontal cortex transforms the stimulus representation through recurrent dynamics, such that a simple error-driven learning rule operating on the transformed representation can implement the Bayesian reinforcement learning update.

computer science, artificial intelligence,neurosciences
Basal ganglia network dynamics and function: Role of direct, indirect and hyper-direct pathways in action selection

Jian Song,Hui Lin,Shenquan Liu

DOI: https://doi.org/10.1080/0954898X.2023.2173816

Network

Abstract:Basal ganglia (BG) are a widely recognized neural basis for action selection, but its decision-making mechanism is still a difficult problem for researchers. Therefore, we constructed a spiking neural network inspired by the BG anatomical data. Simulation experiments were based on the principle of dis-inhibition and our functional hypothesis within the BG: the direct pathway, the indirect pathway, and the hyper-direct pathway of the BG jointly implement the initiation execution and termination of motor programs. Firstly, we studied the dynamic process of action selection with the network, which contained intra-group competition and inter-group competition. Secondly, we focused on the effects of the stimulus intensity and the proportion of excitation and inhibition on the GPi/SNr. The results suggested that inhibition and excitation shape action selection. They also explained why the firing rate of GPi/SNr did not continue to increase in the action-selection experiment. Finally, we discussed the experimental results with the functional hypothesis. Uniquely, this paper summarized the decision-making neural mechanism of action selection based on the direct pathway, the indirect pathway, and the hyper-direct pathway within BG.
Mixed representations of choice and outcome by GABA/glutamate cotransmitting neurons in the entopeduncular nucleus

Julianna Locantore,Jesse White,Janet Berrios Wallace,Celia C Beron,Bernardo L Sabatini,Michael L Wallace

DOI: https://doi.org/10.1101/2024.06.07.597980

2024-06-08

Abstract:The basal ganglia (BG) are an evolutionarily conserved and phylogenetically old set of sub-cortical nuclei that guide action selection, evaluation, and reinforcement. The entopeduncular nucleus (EP) is a major BG output nucleus that contains a population of GABA/glutamate cotransmitting neurons (EPSst+) that specifically target the lateral habenula (LHb) and whose function in behavior remains mysterious. Here we use a probabilistic switching task that requires an animal to maintain flexible relationships between action selection and evaluation to examine when and how GABA/glutamate cotransmitting neurons contribute to behavior. We find that EPSst+ neurons are strongly engaged during this task and show bidirectional changes in activity during the choice and outcome periods of a trial. We then tested the effects of either permanently blocking cotransmission or modifying the GABA/glutamate ratio on behavior in well-trained animals. Neither manipulation produced detectable changes in behavior despite significant changes in synaptic transmission in the LHb, demonstrating that the outputs of these neurons are not required for on-going action-outcome updating in a probabilistic switching task.

Neuroscience
Functional Dissection of Basal Ganglia Inhibitory Inputs onto Substantia Nigra Dopaminergic Neurons

Rebekah C Evans,Emily L Twedell,Manhua Zhu,Jefferson Ascencio,Renshu Zhang,Zayd M Khaliq

DOI: https://doi.org/10.1016/j.celrep.2020.108156

2020-09-15

Abstract:Substantia nigra (SNc) dopaminergic neurons respond to aversive stimuli with inhibitory pauses in firing followed by transient rebound activation. We tested integration of inhibitory synaptic inputs onto SNc neurons from genetically defined populations in dorsal striatum (striosome and matrix) and external globus pallidus (GPe; parvalbumin- and Lhx6-positive), and examined their contribution to pause-rebound firing. Activation of striosome projections, which target "dendron bouquets" in the pars reticulata (SNr), consistently quiets firing and relief from striosome inhibition triggers rebound activity. Striosomal inhibitory postsynaptic currents (IPSCs) display a prominent GABA-B receptor-mediated component that strengthens the impact of SNr dendrite synapses on somatic excitability and enables rebounding. By contrast, GPe projections activate GABA-A receptors on the soma and proximal dendrites but do not result in rebounding. Lastly, optical mapping shows that dorsal striatum selectively inhibits the ventral population of SNc neurons, which are intrinsically capable of rebounding. Therefore, we define a distinct striatonigral circuit for generating dopamine rebound.
Dopamine, Updated: Reward Prediction Error and Beyond

Talia N. Lerner,Ashley L. Holloway,Jillian L. Seiler

DOI: https://doi.org/10.1016/j.conb.2020.10.012

IF: 7.07

2021-04-01

Current Opinion in Neurobiology

Abstract:<p>Dopamine neurons have been intensely studied for their roles in reinforcement learning. A dominant theory of how these neurons contribute to learning is through the encoding of a reward prediction error (RPE) signal. Recent advances in dopamine research have added nuance to RPE theory by incorporating the ideas of sensory prediction error, distributional encoding, and belief states. Further nuance is likely to be added shortly by convergent lines of research on dopamine neuron diversity. Finally, a major challenge is to reconcile RPE theory with other current theories of dopamine function to account for dopamine's role in movement, motivation, and goal-directed planning.</p>

neurosciences

Basal ganglia: Appreciating the 'value' of the GPe

Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations

Properties of Neurons in External Globus Pallidus Can Support Optimal Action Selection

Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states

Organization of reward and movement signals in the basal ganglia and cerebellum

Reward Bases: A simple mechanism for adaptive acquisition of multiple reward types

High-dimensional encoding of movement by single neurons in basal ganglia output

Dopamine Prediction Errors and the Relativity of Value

The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia

Basal ganglia output - entopeduncular nucleus - coding of contextual kinematics and reward in the freely moving mouse

Parallel Excitatory and Inhibitory Neural Circuit Pathways Underlie Reward-Based Phasic Neural Responses

Rethinking dopamine as generalized prediction error

Distinct dynamics and intrinsic properties in ventral tegmental area populations mediate reward association and motivation

Believing in dopamine

How cortico-basal ganglia-thalamic subnetworks can shift decision policies to maximize reward rate

Dopamine, Inference, and Uncertainty

Basal ganglia network dynamics and function: Role of direct, indirect and hyper-direct pathways in action selection

Mixed representations of choice and outcome by GABA/glutamate cotransmitting neurons in the entopeduncular nucleus

Functional Dissection of Basal Ganglia Inhibitory Inputs onto Substantia Nigra Dopaminergic Neurons

Dopamine, Updated: Reward Prediction Error and Beyond