Abstract:Learning to optimally predict rewards requires agents to account for fluctuations in reward value. Recent work suggests that individuals can efficiently learn about variable rewards through adaptation of the learning rate, and coding of prediction errors relative to reward variability. Such adaptive coding has been linked to midbrain dopamine neurons in nonhuman primates, and evidence in support for a similar role of the dopaminergic system in humans is emerging from fMRI data. Here, we sought to investigate the effect of dopaminergic perturbations on adaptive prediction error coding in humans, using a between-subject, placebo-controlled pharmacological fMRI study with a dopaminergic agonist (bromocriptine) and antagonist (sulpiride). Participants performed a previously validated task in which they predicted the magnitude of upcoming rewards drawn from distributions with varying SDs. After each prediction, participants received a reward, yielding trial-by-trial prediction errors. Under placebo, we replicated previous observations of adaptive coding in the midbrain and ventral striatum. Treatment with sulpiride attenuated adaptive coding in both midbrain and ventral striatum, and was associated with a decrease in performance, whereas bromocriptine did not have a significant impact. Although we observed no differential effect of SD on performance between the groups, computational modeling suggested decreased behavioral adaptation in the sulpiride group. These results suggest that normal dopaminergic function is critical for adaptive prediction error coding, a key property of the brain thought to facilitate efficient learning in variable environments. Crucially, these results also offer potential insights for understanding the impact of disrupted dopamine function in mental illness.SIGNIFICANCE STATEMENT To choose optimally, we have to learn what to expect. Humans dampen learning when there is a great deal of variability in reward outcome, and two brain regions that are modulated by the brain chemical dopamine are sensitive to reward variability. Here, we aimed to directly relate dopamine to learning about variable rewards, and the neural encoding of associated teaching signals. We perturbed dopamine in healthy individuals using dopaminergic medication and asked them to predict variable rewards while we made brain scans. Dopamine perturbations impaired learning and the neural encoding of reward variability, thus establishing a direct link between dopamine and adaptation to reward variability. These results aid our understanding of clinical conditions associated with dopaminergic dysfunction, such as psychosis.

Dopamine release in human associative striatum during reversal learning

Dopamine Release Plateau and Outcome Signals in Dorsal Striatum Contrast with Classic Reinforcement Learning Formulations

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

Defining the Neural Mechanisms of Probabilistic Reversal Learning Using Event-Related Functional Magnetic Resonance Imaging

Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans

Dopamine Release in the Human Striatum: Motor and Cognitive Tasks Revisited

Human dorsal striatal activity during choice discriminates reinforcement learning behavior from the gambler's fallacy

Dopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum

Dorsal and Ventral Striatal Dopamine D1 and D2 Receptors Differentially Modulate Distinct Phases of Serial Visual Reversal Learning.

Dopamine transients encode reward prediction errors independent of learning rates

Mesolimbic dopamine encodes reward prediction errors independent of learning rates

Role of Dopamine D2 Receptors in Human Reinforcement Learning.

A mismatch between striatal cholinergic pauses and dopaminergic reward prediction errors

Striatal dopamine reflects individual long-term learning trajectories

D2 dopamine receptor expression, reactivity to rewards, and reinforcement learning in a complex value-based decision-making task

Prediction Error During Retrospective Revaluation of Causal Associations in Humans: Fmri Evidence in Favor of an Associative Model of Learning

Reinforcement Learning Links Spontaneous Cortical Dopamine Impulses to Reward

Reactivation of reward representations associated with the reinforcement of behavior

Dopamine D2-like Receptor Stimulation Blocks Negative Feedback in Visual and Spatial Reversal Learning in the Rat: Behavioural and Computational Evidence

Stimulus-dependent adjustment of reward prediction error in the midbrain