Abstract:Dopamine (DA) signals originating from substantia nigra (SN) neurons are centrally involved in the regulation of motor and reward processing. DA signals behaviorally relevant events where reward outcomes differ from expectations (reward prediction errors, RPEs). RPEs play a crucial role in learning optimal courses of action and in determining response vigor when an agent expects rewards. Nevertheless, how reward expectations, crucial for RPE calculations, are conveyed to and represented in the dopaminergic system is not fully understood, especially in the human brain where the activity of DA neurons is difficult to study. One possibility, suggested by evidence from animal models, is that DA neurons explicitly encode reward expectations. Alternatively, they may receive RPE information directly from upstream brain regions. To address whether SN neuron activity directly reflects reward expectation information, we directly examined the encoding of reward expectation signals in human putative DA neurons by performing single-unit recordings from the SN of patients undergoing neurosurgery. Patients played a two-armed bandit decision-making task in which they attempted to maximize reward. We show that neuronal firing rates (FR) of putative DA neurons during the reward expectation period explicitly encode reward expectations. First, activity in these neurons was modulated by previous trial outcomes, such that FR were greater after positive outcomes than after neutral or negative outcome trials. Second, this increase in FR was associated with shorter reaction times, consistent with an invigorating effect of DA neuron activity during expectation. These results suggest that human DA neurons explicitly encode reward expectations, providing a neurophysiological substrate for a signal critical for reward learning.

Reactivation of reward representations associated with the reinforcement of behavior

State-dependent Online Reactivations for Different Learning Strategies in Foraging

Predictive Coding of Reward in the Hippocampus

Short-term Memory Traces for Action Bias in Human Reinforcement Learning

Reward Recalibrates Rule Representations in Human Amygdala and Hippocampus Intracranial Recordings

Dopamine release in human associative striatum during reversal learning

Reactivation of Learned Reward Association Reduces Retroactive Interference From New Reward Learning

Human Substantia Nigra Neurons Encode Reward Expectations

How Instructed Knowledge Modulates the Neural Systems of Reward Learning

Representation of reward feedback in primate auditory cortex

Prefrontal cortex as a meta-reinforcement learning system

Neural activity ramps in frontal cortex signal extended motivation during learning

Synaptic and Spiking Dynamics Underlying Reward Reversal in the Orbitofrontal Cortex.

Abstract reward and punishment representations in the human orbitofrontal cortex

Representation learning with reward prediction errors

Temporally organized representations of reward and risk in the human brain

Neural Dynamics of Reward-Induced Response Activation and Inhibition.

Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans

Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning.

Human dorsal striatal activity during choice discriminates reinforcement learning behavior from the gambler's fallacy

Reinforcement Learning Links Spontaneous Cortical Dopamine Impulses to Reward