Abstract:Previous reports have described that neural activities in midbrain dopamine areas are sensitive to unexpected reward delivery and omission. These activities are correlated with reward prediction error in reinforcement learning models, the difference between predicted reward values and the obtained reward outcome. These findings suggest that the reward prediction error signal in the brain updates reward prediction through stimulus-reward experiences. It remains unknown, however, how sensory processing of reward-predicting stimuli contributes to the computation of reward prediction error. To elucidate this issue, we examined the relation between stimulus discriminability of the reward-predicting stimuli and the reward prediction error signal in the brain using functional magnetic resonance imaging (fMRI). Before main experiments, subjects learned an association between the orientation of a perceptually salient (high-contrast) Gabor patch and a juice reward. The subjects were then presented with lower-contrast Gabor patch stimuli to predict a reward. We calculated the correlation between fMRI signals and reward prediction error in two reinforcement learning models: a model including the modulation of reward prediction by stimulus discriminability and a model excluding this modulation. Results showed that fMRI signals in the midbrain are more highly correlated with reward prediction error in the model that includes stimulus discriminability than in the model that excludes stimulus discriminability. No regions showed higher correlation with the model that excludes stimulus discriminability. Moreover, results show that the difference in correlation between the two models was significant from the first session of the experiment, suggesting that the reward computation in the midbrain was modulated based on stimulus discriminability before learning a new contingency between perceptually ambiguous stimuli and a reward. These results suggest that the human reward system can incorporate the level of the stimulus discriminability flexibly into reward computations by modulating previously acquired reward values for a typical stimulus.

Reward and punishment contingency shifting reveals distinct roles for VTA dopamine and GABA neurons in behavioral flexibility

Valence ambiguity dynamically shapes striatal dopamine heterogeneity

Mesencephalic projections to the nucleus accumbens shell modulate value updating during probabilistic reversal learning

Distinct dynamics and intrinsic properties in ventral tegmental area populations mediate reward association and motivation

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

The role of prospective contingency in the control of behavior and dopamine signals during associative learning

Mesostriatal dopamine is sensitive to changes in specific cue-reward contingencies

Ventral tegmental area dopamine neural activity switches simultaneously with rule representations in the prefrontal cortex and hippocampus

Dynamic regulation of corticostriatal glutamatergic synaptic expression during reversal learning in male mice

Value-Driven Adaptations of Mesolimbic Dopamine Release Are Governed by Both Model-Based and Model-Free Mechanisms

Functional diversity of dopamine axons in prefrontal cortex during classical conditioning

Optogenetic activation of mesencephalic projections to the nucleus accumbens shell impairs probabilistic reversal learning by disrupting learning from negative reinforcement

Reward Recalibrates Rule Representations in Human Amygdala and Hippocampus Intracranial Recordings

Characterization of Striatal Dopamine Projections Across Striatal Subregions in Behavioral Flexibility

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning

Stimulus-dependent adjustment of reward prediction error in the midbrain

The convergence of aversion and reward signals in individual neurons of the mice lateral habenula

Plasticity of Neuronal Dynamics in the Lateral Habenula for Cue-Punishment Associative Learning.

Synaptic and Spiking Dynamics Underlying Reward Reversal in the Orbitofrontal Cortex.

VTA excitatory neurons impact reward-driven behavior by modulating infralimbic cortical firing