Abstract:Previous reports have described that neural activities in midbrain dopamine areas are sensitive to unexpected reward delivery and omission. These activities are correlated with reward prediction error in reinforcement learning models, the difference between predicted reward values and the obtained reward outcome. These findings suggest that the reward prediction error signal in the brain updates reward prediction through stimulus-reward experiences. It remains unknown, however, how sensory processing of reward-predicting stimuli contributes to the computation of reward prediction error. To elucidate this issue, we examined the relation between stimulus discriminability of the reward-predicting stimuli and the reward prediction error signal in the brain using functional magnetic resonance imaging (fMRI). Before main experiments, subjects learned an association between the orientation of a perceptually salient (high-contrast) Gabor patch and a juice reward. The subjects were then presented with lower-contrast Gabor patch stimuli to predict a reward. We calculated the correlation between fMRI signals and reward prediction error in two reinforcement learning models: a model including the modulation of reward prediction by stimulus discriminability and a model excluding this modulation. Results showed that fMRI signals in the midbrain are more highly correlated with reward prediction error in the model that includes stimulus discriminability than in the model that excludes stimulus discriminability. No regions showed higher correlation with the model that excludes stimulus discriminability. Moreover, results show that the difference in correlation between the two models was significant from the first session of the experiment, suggesting that the reward computation in the midbrain was modulated based on stimulus discriminability before learning a new contingency between perceptually ambiguous stimuli and a reward. These results suggest that the human reward system can incorporate the level of the stimulus discriminability flexibly into reward computations by modulating previously acquired reward values for a typical stimulus.

How Instructed Knowledge Modulates the Neural Systems of Reward Learning

Instructed Knowledge Shapes Feedback-Driven Aversive Learning in Striatum and Orbitofrontal Cortex, but Not the Amygdala.

Signals in Human Striatum Are Appropriate for Policy Update Rather Than Value Prediction

Policy adjustment in a dynamic economic game

Prefrontal cortex as a meta-reinforcement learning system

How instructed knowledge shapes adaptive learning

Nucleus accumbens dopamine release reflects Bayesian inference during instrumental learning

How cortico-basal ganglia-thalamic subnetworks can shift decision policies to maximize reward rate

Neural Dynamics of Reward-Induced Response Activation and Inhibition.

Frontal cortex electrophysiology in reward- and punishment-related feedback processing during advice-guided decision making: An interleaved EEG-DC stimulation study

Neural Representations of Post-Decision Accuracy and Reward Expectation in the Caudate Nucleus and Frontal Eye Field

Electrical brain activations in preadolescents during a probabilistic reward-learning task reflect cognitive processes and behavioral strategy.

Dopamine-independent effect of rewards on choices through hidden-state inference

Stimulus-dependent adjustment of reward prediction error in the midbrain

Reactivation of reward representations associated with the reinforcement of behavior

The Medial Prefrontal Cortex Shapes Dopamine Reward Prediction Errors under State Uncertainty

Hidden Reward: Affect and Its Prediction Errors as Windows Into Subjective Value

Encoding priors in the brain: a reinforcement learning model for mouse decision making

Reward modulates the association between sensory noise and brain activity during perceptual decision-making

Reward Prediction Error in Learning-Related Behaviors

Neural dynamics underlying the illusion of control during reward processing