Abstract:Previous reports have described that neural activities in midbrain dopamine areas are sensitive to unexpected reward delivery and omission. These activities are correlated with reward prediction error in reinforcement learning models, the difference between predicted reward values and the obtained reward outcome. These findings suggest that the reward prediction error signal in the brain updates reward prediction through stimulus-reward experiences. It remains unknown, however, how sensory processing of reward-predicting stimuli contributes to the computation of reward prediction error. To elucidate this issue, we examined the relation between stimulus discriminability of the reward-predicting stimuli and the reward prediction error signal in the brain using functional magnetic resonance imaging (fMRI). Before main experiments, subjects learned an association between the orientation of a perceptually salient (high-contrast) Gabor patch and a juice reward. The subjects were then presented with lower-contrast Gabor patch stimuli to predict a reward. We calculated the correlation between fMRI signals and reward prediction error in two reinforcement learning models: a model including the modulation of reward prediction by stimulus discriminability and a model excluding this modulation. Results showed that fMRI signals in the midbrain are more highly correlated with reward prediction error in the model that includes stimulus discriminability than in the model that excludes stimulus discriminability. No regions showed higher correlation with the model that excludes stimulus discriminability. Moreover, results show that the difference in correlation between the two models was significant from the first session of the experiment, suggesting that the reward computation in the midbrain was modulated based on stimulus discriminability before learning a new contingency between perceptually ambiguous stimuli and a reward. These results suggest that the human reward system can incorporate the level of the stimulus discriminability flexibly into reward computations by modulating previously acquired reward values for a typical stimulus.

Reward prediction error signal enhanced by striatum-amygdala interaction explains the acceleration of probabilistic reward learning by emotion

Dissociable neural signals for reward and emotion prediction errors

Stimulus-dependent adjustment of reward prediction error in the midbrain

Encoding Motivation Prediction Errors in the Human Dopaminergic Reward System

Reward association impairs recognition of incidentally encoded negative information: Electrophysiological evidence

Expectation to rewards modulates learning emotional words: Evidence from a hierarchical Bayesian model

Dopamine Error Signal to Actively Cope with Lack of Expected Reward

Rapid detection of neutral faces associated with emotional value

Mesolimbic dopamine encodes reward prediction errors independent of learning rates

Differential and temporally dynamic involvement of primate amygdala nuclei in face animacy and reward information processing

Human Substantia Nigra Neurons Encode Reward Expectations

The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning

Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning

Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning

Action prediction error: a value-free dopaminergic teaching signal that drives stable learning

A causal link between prediction errors, dopamine neurons and learning

Neural Representations of Post-Decision Accuracy and Reward Expectation in the Caudate Nucleus and Frontal Eye Field

The Role of the Striatum in Aversive Learning and Aversive Prediction Errors

Reward Prediction in Prefrontal Cortex and Striatum

Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time