Abstract:BACKGROUND: Optimal choices require an accurate neuronal representation of economic value. In economics, utility functions are mathematical representations of subjective value that can be constructed from choices under risk. Utility usually exhibits a nonlinear relationship to physical reward value that corresponds to risk attitudes and reflects the increasing or decreasing marginal utility obtained with each additional unit of reward. Accordingly, neuronal reward responses coding utility should robustly reflect this nonlinearity.RESULTS: In two monkeys, we measured utility as a function of physical reward value from meaningful choices under risk (that adhered to first- and second-order stochastic dominance). The resulting nonlinear utility functions predicted the certainty equivalents for new gambles, indicating that the functions' shapes were meaningful. The monkeys were risk seeking (convex utility function) for low reward and risk avoiding (concave utility function) with higher amounts. Critically, the dopamine prediction error responses at the time of reward itself reflected the nonlinear utility functions measured at the time of choices. In particular, the reward response magnitude depended on the first derivative of the utility function and thus reflected the marginal utility. Furthermore, dopamine responses recorded outside of the task reflected the marginal utility of unpredicted reward. Accordingly, these responses were sufficient to train reinforcement learning models to predict the behaviorally defined expected utility of gambles.CONCLUSIONS: These data suggest a neuronal manifestation of marginal utility in dopamine neurons and indicate a common neuronal basis for fundamental explanatory constructs in animal learning theory (prediction error) and economic decision theory (marginal utility).

Hidden Reward: Affect and Its Prediction Errors as Windows Into Subjective Value

Dopamine prediction error responses integrate subjective value from different reward dimensions

Dopamine Reward Prediction Error Responses Reflect Marginal Utility

Dopamine, Prediction Error and Beyond

Representation of Anticipated Rewards and Punishments in the Human Brain

The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning

How Instructed Knowledge Modulates the Neural Systems of Reward Learning

Early Salience Signals Predict Interindividual Asymmetry in Decision Accuracy Across Rewarding and Punishing Contexts

Dopamine, Inference, and Uncertainty

Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning

Modeling Affect-based Intrinsic Rewards for Exploration and Learning

Signals in Human Striatum Are Appropriate for Policy Update Rather Than Value Prediction

Expectation to rewards modulates learning emotional words: Evidence from a hierarchical Bayesian model

Dopamine Prediction Errors and the Relativity of Value

On the Sensitivity of Reward Inference to Misspecified Human Models

Reward Prediction Error in Learning-Related Behaviors

Dissociable neural signals for reward and emotion prediction errors

Affective valence does not reflect progress prediction errors in perceptual decisions