Abstract:BACKGROUND: Optimal choices require an accurate neuronal representation of economic value. In economics, utility functions are mathematical representations of subjective value that can be constructed from choices under risk. Utility usually exhibits a nonlinear relationship to physical reward value that corresponds to risk attitudes and reflects the increasing or decreasing marginal utility obtained with each additional unit of reward. Accordingly, neuronal reward responses coding utility should robustly reflect this nonlinearity.RESULTS: In two monkeys, we measured utility as a function of physical reward value from meaningful choices under risk (that adhered to first- and second-order stochastic dominance). The resulting nonlinear utility functions predicted the certainty equivalents for new gambles, indicating that the functions' shapes were meaningful. The monkeys were risk seeking (convex utility function) for low reward and risk avoiding (concave utility function) with higher amounts. Critically, the dopamine prediction error responses at the time of reward itself reflected the nonlinear utility functions measured at the time of choices. In particular, the reward response magnitude depended on the first derivative of the utility function and thus reflected the marginal utility. Furthermore, dopamine responses recorded outside of the task reflected the marginal utility of unpredicted reward. Accordingly, these responses were sufficient to train reinforcement learning models to predict the behaviorally defined expected utility of gambles.CONCLUSIONS: These data suggest a neuronal manifestation of marginal utility in dopamine neurons and indicate a common neuronal basis for fundamental explanatory constructs in animal learning theory (prediction error) and economic decision theory (marginal utility).

Dopamine-independent effect of rewards on choices through hidden-state inference

Dopamine reports reward prediction errors, but does not update policy, during inference-guided choice

Dopamine prediction error responses integrate subjective value from different reward dimensions

Dopamine transients encode reward prediction errors independent of learning rates

Mesolimbic dopamine encodes reward prediction errors independent of learning rates

Dopamine reveals adaptive learning of actions representation

Dopamine reward prediction error signal codes the temporal evaluation of a perceptual decision report

Dopamine, Updated: Reward Prediction Error and Beyond

Dopamine, Inference, and Uncertainty

A dopamine mechanism for reward maximization

The Medial Prefrontal Cortex Shapes Dopamine Reward Prediction Errors under State Uncertainty

The Dopaminergic Midbrain Encodes the Expected Certainty about Desired Outcomes

A selective role for dopamine in stimulus-reward learning

Dopamine Reward Prediction Error Responses Reflect Marginal Utility

What is dopamine doing in model-based reinforcement learning?

Dopamine, Prediction Error and Beyond

Rethinking dopamine as generalized prediction error

Dopamine firing plays a dual role in coding reward prediction errors and signaling motivation in a working memory task